Apr 16 16:45:06.394655 ip-10-0-140-49 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Apr 16 16:45:06.394748 ip-10-0-140-49 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Apr 16 16:45:06.394758 ip-10-0-140-49 systemd[1]: kubelet.service: Failed with result 'resources'. Apr 16 16:45:06.395061 ip-10-0-140-49 systemd[1]: Failed to start Kubernetes Kubelet. Apr 16 16:45:16.441787 ip-10-0-140-49 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Apr 16 16:45:16.441804 ip-10-0-140-49 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot f572ac1ba5084d629db8adf4d04a841f -- Apr 16 16:47:45.792458 ip-10-0-140-49 systemd[1]: Starting Kubernetes Kubelet... Apr 16 16:47:46.252073 ip-10-0-140-49 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Apr 16 16:47:46.252073 ip-10-0-140-49 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Apr 16 16:47:46.252073 ip-10-0-140-49 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Apr 16 16:47:46.252073 ip-10-0-140-49 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Apr 16 16:47:46.252073 ip-10-0-140-49 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Apr 16 16:47:46.253824 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.253740 2574 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Apr 16 16:47:46.256016 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256002 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 16 16:47:46.256051 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256017 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 16 16:47:46.256051 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256021 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 16 16:47:46.256051 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256024 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 16 16:47:46.256051 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256027 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 16 16:47:46.256051 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256031 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 16 16:47:46.256051 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256034 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 16 16:47:46.256051 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256037 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 16 16:47:46.256051 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256039 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 16 16:47:46.256051 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256042 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 16 16:47:46.256051 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256045 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 16 16:47:46.256051 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256048 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 16 16:47:46.256051 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256050 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 16 16:47:46.256051 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256053 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 16 16:47:46.256051 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256056 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 16 16:47:46.256051 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256058 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 16 16:47:46.256422 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256061 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 16 16:47:46.256422 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256064 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 16 16:47:46.256422 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256066 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 16 16:47:46.256422 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256069 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 16 16:47:46.256422 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256071 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 16 16:47:46.256422 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256073 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 16 16:47:46.256422 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256076 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 16 16:47:46.256422 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256078 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 16 16:47:46.256422 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256081 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 16 16:47:46.256422 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256083 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 16 16:47:46.256422 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256086 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 16 16:47:46.256422 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256088 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 16 16:47:46.256422 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256094 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 16 16:47:46.256422 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256096 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 16 16:47:46.256422 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256099 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 16 16:47:46.256422 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256101 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 16 16:47:46.256422 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256103 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 16 16:47:46.256422 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.256106 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 16 16:47:46.257695 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257686 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 16 16:47:46.257695 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257693 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 16 16:47:46.257695 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257696 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 16 16:47:46.257787 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257700 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 16 16:47:46.257787 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257703 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 16 16:47:46.257787 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257706 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 16 16:47:46.257787 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257709 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 16 16:47:46.257787 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257711 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 16 16:47:46.257787 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257714 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 16 16:47:46.257787 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257717 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 16 16:47:46.257787 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257719 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 16 16:47:46.257787 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257722 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 16 16:47:46.257787 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257725 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 16 16:47:46.257787 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257727 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 16 16:47:46.257787 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257730 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 16 16:47:46.257787 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257733 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 16 16:47:46.257787 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257735 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 16 16:47:46.257787 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257737 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 16 16:47:46.257787 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257740 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 16 16:47:46.257787 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257743 2574 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 16 16:47:46.257787 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257746 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 16 16:47:46.257787 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257748 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 16 16:47:46.257787 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257751 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 16 16:47:46.258315 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257753 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 16 16:47:46.258315 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257756 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 16 16:47:46.258315 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257758 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 16 16:47:46.258315 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257761 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 16 16:47:46.258315 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257764 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 16 16:47:46.258315 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257766 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 16 16:47:46.258315 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257768 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 16 16:47:46.258315 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257771 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 16 16:47:46.258315 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257773 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 16 16:47:46.258315 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257776 2574 feature_gate.go:328] unrecognized feature gate: Example2 Apr 16 16:47:46.258315 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257779 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 16 16:47:46.258315 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257783 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 16 16:47:46.258315 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257786 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 16 16:47:46.258315 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257790 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 16 16:47:46.258315 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257793 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 16 16:47:46.258315 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257797 2574 feature_gate.go:328] unrecognized feature gate: Example Apr 16 16:47:46.258315 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257799 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 16 16:47:46.258315 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257802 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 16 16:47:46.258315 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257805 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 16 16:47:46.258315 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257807 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 16 16:47:46.258791 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257810 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 16 16:47:46.258791 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257813 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 16 16:47:46.258791 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257817 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 16 16:47:46.258791 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257821 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 16 16:47:46.258791 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257823 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 16 16:47:46.258791 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257826 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 16 16:47:46.258791 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257829 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 16 16:47:46.258791 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257832 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 16 16:47:46.258791 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.257835 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 16 16:47:46.258791 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258196 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 16 16:47:46.258791 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258215 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 16 16:47:46.258791 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258218 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 16 16:47:46.258791 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258221 2574 feature_gate.go:328] unrecognized feature gate: Example2 Apr 16 16:47:46.258791 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258224 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 16 16:47:46.258791 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258227 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 16 16:47:46.258791 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258230 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 16 16:47:46.258791 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258233 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 16 16:47:46.258791 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258235 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 16 16:47:46.258791 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258238 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 16 16:47:46.259260 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258240 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 16 16:47:46.259260 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258243 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 16 16:47:46.259260 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258245 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 16 16:47:46.259260 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258248 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 16 16:47:46.259260 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258250 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 16 16:47:46.259260 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258253 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 16 16:47:46.259260 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258256 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 16 16:47:46.259260 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258259 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 16 16:47:46.259260 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258262 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 16 16:47:46.259260 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258264 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 16 16:47:46.259260 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258267 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 16 16:47:46.259260 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258270 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 16 16:47:46.259260 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258272 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 16 16:47:46.259260 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258275 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 16 16:47:46.259260 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258278 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 16 16:47:46.259260 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258280 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 16 16:47:46.259260 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258282 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 16 16:47:46.259260 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258285 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 16 16:47:46.259260 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258287 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 16 16:47:46.259720 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258290 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 16 16:47:46.259720 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258292 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 16 16:47:46.259720 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258294 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 16 16:47:46.259720 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258297 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 16 16:47:46.259720 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258299 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 16 16:47:46.259720 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258302 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 16 16:47:46.259720 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258304 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 16 16:47:46.259720 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258306 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 16 16:47:46.259720 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258309 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 16 16:47:46.259720 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258312 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 16 16:47:46.259720 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258314 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 16 16:47:46.259720 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258317 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 16 16:47:46.259720 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258319 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 16 16:47:46.259720 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258322 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 16 16:47:46.259720 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258324 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 16 16:47:46.259720 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258327 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 16 16:47:46.259720 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258330 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 16 16:47:46.259720 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258332 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 16 16:47:46.259720 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258334 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 16 16:47:46.259720 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258338 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 16 16:47:46.259720 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258340 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 16 16:47:46.260237 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258343 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 16 16:47:46.260237 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258345 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 16 16:47:46.260237 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258348 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 16 16:47:46.260237 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258351 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 16 16:47:46.260237 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258353 2574 feature_gate.go:328] unrecognized feature gate: Example Apr 16 16:47:46.260237 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258355 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 16 16:47:46.260237 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258358 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 16 16:47:46.260237 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258360 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 16 16:47:46.260237 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258363 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 16 16:47:46.260237 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258365 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 16 16:47:46.260237 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258367 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 16 16:47:46.260237 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258370 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 16 16:47:46.260237 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258373 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 16 16:47:46.260237 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258375 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 16 16:47:46.260237 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258378 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 16 16:47:46.260237 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258380 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 16 16:47:46.260237 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258383 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 16 16:47:46.260237 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258387 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 16 16:47:46.260237 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258391 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 16 16:47:46.260703 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258394 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 16 16:47:46.260703 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258397 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 16 16:47:46.260703 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258400 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 16 16:47:46.260703 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258402 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 16 16:47:46.260703 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258405 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 16 16:47:46.260703 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258407 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 16 16:47:46.260703 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258409 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 16 16:47:46.260703 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258412 2574 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 16 16:47:46.260703 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258414 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 16 16:47:46.260703 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258416 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 16 16:47:46.260703 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258419 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 16 16:47:46.260703 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258422 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 16 16:47:46.260703 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258425 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 16 16:47:46.260703 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258427 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 16 16:47:46.260703 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258431 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 16 16:47:46.260703 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258435 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 16 16:47:46.260703 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.258438 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 16 16:47:46.260703 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259772 2574 flags.go:64] FLAG: --address="0.0.0.0" Apr 16 16:47:46.260703 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259780 2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Apr 16 16:47:46.260703 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259786 2574 flags.go:64] FLAG: --anonymous-auth="true" Apr 16 16:47:46.261194 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259791 2574 flags.go:64] FLAG: --application-metrics-count-limit="100" Apr 16 16:47:46.261194 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259795 2574 flags.go:64] FLAG: --authentication-token-webhook="false" Apr 16 16:47:46.261194 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259799 2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Apr 16 16:47:46.261194 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259803 2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Apr 16 16:47:46.261194 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259807 2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Apr 16 16:47:46.261194 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259810 2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Apr 16 16:47:46.261194 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259813 2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Apr 16 16:47:46.261194 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259817 2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Apr 16 16:47:46.261194 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259820 2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Apr 16 16:47:46.261194 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259823 2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Apr 16 16:47:46.261194 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259826 2574 flags.go:64] FLAG: --cgroup-root="" Apr 16 16:47:46.261194 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259829 2574 flags.go:64] FLAG: --cgroups-per-qos="true" Apr 16 16:47:46.261194 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259831 2574 flags.go:64] FLAG: --client-ca-file="" Apr 16 16:47:46.261194 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259834 2574 flags.go:64] FLAG: --cloud-config="" Apr 16 16:47:46.261194 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259837 2574 flags.go:64] FLAG: --cloud-provider="external" Apr 16 16:47:46.261194 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259840 2574 flags.go:64] FLAG: --cluster-dns="[]" Apr 16 16:47:46.261194 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259844 2574 flags.go:64] FLAG: --cluster-domain="" Apr 16 16:47:46.261194 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259846 2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Apr 16 16:47:46.261194 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259849 2574 flags.go:64] FLAG: --config-dir="" Apr 16 16:47:46.261194 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259852 2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Apr 16 16:47:46.261194 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259855 2574 flags.go:64] FLAG: --container-log-max-files="5" Apr 16 16:47:46.261194 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259860 2574 flags.go:64] FLAG: --container-log-max-size="10Mi" Apr 16 16:47:46.261194 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259863 2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Apr 16 16:47:46.261194 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259867 2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Apr 16 16:47:46.261794 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259870 2574 flags.go:64] FLAG: --containerd-namespace="k8s.io" Apr 16 16:47:46.261794 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259873 2574 flags.go:64] FLAG: --contention-profiling="false" Apr 16 16:47:46.261794 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259876 2574 flags.go:64] FLAG: --cpu-cfs-quota="true" Apr 16 16:47:46.261794 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259879 2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Apr 16 16:47:46.261794 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259882 2574 flags.go:64] FLAG: --cpu-manager-policy="none" Apr 16 16:47:46.261794 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259885 2574 flags.go:64] FLAG: --cpu-manager-policy-options="" Apr 16 16:47:46.261794 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259890 2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Apr 16 16:47:46.261794 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259893 2574 flags.go:64] FLAG: --enable-controller-attach-detach="true" Apr 16 16:47:46.261794 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259896 2574 flags.go:64] FLAG: --enable-debugging-handlers="true" Apr 16 16:47:46.261794 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259898 2574 flags.go:64] FLAG: --enable-load-reader="false" Apr 16 16:47:46.261794 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259901 2574 flags.go:64] FLAG: --enable-server="true" Apr 16 16:47:46.261794 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259904 2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Apr 16 16:47:46.261794 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259908 2574 flags.go:64] FLAG: --event-burst="100" Apr 16 16:47:46.261794 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259911 2574 flags.go:64] FLAG: --event-qps="50" Apr 16 16:47:46.261794 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259914 2574 flags.go:64] FLAG: --event-storage-age-limit="default=0" Apr 16 16:47:46.261794 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259917 2574 flags.go:64] FLAG: --event-storage-event-limit="default=0" Apr 16 16:47:46.261794 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259920 2574 flags.go:64] FLAG: --eviction-hard="" Apr 16 16:47:46.261794 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259924 2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Apr 16 16:47:46.261794 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259927 2574 flags.go:64] FLAG: --eviction-minimum-reclaim="" Apr 16 16:47:46.261794 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259929 2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Apr 16 16:47:46.261794 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259932 2574 flags.go:64] FLAG: --eviction-soft="" Apr 16 16:47:46.261794 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259935 2574 flags.go:64] FLAG: --eviction-soft-grace-period="" Apr 16 16:47:46.261794 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259938 2574 flags.go:64] FLAG: --exit-on-lock-contention="false" Apr 16 16:47:46.261794 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259941 2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Apr 16 16:47:46.261794 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259944 2574 flags.go:64] FLAG: --experimental-mounter-path="" Apr 16 16:47:46.262406 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259946 2574 flags.go:64] FLAG: --fail-cgroupv1="false" Apr 16 16:47:46.262406 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259949 2574 flags.go:64] FLAG: --fail-swap-on="true" Apr 16 16:47:46.262406 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259952 2574 flags.go:64] FLAG: --feature-gates="" Apr 16 16:47:46.262406 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259956 2574 flags.go:64] FLAG: --file-check-frequency="20s" Apr 16 16:47:46.262406 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259959 2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Apr 16 16:47:46.262406 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259962 2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Apr 16 16:47:46.262406 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259965 2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Apr 16 16:47:46.262406 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259968 2574 flags.go:64] FLAG: --healthz-port="10248" Apr 16 16:47:46.262406 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259971 2574 flags.go:64] FLAG: --help="false" Apr 16 16:47:46.262406 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259973 2574 flags.go:64] FLAG: --hostname-override="ip-10-0-140-49.ec2.internal" Apr 16 16:47:46.262406 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259976 2574 flags.go:64] FLAG: --housekeeping-interval="10s" Apr 16 16:47:46.262406 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259979 2574 flags.go:64] FLAG: --http-check-frequency="20s" Apr 16 16:47:46.262406 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259982 2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Apr 16 16:47:46.262406 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259985 2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Apr 16 16:47:46.262406 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259988 2574 flags.go:64] FLAG: --image-gc-high-threshold="85" Apr 16 16:47:46.262406 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259991 2574 flags.go:64] FLAG: --image-gc-low-threshold="80" Apr 16 16:47:46.262406 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259994 2574 flags.go:64] FLAG: --image-service-endpoint="" Apr 16 16:47:46.262406 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259996 2574 flags.go:64] FLAG: --kernel-memcg-notification="false" Apr 16 16:47:46.262406 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.259999 2574 flags.go:64] FLAG: --kube-api-burst="100" Apr 16 16:47:46.262406 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260002 2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Apr 16 16:47:46.262406 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260005 2574 flags.go:64] FLAG: --kube-api-qps="50" Apr 16 16:47:46.262406 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260007 2574 flags.go:64] FLAG: --kube-reserved="" Apr 16 16:47:46.262406 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260010 2574 flags.go:64] FLAG: --kube-reserved-cgroup="" Apr 16 16:47:46.262406 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260013 2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Apr 16 16:47:46.262969 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260016 2574 flags.go:64] FLAG: --kubelet-cgroups="" Apr 16 16:47:46.262969 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260018 2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Apr 16 16:47:46.262969 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260021 2574 flags.go:64] FLAG: --lock-file="" Apr 16 16:47:46.262969 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260023 2574 flags.go:64] FLAG: --log-cadvisor-usage="false" Apr 16 16:47:46.262969 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260026 2574 flags.go:64] FLAG: --log-flush-frequency="5s" Apr 16 16:47:46.262969 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260029 2574 flags.go:64] FLAG: --log-json-info-buffer-size="0" Apr 16 16:47:46.262969 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260034 2574 flags.go:64] FLAG: --log-json-split-stream="false" Apr 16 16:47:46.262969 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260037 2574 flags.go:64] FLAG: --log-text-info-buffer-size="0" Apr 16 16:47:46.262969 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260039 2574 flags.go:64] FLAG: --log-text-split-stream="false" Apr 16 16:47:46.262969 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260042 2574 flags.go:64] FLAG: --logging-format="text" Apr 16 16:47:46.262969 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260045 2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Apr 16 16:47:46.262969 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260048 2574 flags.go:64] FLAG: --make-iptables-util-chains="true" Apr 16 16:47:46.262969 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260051 2574 flags.go:64] FLAG: --manifest-url="" Apr 16 16:47:46.262969 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260054 2574 flags.go:64] FLAG: --manifest-url-header="" Apr 16 16:47:46.262969 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260058 2574 flags.go:64] FLAG: --max-housekeeping-interval="15s" Apr 16 16:47:46.262969 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260061 2574 flags.go:64] FLAG: --max-open-files="1000000" Apr 16 16:47:46.262969 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260065 2574 flags.go:64] FLAG: --max-pods="110" Apr 16 16:47:46.262969 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260068 2574 flags.go:64] FLAG: --maximum-dead-containers="-1" Apr 16 16:47:46.262969 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260070 2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Apr 16 16:47:46.262969 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260073 2574 flags.go:64] FLAG: --memory-manager-policy="None" Apr 16 16:47:46.262969 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260076 2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Apr 16 16:47:46.262969 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260079 2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Apr 16 16:47:46.262969 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260081 2574 flags.go:64] FLAG: --node-ip="0.0.0.0" Apr 16 16:47:46.262969 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260085 2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Apr 16 16:47:46.262969 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260091 2574 flags.go:64] FLAG: --node-status-max-images="50" Apr 16 16:47:46.263612 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260094 2574 flags.go:64] FLAG: --node-status-update-frequency="10s" Apr 16 16:47:46.263612 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260097 2574 flags.go:64] FLAG: --oom-score-adj="-999" Apr 16 16:47:46.263612 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260100 2574 flags.go:64] FLAG: --pod-cidr="" Apr 16 16:47:46.263612 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260103 2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:dc76bab72f320de3d4105c90d73c4fb139c09e20ce0fa8dcbc0cb59920d27dec" Apr 16 16:47:46.263612 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260108 2574 flags.go:64] FLAG: --pod-manifest-path="" Apr 16 16:47:46.263612 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260111 2574 flags.go:64] FLAG: --pod-max-pids="-1" Apr 16 16:47:46.263612 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260114 2574 flags.go:64] FLAG: --pods-per-core="0" Apr 16 16:47:46.263612 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260117 2574 flags.go:64] FLAG: --port="10250" Apr 16 16:47:46.263612 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260120 2574 flags.go:64] FLAG: --protect-kernel-defaults="false" Apr 16 16:47:46.263612 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260123 2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0899e3337d2cd1c55" Apr 16 16:47:46.263612 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260126 2574 flags.go:64] FLAG: --qos-reserved="" Apr 16 16:47:46.263612 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260129 2574 flags.go:64] FLAG: --read-only-port="10255" Apr 16 16:47:46.263612 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260131 2574 flags.go:64] FLAG: --register-node="true" Apr 16 16:47:46.263612 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260134 2574 flags.go:64] FLAG: --register-schedulable="true" Apr 16 16:47:46.263612 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260137 2574 flags.go:64] FLAG: --register-with-taints="" Apr 16 16:47:46.263612 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260140 2574 flags.go:64] FLAG: --registry-burst="10" Apr 16 16:47:46.263612 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260143 2574 flags.go:64] FLAG: --registry-qps="5" Apr 16 16:47:46.263612 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260146 2574 flags.go:64] FLAG: --reserved-cpus="" Apr 16 16:47:46.263612 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260148 2574 flags.go:64] FLAG: --reserved-memory="" Apr 16 16:47:46.263612 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260152 2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Apr 16 16:47:46.263612 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260155 2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Apr 16 16:47:46.263612 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260157 2574 flags.go:64] FLAG: --rotate-certificates="false" Apr 16 16:47:46.263612 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260160 2574 flags.go:64] FLAG: --rotate-server-certificates="false" Apr 16 16:47:46.263612 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260163 2574 flags.go:64] FLAG: --runonce="false" Apr 16 16:47:46.263612 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260166 2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Apr 16 16:47:46.264215 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260169 2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Apr 16 16:47:46.264215 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260172 2574 flags.go:64] FLAG: --seccomp-default="false" Apr 16 16:47:46.264215 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260174 2574 flags.go:64] FLAG: --serialize-image-pulls="true" Apr 16 16:47:46.264215 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260177 2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Apr 16 16:47:46.264215 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260180 2574 flags.go:64] FLAG: --storage-driver-db="cadvisor" Apr 16 16:47:46.264215 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260183 2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Apr 16 16:47:46.264215 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260186 2574 flags.go:64] FLAG: --storage-driver-password="root" Apr 16 16:47:46.264215 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260189 2574 flags.go:64] FLAG: --storage-driver-secure="false" Apr 16 16:47:46.264215 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260191 2574 flags.go:64] FLAG: --storage-driver-table="stats" Apr 16 16:47:46.264215 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260194 2574 flags.go:64] FLAG: --storage-driver-user="root" Apr 16 16:47:46.264215 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260197 2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Apr 16 16:47:46.264215 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260210 2574 flags.go:64] FLAG: --sync-frequency="1m0s" Apr 16 16:47:46.264215 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260213 2574 flags.go:64] FLAG: --system-cgroups="" Apr 16 16:47:46.264215 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260216 2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Apr 16 16:47:46.264215 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260221 2574 flags.go:64] FLAG: --system-reserved-cgroup="" Apr 16 16:47:46.264215 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260224 2574 flags.go:64] FLAG: --tls-cert-file="" Apr 16 16:47:46.264215 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260226 2574 flags.go:64] FLAG: --tls-cipher-suites="[]" Apr 16 16:47:46.264215 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260230 2574 flags.go:64] FLAG: --tls-min-version="" Apr 16 16:47:46.264215 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260233 2574 flags.go:64] FLAG: --tls-private-key-file="" Apr 16 16:47:46.264215 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260235 2574 flags.go:64] FLAG: --topology-manager-policy="none" Apr 16 16:47:46.264215 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260238 2574 flags.go:64] FLAG: --topology-manager-policy-options="" Apr 16 16:47:46.264215 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260241 2574 flags.go:64] FLAG: --topology-manager-scope="container" Apr 16 16:47:46.264215 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260244 2574 flags.go:64] FLAG: --v="2" Apr 16 16:47:46.264215 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260249 2574 flags.go:64] FLAG: --version="false" Apr 16 16:47:46.264215 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260252 2574 flags.go:64] FLAG: --vmodule="" Apr 16 16:47:46.264834 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260256 2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Apr 16 16:47:46.264834 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.260259 2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Apr 16 16:47:46.264834 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260341 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 16 16:47:46.264834 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260345 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 16 16:47:46.264834 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260348 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 16 16:47:46.264834 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260350 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 16 16:47:46.264834 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260355 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 16 16:47:46.264834 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260358 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 16 16:47:46.264834 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260361 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 16 16:47:46.264834 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260364 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 16 16:47:46.264834 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260369 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 16 16:47:46.264834 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260371 2574 feature_gate.go:328] unrecognized feature gate: Example2 Apr 16 16:47:46.264834 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260374 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 16 16:47:46.264834 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260376 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 16 16:47:46.264834 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260380 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 16 16:47:46.264834 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260382 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 16 16:47:46.264834 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260385 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 16 16:47:46.264834 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260387 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 16 16:47:46.264834 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260390 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 16 16:47:46.264834 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260392 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 16 16:47:46.265426 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260394 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 16 16:47:46.265426 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260397 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 16 16:47:46.265426 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260399 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 16 16:47:46.265426 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260402 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 16 16:47:46.265426 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260404 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 16 16:47:46.265426 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260407 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 16 16:47:46.265426 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260409 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 16 16:47:46.265426 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260412 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 16 16:47:46.265426 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260415 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 16 16:47:46.265426 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260417 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 16 16:47:46.265426 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260419 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 16 16:47:46.265426 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260422 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 16 16:47:46.265426 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260429 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 16 16:47:46.265426 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260432 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 16 16:47:46.265426 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260435 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 16 16:47:46.265426 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260438 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 16 16:47:46.265426 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260440 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 16 16:47:46.265426 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260443 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 16 16:47:46.265426 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260450 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 16 16:47:46.265426 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260453 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 16 16:47:46.265941 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260455 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 16 16:47:46.265941 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260458 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 16 16:47:46.265941 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260462 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 16 16:47:46.265941 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260464 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 16 16:47:46.265941 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260467 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 16 16:47:46.265941 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260470 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 16 16:47:46.265941 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260472 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 16 16:47:46.265941 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260475 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 16 16:47:46.265941 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260477 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 16 16:47:46.265941 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260479 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 16 16:47:46.265941 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260482 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 16 16:47:46.265941 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260485 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 16 16:47:46.265941 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260487 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 16 16:47:46.265941 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260489 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 16 16:47:46.265941 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260492 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 16 16:47:46.265941 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260494 2574 feature_gate.go:328] unrecognized feature gate: Example Apr 16 16:47:46.265941 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260497 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 16 16:47:46.265941 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260499 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 16 16:47:46.265941 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260501 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 16 16:47:46.265941 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260504 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 16 16:47:46.266537 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260507 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 16 16:47:46.266537 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260509 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 16 16:47:46.266537 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260511 2574 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 16 16:47:46.266537 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260514 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 16 16:47:46.266537 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260517 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 16 16:47:46.266537 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260520 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 16 16:47:46.266537 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260523 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 16 16:47:46.266537 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260525 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 16 16:47:46.266537 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260527 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 16 16:47:46.266537 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260530 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 16 16:47:46.266537 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260532 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 16 16:47:46.266537 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260535 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 16 16:47:46.266537 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260537 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 16 16:47:46.266537 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260540 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 16 16:47:46.266537 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260543 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 16 16:47:46.266537 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260546 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 16 16:47:46.266537 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260548 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 16 16:47:46.266537 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260551 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 16 16:47:46.266537 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260553 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 16 16:47:46.267023 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260555 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 16 16:47:46.267023 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260559 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 16 16:47:46.267023 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260561 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 16 16:47:46.267023 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260564 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 16 16:47:46.267023 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260566 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 16 16:47:46.267023 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260569 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 16 16:47:46.267023 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260571 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 16 16:47:46.267023 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260574 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 16 16:47:46.267023 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.260576 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 16 16:47:46.267023 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.261214 2574 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Apr 16 16:47:46.268386 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.268369 2574 server.go:530] "Kubelet version" kubeletVersion="v1.33.9" Apr 16 16:47:46.268418 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.268388 2574 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Apr 16 16:47:46.268450 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268434 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 16 16:47:46.268450 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268440 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 16 16:47:46.268450 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268443 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 16 16:47:46.268450 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268446 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 16 16:47:46.268450 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268449 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 16 16:47:46.268579 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268454 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 16 16:47:46.268579 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268458 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 16 16:47:46.268579 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268461 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 16 16:47:46.268579 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268465 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 16 16:47:46.268579 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268468 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 16 16:47:46.268579 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268471 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 16 16:47:46.268579 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268474 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 16 16:47:46.268579 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268476 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 16 16:47:46.268579 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268479 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 16 16:47:46.268579 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268481 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 16 16:47:46.268579 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268484 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 16 16:47:46.268579 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268486 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 16 16:47:46.268579 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268489 2574 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 16 16:47:46.268579 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268492 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 16 16:47:46.268579 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268494 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 16 16:47:46.268579 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268497 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 16 16:47:46.268579 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268500 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 16 16:47:46.268579 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268502 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 16 16:47:46.268579 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268505 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 16 16:47:46.269048 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268508 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 16 16:47:46.269048 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268510 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 16 16:47:46.269048 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268513 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 16 16:47:46.269048 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268515 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 16 16:47:46.269048 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268518 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 16 16:47:46.269048 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268520 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 16 16:47:46.269048 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268523 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 16 16:47:46.269048 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268525 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 16 16:47:46.269048 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268528 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 16 16:47:46.269048 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268531 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 16 16:47:46.269048 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268533 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 16 16:47:46.269048 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268536 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 16 16:47:46.269048 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268538 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 16 16:47:46.269048 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268541 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 16 16:47:46.269048 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268544 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 16 16:47:46.269048 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268547 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 16 16:47:46.269048 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268550 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 16 16:47:46.269048 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268552 2574 feature_gate.go:328] unrecognized feature gate: Example2 Apr 16 16:47:46.269048 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268555 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 16 16:47:46.269048 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268558 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 16 16:47:46.269556 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268560 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 16 16:47:46.269556 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268563 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 16 16:47:46.269556 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268565 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 16 16:47:46.269556 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268568 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 16 16:47:46.269556 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268570 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 16 16:47:46.269556 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268573 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 16 16:47:46.269556 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268575 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 16 16:47:46.269556 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268578 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 16 16:47:46.269556 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268581 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 16 16:47:46.269556 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268583 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 16 16:47:46.269556 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268586 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 16 16:47:46.269556 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268589 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 16 16:47:46.269556 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268592 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 16 16:47:46.269556 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268595 2574 feature_gate.go:328] unrecognized feature gate: Example Apr 16 16:47:46.269556 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268597 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 16 16:47:46.269556 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268600 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 16 16:47:46.269556 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268603 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 16 16:47:46.269556 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268606 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 16 16:47:46.269556 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268610 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 16 16:47:46.269556 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268613 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 16 16:47:46.270037 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268615 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 16 16:47:46.270037 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268618 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 16 16:47:46.270037 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268620 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 16 16:47:46.270037 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268623 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 16 16:47:46.270037 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268625 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 16 16:47:46.270037 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268628 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 16 16:47:46.270037 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268630 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 16 16:47:46.270037 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268633 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 16 16:47:46.270037 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268635 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 16 16:47:46.270037 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268638 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 16 16:47:46.270037 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268640 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 16 16:47:46.270037 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268643 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 16 16:47:46.270037 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268645 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 16 16:47:46.270037 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268648 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 16 16:47:46.270037 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268650 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 16 16:47:46.270037 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268653 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 16 16:47:46.270037 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268655 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 16 16:47:46.270037 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268658 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 16 16:47:46.270037 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268660 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 16 16:47:46.270574 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268663 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 16 16:47:46.270574 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268665 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 16 16:47:46.270574 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268667 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 16 16:47:46.270574 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.268672 2574 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Apr 16 16:47:46.270574 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268765 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 16 16:47:46.270574 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268769 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 16 16:47:46.270574 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268772 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 16 16:47:46.270574 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268775 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 16 16:47:46.270574 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268777 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 16 16:47:46.270574 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268780 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 16 16:47:46.270574 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268783 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 16 16:47:46.270574 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268785 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 16 16:47:46.270574 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268788 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 16 16:47:46.270574 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268791 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 16 16:47:46.270574 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268794 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 16 16:47:46.270942 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268796 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 16 16:47:46.270942 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268799 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 16 16:47:46.270942 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268801 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 16 16:47:46.270942 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268803 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 16 16:47:46.270942 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268806 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 16 16:47:46.270942 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268809 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 16 16:47:46.270942 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268811 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 16 16:47:46.270942 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268814 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 16 16:47:46.270942 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268816 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 16 16:47:46.270942 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268838 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 16 16:47:46.270942 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268843 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 16 16:47:46.270942 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268846 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 16 16:47:46.270942 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268850 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 16 16:47:46.270942 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268852 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 16 16:47:46.270942 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268855 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 16 16:47:46.270942 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268858 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 16 16:47:46.270942 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268860 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 16 16:47:46.270942 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268863 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 16 16:47:46.270942 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268865 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 16 16:47:46.271418 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268867 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 16 16:47:46.271418 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268870 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 16 16:47:46.271418 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268873 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 16 16:47:46.271418 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268876 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 16 16:47:46.271418 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268879 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 16 16:47:46.271418 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268882 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 16 16:47:46.271418 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268884 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 16 16:47:46.271418 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268887 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 16 16:47:46.271418 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268889 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 16 16:47:46.271418 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268892 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 16 16:47:46.271418 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268894 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 16 16:47:46.271418 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268896 2574 feature_gate.go:328] unrecognized feature gate: Example2 Apr 16 16:47:46.271418 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268899 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 16 16:47:46.271418 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268901 2574 feature_gate.go:328] unrecognized feature gate: Example Apr 16 16:47:46.271418 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268904 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 16 16:47:46.271418 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268906 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 16 16:47:46.271418 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268909 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 16 16:47:46.271418 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268911 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 16 16:47:46.271418 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268914 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 16 16:47:46.271418 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268916 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 16 16:47:46.271895 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268919 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 16 16:47:46.271895 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268921 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 16 16:47:46.271895 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268924 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 16 16:47:46.271895 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268926 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 16 16:47:46.271895 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268928 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 16 16:47:46.271895 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268931 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 16 16:47:46.271895 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268933 2574 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 16 16:47:46.271895 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268935 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 16 16:47:46.271895 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268938 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 16 16:47:46.271895 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268940 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 16 16:47:46.271895 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268943 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 16 16:47:46.271895 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268945 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 16 16:47:46.271895 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268947 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 16 16:47:46.271895 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268950 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 16 16:47:46.271895 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268953 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 16 16:47:46.271895 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268955 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 16 16:47:46.271895 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268958 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 16 16:47:46.271895 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268960 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 16 16:47:46.271895 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268963 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 16 16:47:46.271895 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268965 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 16 16:47:46.272387 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268969 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 16 16:47:46.272387 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268971 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 16 16:47:46.272387 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268975 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 16 16:47:46.272387 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268978 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 16 16:47:46.272387 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268981 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 16 16:47:46.272387 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268983 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 16 16:47:46.272387 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268986 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 16 16:47:46.272387 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268989 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 16 16:47:46.272387 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268992 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 16 16:47:46.272387 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268994 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 16 16:47:46.272387 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268997 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 16 16:47:46.272387 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.268999 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 16 16:47:46.272387 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.269001 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 16 16:47:46.272387 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.269004 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 16 16:47:46.272387 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.269006 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 16 16:47:46.272387 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:46.269009 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 16 16:47:46.272768 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.269013 2574 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Apr 16 16:47:46.272768 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.270106 2574 server.go:962] "Client rotation is on, will bootstrap in background" Apr 16 16:47:46.272768 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.272761 2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Apr 16 16:47:46.273883 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.273872 2574 server.go:1019] "Starting client certificate rotation" Apr 16 16:47:46.273981 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.273966 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Apr 16 16:47:46.274017 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.274005 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Apr 16 16:47:46.303768 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.303753 2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Apr 16 16:47:46.306738 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.306713 2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Apr 16 16:47:46.323734 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.323716 2574 log.go:25] "Validated CRI v1 runtime API" Apr 16 16:47:46.329515 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.329497 2574 log.go:25] "Validated CRI v1 image API" Apr 16 16:47:46.331485 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.331469 2574 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Apr 16 16:47:46.335350 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.335332 2574 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9cbd8581-515e-4869-a7c6-b6ff9c3057ff:/dev/nvme0n1p4 e0d05f56-c480-42f0-be11-7520ebf1ab42:/dev/nvme0n1p3] Apr 16 16:47:46.335413 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.335349 2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Apr 16 16:47:46.341271 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.341158 2574 manager.go:217] Machine: {Timestamp:2026-04-16 16:47:46.338925379 +0000 UTC m=+0.420659604 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3210418 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e96396836f9cc400ef77b93accc87 SystemUUID:ec2e9639-6836-f9cc-400e-f77b93accc87 BootID:f572ac1b-a508-4d62-9db8-adf4d04a841f Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6098944 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:fd:49:01:b6:f7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:fd:49:01:b6:f7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6a:95:47:13:6f:2e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Apr 16 16:47:46.341271 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.341265 2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Apr 16 16:47:46.341385 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.341337 2574 manager.go:233] Version: {KernelVersion:5.14.0-570.104.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260401-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Apr 16 16:47:46.343743 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.343719 2574 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Apr 16 16:47:46.343865 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.343744 2574 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-49.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Apr 16 16:47:46.343944 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.343874 2574 topology_manager.go:138] "Creating topology manager with none policy" Apr 16 16:47:46.343944 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.343882 2574 container_manager_linux.go:306] "Creating device plugin manager" Apr 16 16:47:46.343944 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.343894 2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Apr 16 16:47:46.344932 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.344921 2574 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Apr 16 16:47:46.347026 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.347016 2574 state_mem.go:36] "Initialized new in-memory state store" Apr 16 16:47:46.347126 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.347117 2574 server.go:1267] "Using root directory" path="/var/lib/kubelet" Apr 16 16:47:46.349631 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.349621 2574 kubelet.go:491] "Attempting to sync node with API server" Apr 16 16:47:46.350154 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.350138 2574 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Apr 16 16:47:46.350195 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.350142 2574 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Apr 16 16:47:46.350195 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.350173 2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Apr 16 16:47:46.350195 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.350183 2574 kubelet.go:397] "Adding apiserver pod source" Apr 16 16:47:46.350195 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.350192 2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Apr 16 16:47:46.351492 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.351480 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Apr 16 16:47:46.351551 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.351502 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Apr 16 16:47:46.356440 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.356416 2574 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.10-2.rhaos4.20.gita4d0894.el9" apiVersion="v1" Apr 16 16:47:46.358504 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.358353 2574 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Apr 16 16:47:46.359861 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.359849 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Apr 16 16:47:46.359905 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.359864 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Apr 16 16:47:46.359905 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.359870 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Apr 16 16:47:46.359905 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.359875 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Apr 16 16:47:46.359905 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.359880 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Apr 16 16:47:46.359905 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.359886 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Apr 16 16:47:46.359905 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.359891 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Apr 16 16:47:46.359905 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.359896 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Apr 16 16:47:46.359905 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.359903 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Apr 16 16:47:46.359905 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.359908 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Apr 16 16:47:46.360138 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.359917 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Apr 16 16:47:46.360138 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.359925 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Apr 16 16:47:46.360893 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.360883 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Apr 16 16:47:46.360893 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.360893 2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Apr 16 16:47:46.364236 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:46.364215 2574 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-49.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Apr 16 16:47:46.364293 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.364247 2574 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-49.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Apr 16 16:47:46.364293 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:46.364250 2574 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Apr 16 16:47:46.364387 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.364377 2574 watchdog_linux.go:99] "Systemd watchdog is not enabled" Apr 16 16:47:46.364418 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.364410 2574 server.go:1295] "Started kubelet" Apr 16 16:47:46.364523 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.364489 2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Apr 16 16:47:46.364633 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.364506 2574 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Apr 16 16:47:46.364633 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.364577 2574 server_v1.go:47] "podresources" method="list" useActivePods=true Apr 16 16:47:46.365127 ip-10-0-140-49 systemd[1]: Started Kubernetes Kubelet. Apr 16 16:47:46.366529 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.366508 2574 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Apr 16 16:47:46.367573 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.367554 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8mkd7" Apr 16 16:47:46.368064 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.368053 2574 server.go:317] "Adding debug handlers to kubelet server" Apr 16 16:47:46.371243 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:46.370090 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-49.ec2.internal.18a6e43ff38c9134 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-49.ec2.internal,UID:ip-10-0-140-49.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-49.ec2.internal,},FirstTimestamp:2026-04-16 16:47:46.364387636 +0000 UTC m=+0.446121841,LastTimestamp:2026-04-16 16:47:46.364387636 +0000 UTC m=+0.446121841,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-49.ec2.internal,}" Apr 16 16:47:46.372906 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.372889 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Apr 16 16:47:46.373420 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.373406 2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Apr 16 16:47:46.374018 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.374000 2574 volume_manager.go:295] "The desired_state_of_world populator starts" Apr 16 16:47:46.374018 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.374019 2574 volume_manager.go:297] "Starting Kubelet Volume Manager" Apr 16 16:47:46.374136 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.374003 2574 factory.go:55] Registering systemd factory Apr 16 16:47:46.374136 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.374046 2574 factory.go:223] Registration of the systemd container factory successfully Apr 16 16:47:46.374136 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.373998 2574 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Apr 16 16:47:46.374136 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.374130 2574 reconstruct.go:97] "Volume reconstruction finished" Apr 16 16:47:46.374288 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.374138 2574 reconciler.go:26] "Reconciler: start to sync state" Apr 16 16:47:46.374288 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.374278 2574 factory.go:153] Registering CRI-O factory Apr 16 16:47:46.374288 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.374286 2574 factory.go:223] Registration of the crio container factory successfully Apr 16 16:47:46.374366 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:46.374288 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-49.ec2.internal\" not found" Apr 16 16:47:46.374366 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.374331 2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Apr 16 16:47:46.374366 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.374344 2574 factory.go:103] Registering Raw factory Apr 16 16:47:46.374366 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.374353 2574 manager.go:1196] Started watching for new ooms in manager Apr 16 16:47:46.375072 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:46.375045 2574 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Apr 16 16:47:46.375176 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.375155 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8mkd7" Apr 16 16:47:46.375274 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.375262 2574 manager.go:319] Starting recovery of all containers Apr 16 16:47:46.384118 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.383997 2574 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Apr 16 16:47:46.385609 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.385591 2574 manager.go:324] Recovery completed Apr 16 16:47:46.386988 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:46.386967 2574 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-140-49.ec2.internal\" not found" node="ip-10-0-140-49.ec2.internal" Apr 16 16:47:46.390104 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.390092 2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 16 16:47:46.393620 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.393608 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-49.ec2.internal" event="NodeHasSufficientMemory" Apr 16 16:47:46.393673 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.393631 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-49.ec2.internal" event="NodeHasNoDiskPressure" Apr 16 16:47:46.393673 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.393643 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-49.ec2.internal" event="NodeHasSufficientPID" Apr 16 16:47:46.394499 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.394484 2574 cpu_manager.go:222] "Starting CPU manager" policy="none" Apr 16 16:47:46.394542 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.394498 2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Apr 16 16:47:46.394542 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.394513 2574 state_mem.go:36] "Initialized new in-memory state store" Apr 16 16:47:46.396830 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.396816 2574 policy_none.go:49] "None policy: Start" Apr 16 16:47:46.396898 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.396835 2574 memory_manager.go:186] "Starting memorymanager" policy="None" Apr 16 16:47:46.396898 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.396849 2574 state_mem.go:35] "Initializing new in-memory state store" Apr 16 16:47:46.427738 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.427724 2574 manager.go:341] "Starting Device Plugin manager" Apr 16 16:47:46.443962 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:46.427761 2574 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Apr 16 16:47:46.443962 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.427774 2574 server.go:85] "Starting device plugin registration server" Apr 16 16:47:46.443962 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.427945 2574 eviction_manager.go:189] "Eviction manager: starting control loop" Apr 16 16:47:46.443962 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.427958 2574 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Apr 16 16:47:46.443962 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.428031 2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Apr 16 16:47:46.443962 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.428116 2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Apr 16 16:47:46.443962 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.428123 2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Apr 16 16:47:46.443962 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:46.428966 2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Apr 16 16:47:46.443962 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:46.429001 2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-49.ec2.internal\" not found" Apr 16 16:47:46.505705 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.505650 2574 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Apr 16 16:47:46.506727 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.506709 2574 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Apr 16 16:47:46.506783 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.506745 2574 status_manager.go:230] "Starting to sync pod status with apiserver" Apr 16 16:47:46.506783 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.506767 2574 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Apr 16 16:47:46.506783 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.506776 2574 kubelet.go:2451] "Starting kubelet main sync loop" Apr 16 16:47:46.506900 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:46.506816 2574 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Apr 16 16:47:46.511211 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.511177 2574 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Apr 16 16:47:46.528409 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.528394 2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 16 16:47:46.529145 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.529130 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-49.ec2.internal" event="NodeHasSufficientMemory" Apr 16 16:47:46.529236 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.529162 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-49.ec2.internal" event="NodeHasNoDiskPressure" Apr 16 16:47:46.529236 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.529174 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-49.ec2.internal" event="NodeHasSufficientPID" Apr 16 16:47:46.529236 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.529193 2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-49.ec2.internal" Apr 16 16:47:46.537240 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.537227 2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-49.ec2.internal" Apr 16 16:47:46.537288 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:46.537246 2574 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-49.ec2.internal\": node \"ip-10-0-140-49.ec2.internal\" not found" Apr 16 16:47:46.556419 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:46.556400 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-49.ec2.internal\" not found" Apr 16 16:47:46.606888 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.606860 2574 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-49.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-49.ec2.internal"] Apr 16 16:47:46.606953 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.606926 2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 16 16:47:46.608229 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.608214 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-49.ec2.internal" event="NodeHasSufficientMemory" Apr 16 16:47:46.608319 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.608239 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-49.ec2.internal" event="NodeHasNoDiskPressure" Apr 16 16:47:46.608319 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.608248 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-49.ec2.internal" event="NodeHasSufficientPID" Apr 16 16:47:46.609509 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.609497 2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 16 16:47:46.609645 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.609631 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-49.ec2.internal" Apr 16 16:47:46.609699 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.609665 2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 16 16:47:46.610103 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.610090 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-49.ec2.internal" event="NodeHasSufficientMemory" Apr 16 16:47:46.610174 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.610101 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-49.ec2.internal" event="NodeHasSufficientMemory" Apr 16 16:47:46.610174 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.610114 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-49.ec2.internal" event="NodeHasNoDiskPressure" Apr 16 16:47:46.610174 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.610122 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-49.ec2.internal" event="NodeHasNoDiskPressure" Apr 16 16:47:46.610174 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.610127 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-49.ec2.internal" event="NodeHasSufficientPID" Apr 16 16:47:46.610174 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.610138 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-49.ec2.internal" event="NodeHasSufficientPID" Apr 16 16:47:46.612266 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.612252 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-49.ec2.internal" Apr 16 16:47:46.612331 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.612276 2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 16 16:47:46.612865 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.612846 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-49.ec2.internal" event="NodeHasSufficientMemory" Apr 16 16:47:46.612865 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.612874 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-49.ec2.internal" event="NodeHasNoDiskPressure" Apr 16 16:47:46.612972 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.612887 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-49.ec2.internal" event="NodeHasSufficientPID" Apr 16 16:47:46.637731 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:46.637712 2574 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-49.ec2.internal\" not found" node="ip-10-0-140-49.ec2.internal" Apr 16 16:47:46.641912 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:46.641898 2574 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-49.ec2.internal\" not found" node="ip-10-0-140-49.ec2.internal" Apr 16 16:47:46.657491 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:46.657464 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-49.ec2.internal\" not found" Apr 16 16:47:46.675614 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.675584 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c0e6e3b61a3e8cfc6394e27b3b2ac0f3-config\") pod \"kube-apiserver-proxy-ip-10-0-140-49.ec2.internal\" (UID: \"c0e6e3b61a3e8cfc6394e27b3b2ac0f3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-49.ec2.internal" Apr 16 16:47:46.675614 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.675614 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/985f218434988cfff775e487e766fd2a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-49.ec2.internal\" (UID: \"985f218434988cfff775e487e766fd2a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-49.ec2.internal" Apr 16 16:47:46.675769 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.675631 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/985f218434988cfff775e487e766fd2a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-49.ec2.internal\" (UID: \"985f218434988cfff775e487e766fd2a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-49.ec2.internal" Apr 16 16:47:46.758164 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:46.758088 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-49.ec2.internal\" not found" Apr 16 16:47:46.776434 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.776414 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/985f218434988cfff775e487e766fd2a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-49.ec2.internal\" (UID: \"985f218434988cfff775e487e766fd2a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-49.ec2.internal" Apr 16 16:47:46.776531 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.776434 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/985f218434988cfff775e487e766fd2a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-49.ec2.internal\" (UID: \"985f218434988cfff775e487e766fd2a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-49.ec2.internal" Apr 16 16:47:46.776531 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.776449 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/985f218434988cfff775e487e766fd2a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-49.ec2.internal\" (UID: \"985f218434988cfff775e487e766fd2a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-49.ec2.internal" Apr 16 16:47:46.776531 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.776465 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c0e6e3b61a3e8cfc6394e27b3b2ac0f3-config\") pod \"kube-apiserver-proxy-ip-10-0-140-49.ec2.internal\" (UID: \"c0e6e3b61a3e8cfc6394e27b3b2ac0f3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-49.ec2.internal" Apr 16 16:47:46.776531 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.776494 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c0e6e3b61a3e8cfc6394e27b3b2ac0f3-config\") pod \"kube-apiserver-proxy-ip-10-0-140-49.ec2.internal\" (UID: \"c0e6e3b61a3e8cfc6394e27b3b2ac0f3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-49.ec2.internal" Apr 16 16:47:46.776531 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.776498 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/985f218434988cfff775e487e766fd2a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-49.ec2.internal\" (UID: \"985f218434988cfff775e487e766fd2a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-49.ec2.internal" Apr 16 16:47:46.858748 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:46.858728 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-49.ec2.internal\" not found" Apr 16 16:47:46.941260 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.941234 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-49.ec2.internal" Apr 16 16:47:46.944290 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:46.944253 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-49.ec2.internal" Apr 16 16:47:46.958796 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:46.958768 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-49.ec2.internal\" not found" Apr 16 16:47:47.059364 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:47.059300 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-49.ec2.internal\" not found" Apr 16 16:47:47.159750 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:47.159726 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-49.ec2.internal\" not found" Apr 16 16:47:47.260372 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:47.260350 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-49.ec2.internal\" not found" Apr 16 16:47:47.273519 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:47.273501 2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Apr 16 16:47:47.273664 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:47.273644 2574 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Apr 16 16:47:47.273699 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:47.273653 2574 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Apr 16 16:47:47.361260 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:47.361219 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-49.ec2.internal\" not found" Apr 16 16:47:47.373779 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:47.373761 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Apr 16 16:47:47.376869 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:47.376840 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-04-15 16:42:46 +0000 UTC" deadline="2028-01-07 21:50:53.399889746 +0000 UTC" Apr 16 16:47:47.377007 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:47.376869 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15149h3m6.023023483s" Apr 16 16:47:47.388135 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:47.388117 2574 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Apr 16 16:47:47.418043 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:47.418022 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5djsq" Apr 16 16:47:47.426959 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:47.426942 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5djsq" Apr 16 16:47:47.461445 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:47.461427 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-49.ec2.internal\" not found" Apr 16 16:47:47.468858 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:47.468836 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod985f218434988cfff775e487e766fd2a.slice/crio-0ffd07e4bf7a2e3ba3df5f232aedd7d9f5f354689022875f9aa7c158eca2d7e5 WatchSource:0}: Error finding container 0ffd07e4bf7a2e3ba3df5f232aedd7d9f5f354689022875f9aa7c158eca2d7e5: Status 404 returned error can't find the container with id 0ffd07e4bf7a2e3ba3df5f232aedd7d9f5f354689022875f9aa7c158eca2d7e5 Apr 16 16:47:47.471388 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:47.471367 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0e6e3b61a3e8cfc6394e27b3b2ac0f3.slice/crio-262aa023685e4c7f3be26857c6752d958d541f6c152ebe8f8f182b4c534fb600 WatchSource:0}: Error finding container 262aa023685e4c7f3be26857c6752d958d541f6c152ebe8f8f182b4c534fb600: Status 404 returned error can't find the container with id 262aa023685e4c7f3be26857c6752d958d541f6c152ebe8f8f182b4c534fb600 Apr 16 16:47:47.473292 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:47.473279 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 16 16:47:47.509321 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:47.509284 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-49.ec2.internal" event={"ID":"c0e6e3b61a3e8cfc6394e27b3b2ac0f3","Type":"ContainerStarted","Data":"262aa023685e4c7f3be26857c6752d958d541f6c152ebe8f8f182b4c534fb600"} Apr 16 16:47:47.510048 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:47.510033 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-49.ec2.internal" event={"ID":"985f218434988cfff775e487e766fd2a","Type":"ContainerStarted","Data":"0ffd07e4bf7a2e3ba3df5f232aedd7d9f5f354689022875f9aa7c158eca2d7e5"} Apr 16 16:47:47.535043 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:47.535021 2574 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Apr 16 16:47:47.562153 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:47.562134 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-49.ec2.internal\" not found" Apr 16 16:47:47.662650 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:47.662594 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-49.ec2.internal\" not found" Apr 16 16:47:47.763102 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:47.763084 2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-140-49.ec2.internal\" not found" Apr 16 16:47:47.859209 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:47.859187 2574 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Apr 16 16:47:47.874273 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:47.874253 2574 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-49.ec2.internal" Apr 16 16:47:47.881493 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:47.881466 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Apr 16 16:47:47.882510 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:47.882493 2574 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-49.ec2.internal" Apr 16 16:47:47.892110 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:47.892069 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Apr 16 16:47:48.322259 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.322229 2574 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Apr 16 16:47:48.351320 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.351299 2574 apiserver.go:52] "Watching apiserver" Apr 16 16:47:48.361012 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.360990 2574 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Apr 16 16:47:48.362918 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.362889 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-4k9v9","openshift-multus/multus-additional-cni-plugins-kfh7g","openshift-multus/network-metrics-daemon-kfxzj","openshift-ovn-kubernetes/ovnkube-node-hfbd8","kube-system/kube-apiserver-proxy-ip-10-0-140-49.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xn9cg","openshift-dns/node-resolver-5zftw","openshift-network-diagnostics/network-check-target-6kmtd","openshift-network-operator/iptables-alerter-66wx7","kube-system/konnectivity-agent-wtvc6","openshift-cluster-node-tuning-operator/tuned-2m2qc","openshift-image-registry/node-ca-mx85v","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-49.ec2.internal"] Apr 16 16:47:48.364446 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.364414 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5zftw" Apr 16 16:47:48.365671 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.365648 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kfh7g" Apr 16 16:47:48.366725 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.366702 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:47:48.366859 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:48.366814 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfxzj" podUID="d2114257-06f1-4070-ad8d-afce484cc205" Apr 16 16:47:48.367308 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.367288 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vkzzw\"" Apr 16 16:47:48.367409 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.367331 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Apr 16 16:47:48.367409 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.367293 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Apr 16 16:47:48.368109 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.368055 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.368737 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.368323 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Apr 16 16:47:48.368737 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.368345 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Apr 16 16:47:48.368737 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.368493 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Apr 16 16:47:48.368983 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.368770 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vbwx8\"" Apr 16 16:47:48.368983 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.368823 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Apr 16 16:47:48.368983 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.368894 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Apr 16 16:47:48.369508 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.369486 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xn9cg" Apr 16 16:47:48.370874 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.370857 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.371236 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.371075 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-p5pmh\"" Apr 16 16:47:48.371236 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.371122 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Apr 16 16:47:48.371236 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.370859 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Apr 16 16:47:48.371236 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.371173 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Apr 16 16:47:48.371538 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.371517 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Apr 16 16:47:48.371921 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.371897 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Apr 16 16:47:48.372026 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.371998 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Apr 16 16:47:48.372611 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.372594 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Apr 16 16:47:48.372695 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.372610 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Apr 16 16:47:48.372745 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.372692 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sf5tz\"" Apr 16 16:47:48.374696 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.373585 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Apr 16 16:47:48.374696 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.373905 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Apr 16 16:47:48.374696 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.374662 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qg6jz\"" Apr 16 16:47:48.375781 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.375666 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kmtd" Apr 16 16:47:48.375781 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.375749 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-66wx7" Apr 16 16:47:48.375781 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:48.375752 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kmtd" podUID="1d844889-4f54-47ae-8c62-03b86abb6d39" Apr 16 16:47:48.378110 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.377761 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wtvc6" Apr 16 16:47:48.378226 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.378160 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Apr 16 16:47:48.378375 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.378350 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Apr 16 16:47:48.378460 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.378418 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wlmjm\"" Apr 16 16:47:48.378666 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.378648 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Apr 16 16:47:48.379225 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.379196 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.380577 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.380552 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mx85v" Apr 16 16:47:48.380988 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.380648 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Apr 16 16:47:48.380988 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.380700 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Apr 16 16:47:48.380988 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.380890 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9cs84\"" Apr 16 16:47:48.381897 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.381722 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Apr 16 16:47:48.381897 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.381767 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Apr 16 16:47:48.381897 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.381848 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2c9dm\"" Apr 16 16:47:48.383138 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.383119 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-hnwcx\"" Apr 16 16:47:48.383230 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.383123 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Apr 16 16:47:48.383291 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.383127 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Apr 16 16:47:48.383402 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.383384 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Apr 16 16:47:48.385143 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.385120 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs\") pod \"network-metrics-daemon-kfxzj\" (UID: \"d2114257-06f1-4070-ad8d-afce484cc205\") " pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:47:48.385238 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.385152 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqpn2\" (UniqueName: \"kubernetes.io/projected/a63483e4-ee18-41f8-b36c-45b8cb789405-kube-api-access-kqpn2\") pod \"iptables-alerter-66wx7\" (UID: \"a63483e4-ee18-41f8-b36c-45b8cb789405\") " pod="openshift-network-operator/iptables-alerter-66wx7" Apr 16 16:47:48.385238 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.385177 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/62b3fae0-34f0-4dab-871f-12838879bf70-registration-dir\") pod \"aws-ebs-csi-driver-node-xn9cg\" (UID: \"62b3fae0-34f0-4dab-871f-12838879bf70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xn9cg" Apr 16 16:47:48.385238 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.385195 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/62b3fae0-34f0-4dab-871f-12838879bf70-etc-selinux\") pod \"aws-ebs-csi-driver-node-xn9cg\" (UID: \"62b3fae0-34f0-4dab-871f-12838879bf70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xn9cg" Apr 16 16:47:48.385238 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.385233 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-multus-cni-dir\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.385438 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.385255 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-multus-daemon-config\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.385438 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.385279 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fa3397b8-3244-4c47-bf40-735b4b386fb5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kfh7g\" (UID: \"fa3397b8-3244-4c47-bf40-735b4b386fb5\") " pod="openshift-multus/multus-additional-cni-plugins-kfh7g" Apr 16 16:47:48.385438 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.385341 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a63483e4-ee18-41f8-b36c-45b8cb789405-iptables-alerter-script\") pod \"iptables-alerter-66wx7\" (UID: \"a63483e4-ee18-41f8-b36c-45b8cb789405\") " pod="openshift-network-operator/iptables-alerter-66wx7" Apr 16 16:47:48.385438 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.385371 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-run-systemd\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.385438 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.385392 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-ovnkube-config\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.385438 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.385414 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-ovn-node-metrics-cert\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.385695 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.385457 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/62b3fae0-34f0-4dab-871f-12838879bf70-device-dir\") pod \"aws-ebs-csi-driver-node-xn9cg\" (UID: \"62b3fae0-34f0-4dab-871f-12838879bf70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xn9cg" Apr 16 16:47:48.385695 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.385492 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-host-var-lib-cni-multus\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.385695 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.385520 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-host-var-lib-kubelet\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.385695 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.385544 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-etc-openvswitch\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.385695 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.385567 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-host-run-multus-certs\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.385695 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.385591 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjtgp\" (UniqueName: \"kubernetes.io/projected/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-kube-api-access-sjtgp\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.385695 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.385638 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fa3397b8-3244-4c47-bf40-735b4b386fb5-os-release\") pod \"multus-additional-cni-plugins-kfh7g\" (UID: \"fa3397b8-3244-4c47-bf40-735b4b386fb5\") " pod="openshift-multus/multus-additional-cni-plugins-kfh7g" Apr 16 16:47:48.385695 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.385674 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fa3397b8-3244-4c47-bf40-735b4b386fb5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kfh7g\" (UID: \"fa3397b8-3244-4c47-bf40-735b4b386fb5\") " pod="openshift-multus/multus-additional-cni-plugins-kfh7g" Apr 16 16:47:48.385975 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.385725 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnln2\" (UniqueName: \"kubernetes.io/projected/fa3397b8-3244-4c47-bf40-735b4b386fb5-kube-api-access-cnln2\") pod \"multus-additional-cni-plugins-kfh7g\" (UID: \"fa3397b8-3244-4c47-bf40-735b4b386fb5\") " pod="openshift-multus/multus-additional-cni-plugins-kfh7g" Apr 16 16:47:48.385975 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.385754 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-host-cni-bin\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.385975 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.385778 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-cnibin\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.385975 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.385802 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8eb081ed-d37b-42a9-a3c9-8549bcb2da0d-hosts-file\") pod \"node-resolver-5zftw\" (UID: \"8eb081ed-d37b-42a9-a3c9-8549bcb2da0d\") " pod="openshift-dns/node-resolver-5zftw" Apr 16 16:47:48.385975 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.385824 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-run-ovn\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.385975 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.385848 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-cni-binary-copy\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.385975 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.385874 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-multus-socket-dir-parent\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.385975 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.385897 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-multus-conf-dir\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.385975 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.385922 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8eb081ed-d37b-42a9-a3c9-8549bcb2da0d-tmp-dir\") pod \"node-resolver-5zftw\" (UID: \"8eb081ed-d37b-42a9-a3c9-8549bcb2da0d\") " pod="openshift-dns/node-resolver-5zftw" Apr 16 16:47:48.385975 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.385967 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-host-kubelet\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.386388 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386003 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-systemd-units\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.386388 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386038 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-host-run-netns\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.386388 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386066 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-var-lib-openvswitch\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.386388 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386091 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-host-run-k8s-cni-cncf-io\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.386388 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386116 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfq92\" (UniqueName: \"kubernetes.io/projected/8eb081ed-d37b-42a9-a3c9-8549bcb2da0d-kube-api-access-hfq92\") pod \"node-resolver-5zftw\" (UID: \"8eb081ed-d37b-42a9-a3c9-8549bcb2da0d\") " pod="openshift-dns/node-resolver-5zftw" Apr 16 16:47:48.386388 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386141 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-host-cni-netd\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.386388 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386166 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-system-cni-dir\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.386388 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386190 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-node-log\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.386388 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386252 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-etc-kubernetes\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.386388 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386278 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8srq7\" (UniqueName: \"kubernetes.io/projected/1d844889-4f54-47ae-8c62-03b86abb6d39-kube-api-access-8srq7\") pod \"network-check-target-6kmtd\" (UID: \"1d844889-4f54-47ae-8c62-03b86abb6d39\") " pod="openshift-network-diagnostics/network-check-target-6kmtd" Apr 16 16:47:48.386388 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386303 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fa3397b8-3244-4c47-bf40-735b4b386fb5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kfh7g\" (UID: \"fa3397b8-3244-4c47-bf40-735b4b386fb5\") " pod="openshift-multus/multus-additional-cni-plugins-kfh7g" Apr 16 16:47:48.386388 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386328 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-env-overrides\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.386388 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386352 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-ovnkube-script-lib\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.386388 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386386 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/62b3fae0-34f0-4dab-871f-12838879bf70-socket-dir\") pod \"aws-ebs-csi-driver-node-xn9cg\" (UID: \"62b3fae0-34f0-4dab-871f-12838879bf70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xn9cg" Apr 16 16:47:48.386941 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386410 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/62b3fae0-34f0-4dab-871f-12838879bf70-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xn9cg\" (UID: \"62b3fae0-34f0-4dab-871f-12838879bf70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xn9cg" Apr 16 16:47:48.386941 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386436 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/62b3fae0-34f0-4dab-871f-12838879bf70-sys-fs\") pod \"aws-ebs-csi-driver-node-xn9cg\" (UID: \"62b3fae0-34f0-4dab-871f-12838879bf70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xn9cg" Apr 16 16:47:48.386941 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386460 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-hostroot\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.386941 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386485 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx2sj\" (UniqueName: \"kubernetes.io/projected/d2114257-06f1-4070-ad8d-afce484cc205-kube-api-access-dx2sj\") pod \"network-metrics-daemon-kfxzj\" (UID: \"d2114257-06f1-4070-ad8d-afce484cc205\") " pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:47:48.386941 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386508 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-host-run-ovn-kubernetes\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.386941 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386540 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-os-release\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.386941 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386586 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.386941 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386623 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htnkx\" (UniqueName: \"kubernetes.io/projected/62b3fae0-34f0-4dab-871f-12838879bf70-kube-api-access-htnkx\") pod \"aws-ebs-csi-driver-node-xn9cg\" (UID: \"62b3fae0-34f0-4dab-871f-12838879bf70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xn9cg" Apr 16 16:47:48.386941 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386649 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-host-var-lib-cni-bin\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.386941 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386674 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa3397b8-3244-4c47-bf40-735b4b386fb5-system-cni-dir\") pod \"multus-additional-cni-plugins-kfh7g\" (UID: \"fa3397b8-3244-4c47-bf40-735b4b386fb5\") " pod="openshift-multus/multus-additional-cni-plugins-kfh7g" Apr 16 16:47:48.386941 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386708 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-log-socket\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.386941 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386745 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqkjc\" (UniqueName: \"kubernetes.io/projected/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-kube-api-access-rqkjc\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.386941 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386770 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-host-run-netns\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.386941 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386796 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fa3397b8-3244-4c47-bf40-735b4b386fb5-cnibin\") pod \"multus-additional-cni-plugins-kfh7g\" (UID: \"fa3397b8-3244-4c47-bf40-735b4b386fb5\") " pod="openshift-multus/multus-additional-cni-plugins-kfh7g" Apr 16 16:47:48.386941 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386838 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fa3397b8-3244-4c47-bf40-735b4b386fb5-cni-binary-copy\") pod \"multus-additional-cni-plugins-kfh7g\" (UID: \"fa3397b8-3244-4c47-bf40-735b4b386fb5\") " pod="openshift-multus/multus-additional-cni-plugins-kfh7g" Apr 16 16:47:48.386941 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386870 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a63483e4-ee18-41f8-b36c-45b8cb789405-host-slash\") pod \"iptables-alerter-66wx7\" (UID: \"a63483e4-ee18-41f8-b36c-45b8cb789405\") " pod="openshift-network-operator/iptables-alerter-66wx7" Apr 16 16:47:48.387495 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386912 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-host-slash\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.387495 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.386953 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-run-openvswitch\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.429247 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.427599 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-04-15 16:42:47 +0000 UTC" deadline="2027-12-26 03:21:17.294527172 +0000 UTC" Apr 16 16:47:48.429247 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.427625 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14842h33m28.866905465s" Apr 16 16:47:48.443005 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.442978 2574 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Apr 16 16:47:48.475026 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.475003 2574 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Apr 16 16:47:48.487902 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.487879 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-etc-kubernetes\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.488020 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.487907 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8srq7\" (UniqueName: \"kubernetes.io/projected/1d844889-4f54-47ae-8c62-03b86abb6d39-kube-api-access-8srq7\") pod \"network-check-target-6kmtd\" (UID: \"1d844889-4f54-47ae-8c62-03b86abb6d39\") " pod="openshift-network-diagnostics/network-check-target-6kmtd" Apr 16 16:47:48.488020 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.487932 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fa3397b8-3244-4c47-bf40-735b4b386fb5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kfh7g\" (UID: \"fa3397b8-3244-4c47-bf40-735b4b386fb5\") " pod="openshift-multus/multus-additional-cni-plugins-kfh7g" Apr 16 16:47:48.488020 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.487951 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-env-overrides\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.488020 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.487973 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-ovnkube-script-lib\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.488020 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.487979 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-etc-kubernetes\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.488295 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488088 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/62b3fae0-34f0-4dab-871f-12838879bf70-socket-dir\") pod \"aws-ebs-csi-driver-node-xn9cg\" (UID: \"62b3fae0-34f0-4dab-871f-12838879bf70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xn9cg" Apr 16 16:47:48.488295 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488139 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/29ee980f-ef4e-416f-a7db-2b23ef70873e-serviceca\") pod \"node-ca-mx85v\" (UID: \"29ee980f-ef4e-416f-a7db-2b23ef70873e\") " pod="openshift-image-registry/node-ca-mx85v" Apr 16 16:47:48.488295 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488170 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/62b3fae0-34f0-4dab-871f-12838879bf70-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xn9cg\" (UID: \"62b3fae0-34f0-4dab-871f-12838879bf70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xn9cg" Apr 16 16:47:48.488295 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488254 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/62b3fae0-34f0-4dab-871f-12838879bf70-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xn9cg\" (UID: \"62b3fae0-34f0-4dab-871f-12838879bf70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xn9cg" Apr 16 16:47:48.488295 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488266 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/62b3fae0-34f0-4dab-871f-12838879bf70-socket-dir\") pod \"aws-ebs-csi-driver-node-xn9cg\" (UID: \"62b3fae0-34f0-4dab-871f-12838879bf70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xn9cg" Apr 16 16:47:48.488295 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488244 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/62b3fae0-34f0-4dab-871f-12838879bf70-sys-fs\") pod \"aws-ebs-csi-driver-node-xn9cg\" (UID: \"62b3fae0-34f0-4dab-871f-12838879bf70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xn9cg" Apr 16 16:47:48.488547 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488302 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/62b3fae0-34f0-4dab-871f-12838879bf70-sys-fs\") pod \"aws-ebs-csi-driver-node-xn9cg\" (UID: \"62b3fae0-34f0-4dab-871f-12838879bf70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xn9cg" Apr 16 16:47:48.488547 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488310 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-hostroot\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.488547 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488329 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-etc-systemd\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.488547 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488348 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-sys\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.488547 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488370 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-host\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.488547 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488349 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-hostroot\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.488547 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488392 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dx2sj\" (UniqueName: \"kubernetes.io/projected/d2114257-06f1-4070-ad8d-afce484cc205-kube-api-access-dx2sj\") pod \"network-metrics-daemon-kfxzj\" (UID: \"d2114257-06f1-4070-ad8d-afce484cc205\") " pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:47:48.488547 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488480 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-host-run-ovn-kubernetes\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.488547 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488495 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fa3397b8-3244-4c47-bf40-735b4b386fb5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kfh7g\" (UID: \"fa3397b8-3244-4c47-bf40-735b4b386fb5\") " pod="openshift-multus/multus-additional-cni-plugins-kfh7g" Apr 16 16:47:48.488547 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488519 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-os-release\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.488979 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488552 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-etc-kubernetes\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.488979 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488573 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-host-run-ovn-kubernetes\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.488979 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488582 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.488979 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488600 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-os-release\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.488979 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488624 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-htnkx\" (UniqueName: \"kubernetes.io/projected/62b3fae0-34f0-4dab-871f-12838879bf70-kube-api-access-htnkx\") pod \"aws-ebs-csi-driver-node-xn9cg\" (UID: \"62b3fae0-34f0-4dab-871f-12838879bf70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xn9cg" Apr 16 16:47:48.488979 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488627 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.488979 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488649 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-host-var-lib-cni-bin\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.488979 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488673 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa3397b8-3244-4c47-bf40-735b4b386fb5-system-cni-dir\") pod \"multus-additional-cni-plugins-kfh7g\" (UID: \"fa3397b8-3244-4c47-bf40-735b4b386fb5\") " pod="openshift-multus/multus-additional-cni-plugins-kfh7g" Apr 16 16:47:48.488979 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488696 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-log-socket\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.488979 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488723 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rqkjc\" (UniqueName: \"kubernetes.io/projected/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-kube-api-access-rqkjc\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.488979 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488747 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-host-run-netns\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.488979 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488745 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-host-var-lib-cni-bin\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.488979 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488770 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-log-socket\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.488979 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488771 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fa3397b8-3244-4c47-bf40-735b4b386fb5-cnibin\") pod \"multus-additional-cni-plugins-kfh7g\" (UID: \"fa3397b8-3244-4c47-bf40-735b4b386fb5\") " pod="openshift-multus/multus-additional-cni-plugins-kfh7g" Apr 16 16:47:48.488979 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488800 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fa3397b8-3244-4c47-bf40-735b4b386fb5-system-cni-dir\") pod \"multus-additional-cni-plugins-kfh7g\" (UID: \"fa3397b8-3244-4c47-bf40-735b4b386fb5\") " pod="openshift-multus/multus-additional-cni-plugins-kfh7g" Apr 16 16:47:48.488979 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488808 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fa3397b8-3244-4c47-bf40-735b4b386fb5-cni-binary-copy\") pod \"multus-additional-cni-plugins-kfh7g\" (UID: \"fa3397b8-3244-4c47-bf40-735b4b386fb5\") " pod="openshift-multus/multus-additional-cni-plugins-kfh7g" Apr 16 16:47:48.488979 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488803 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fa3397b8-3244-4c47-bf40-735b4b386fb5-cnibin\") pod \"multus-additional-cni-plugins-kfh7g\" (UID: \"fa3397b8-3244-4c47-bf40-735b4b386fb5\") " pod="openshift-multus/multus-additional-cni-plugins-kfh7g" Apr 16 16:47:48.489823 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488837 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b7ff6bde-0c70-4148-bec9-b41af34740f1-konnectivity-ca\") pod \"konnectivity-agent-wtvc6\" (UID: \"b7ff6bde-0c70-4148-bec9-b41af34740f1\") " pod="kube-system/konnectivity-agent-wtvc6" Apr 16 16:47:48.489823 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488842 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-host-run-netns\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.489823 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488865 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-etc-sysconfig\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.489823 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488889 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a63483e4-ee18-41f8-b36c-45b8cb789405-host-slash\") pod \"iptables-alerter-66wx7\" (UID: \"a63483e4-ee18-41f8-b36c-45b8cb789405\") " pod="openshift-network-operator/iptables-alerter-66wx7" Apr 16 16:47:48.489823 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488913 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-host-slash\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.489823 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488936 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-run-openvswitch\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.489823 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488947 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a63483e4-ee18-41f8-b36c-45b8cb789405-host-slash\") pod \"iptables-alerter-66wx7\" (UID: \"a63483e4-ee18-41f8-b36c-45b8cb789405\") " pod="openshift-network-operator/iptables-alerter-66wx7" Apr 16 16:47:48.489823 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488960 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-etc-modprobe-d\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.489823 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488986 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-etc-sysctl-conf\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.489823 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.488988 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-host-slash\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.489823 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489008 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d4923458-dd09-4ed9-877f-7223efd21713-etc-tuned\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.489823 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489048 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-run-openvswitch\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.489823 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489050 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4923458-dd09-4ed9-877f-7223efd21713-tmp\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.489823 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489068 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-env-overrides\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.489823 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489082 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs\") pod \"network-metrics-daemon-kfxzj\" (UID: \"d2114257-06f1-4070-ad8d-afce484cc205\") " pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:47:48.489823 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489108 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kqpn2\" (UniqueName: \"kubernetes.io/projected/a63483e4-ee18-41f8-b36c-45b8cb789405-kube-api-access-kqpn2\") pod \"iptables-alerter-66wx7\" (UID: \"a63483e4-ee18-41f8-b36c-45b8cb789405\") " pod="openshift-network-operator/iptables-alerter-66wx7" Apr 16 16:47:48.489823 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489101 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-ovnkube-script-lib\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.490523 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489131 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/62b3fae0-34f0-4dab-871f-12838879bf70-registration-dir\") pod \"aws-ebs-csi-driver-node-xn9cg\" (UID: \"62b3fae0-34f0-4dab-871f-12838879bf70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xn9cg" Apr 16 16:47:48.490523 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489154 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/62b3fae0-34f0-4dab-871f-12838879bf70-etc-selinux\") pod \"aws-ebs-csi-driver-node-xn9cg\" (UID: \"62b3fae0-34f0-4dab-871f-12838879bf70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xn9cg" Apr 16 16:47:48.490523 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:48.489173 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 16:47:48.490523 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489217 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/62b3fae0-34f0-4dab-871f-12838879bf70-registration-dir\") pod \"aws-ebs-csi-driver-node-xn9cg\" (UID: \"62b3fae0-34f0-4dab-871f-12838879bf70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xn9cg" Apr 16 16:47:48.490523 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489235 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fa3397b8-3244-4c47-bf40-735b4b386fb5-cni-binary-copy\") pod \"multus-additional-cni-plugins-kfh7g\" (UID: \"fa3397b8-3244-4c47-bf40-735b4b386fb5\") " pod="openshift-multus/multus-additional-cni-plugins-kfh7g" Apr 16 16:47:48.490523 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489176 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-multus-cni-dir\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.490523 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:48.489268 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs podName:d2114257-06f1-4070-ad8d-afce484cc205 nodeName:}" failed. No retries permitted until 2026-04-16 16:47:48.989247434 +0000 UTC m=+3.070981626 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs") pod "network-metrics-daemon-kfxzj" (UID: "d2114257-06f1-4070-ad8d-afce484cc205") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 16:47:48.490523 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489308 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/62b3fae0-34f0-4dab-871f-12838879bf70-etc-selinux\") pod \"aws-ebs-csi-driver-node-xn9cg\" (UID: \"62b3fae0-34f0-4dab-871f-12838879bf70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xn9cg" Apr 16 16:47:48.490523 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489316 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-multus-cni-dir\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.490523 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489308 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-multus-daemon-config\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.490523 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489355 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fa3397b8-3244-4c47-bf40-735b4b386fb5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kfh7g\" (UID: \"fa3397b8-3244-4c47-bf40-735b4b386fb5\") " pod="openshift-multus/multus-additional-cni-plugins-kfh7g" Apr 16 16:47:48.490523 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489383 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-etc-sysctl-d\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.490523 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489410 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a63483e4-ee18-41f8-b36c-45b8cb789405-iptables-alerter-script\") pod \"iptables-alerter-66wx7\" (UID: \"a63483e4-ee18-41f8-b36c-45b8cb789405\") " pod="openshift-network-operator/iptables-alerter-66wx7" Apr 16 16:47:48.490523 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489453 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-run-systemd\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.490523 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489482 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-ovnkube-config\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.490523 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489513 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-ovn-node-metrics-cert\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.490523 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489529 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-run-systemd\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.491319 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489537 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/62b3fae0-34f0-4dab-871f-12838879bf70-device-dir\") pod \"aws-ebs-csi-driver-node-xn9cg\" (UID: \"62b3fae0-34f0-4dab-871f-12838879bf70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xn9cg" Apr 16 16:47:48.491319 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489560 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-host-var-lib-cni-multus\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.491319 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489585 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-host-var-lib-kubelet\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.491319 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489609 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-etc-openvswitch\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.491319 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489636 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-host-run-multus-certs\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.491319 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489658 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-host-var-lib-kubelet\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.491319 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489659 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sjtgp\" (UniqueName: \"kubernetes.io/projected/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-kube-api-access-sjtgp\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.491319 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489695 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fa3397b8-3244-4c47-bf40-735b4b386fb5-os-release\") pod \"multus-additional-cni-plugins-kfh7g\" (UID: \"fa3397b8-3244-4c47-bf40-735b4b386fb5\") " pod="openshift-multus/multus-additional-cni-plugins-kfh7g" Apr 16 16:47:48.491319 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489723 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fa3397b8-3244-4c47-bf40-735b4b386fb5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kfh7g\" (UID: \"fa3397b8-3244-4c47-bf40-735b4b386fb5\") " pod="openshift-multus/multus-additional-cni-plugins-kfh7g" Apr 16 16:47:48.491319 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489774 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-host-var-lib-cni-multus\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.491319 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489613 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/62b3fae0-34f0-4dab-871f-12838879bf70-device-dir\") pod \"aws-ebs-csi-driver-node-xn9cg\" (UID: \"62b3fae0-34f0-4dab-871f-12838879bf70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xn9cg" Apr 16 16:47:48.491319 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489807 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-multus-daemon-config\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.491319 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489823 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-etc-openvswitch\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.491319 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489866 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-host-run-multus-certs\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.491319 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489900 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cnln2\" (UniqueName: \"kubernetes.io/projected/fa3397b8-3244-4c47-bf40-735b4b386fb5-kube-api-access-cnln2\") pod \"multus-additional-cni-plugins-kfh7g\" (UID: \"fa3397b8-3244-4c47-bf40-735b4b386fb5\") " pod="openshift-multus/multus-additional-cni-plugins-kfh7g" Apr 16 16:47:48.491319 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489958 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fa3397b8-3244-4c47-bf40-735b4b386fb5-os-release\") pod \"multus-additional-cni-plugins-kfh7g\" (UID: \"fa3397b8-3244-4c47-bf40-735b4b386fb5\") " pod="openshift-multus/multus-additional-cni-plugins-kfh7g" Apr 16 16:47:48.491319 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490037 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fa3397b8-3244-4c47-bf40-735b4b386fb5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kfh7g\" (UID: \"fa3397b8-3244-4c47-bf40-735b4b386fb5\") " pod="openshift-multus/multus-additional-cni-plugins-kfh7g" Apr 16 16:47:48.491897 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.489926 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/29ee980f-ef4e-416f-a7db-2b23ef70873e-host\") pod \"node-ca-mx85v\" (UID: \"29ee980f-ef4e-416f-a7db-2b23ef70873e\") " pod="openshift-image-registry/node-ca-mx85v" Apr 16 16:47:48.491897 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490158 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv469\" (UniqueName: \"kubernetes.io/projected/29ee980f-ef4e-416f-a7db-2b23ef70873e-kube-api-access-xv469\") pod \"node-ca-mx85v\" (UID: \"29ee980f-ef4e-416f-a7db-2b23ef70873e\") " pod="openshift-image-registry/node-ca-mx85v" Apr 16 16:47:48.491897 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490182 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-host-cni-bin\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.491897 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490197 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-cnibin\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.491897 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490234 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8eb081ed-d37b-42a9-a3c9-8549bcb2da0d-hosts-file\") pod \"node-resolver-5zftw\" (UID: \"8eb081ed-d37b-42a9-a3c9-8549bcb2da0d\") " pod="openshift-dns/node-resolver-5zftw" Apr 16 16:47:48.491897 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490281 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-host-cni-bin\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.491897 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490288 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fa3397b8-3244-4c47-bf40-735b4b386fb5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kfh7g\" (UID: \"fa3397b8-3244-4c47-bf40-735b4b386fb5\") " pod="openshift-multus/multus-additional-cni-plugins-kfh7g" Apr 16 16:47:48.491897 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490300 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-cnibin\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.491897 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490326 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-run-ovn\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.491897 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490354 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8eb081ed-d37b-42a9-a3c9-8549bcb2da0d-hosts-file\") pod \"node-resolver-5zftw\" (UID: \"8eb081ed-d37b-42a9-a3c9-8549bcb2da0d\") " pod="openshift-dns/node-resolver-5zftw" Apr 16 16:47:48.491897 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490351 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-cni-binary-copy\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.491897 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490385 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-multus-socket-dir-parent\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.491897 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490399 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-run-ovn\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.491897 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490403 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a63483e4-ee18-41f8-b36c-45b8cb789405-iptables-alerter-script\") pod \"iptables-alerter-66wx7\" (UID: \"a63483e4-ee18-41f8-b36c-45b8cb789405\") " pod="openshift-network-operator/iptables-alerter-66wx7" Apr 16 16:47:48.491897 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490426 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-multus-conf-dir\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.491897 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490469 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-multus-conf-dir\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.491897 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490477 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-multus-socket-dir-parent\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.491897 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490509 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-run\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.492507 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490535 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8eb081ed-d37b-42a9-a3c9-8549bcb2da0d-tmp-dir\") pod \"node-resolver-5zftw\" (UID: \"8eb081ed-d37b-42a9-a3c9-8549bcb2da0d\") " pod="openshift-dns/node-resolver-5zftw" Apr 16 16:47:48.492507 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490559 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-host-kubelet\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.492507 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490583 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-systemd-units\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.492507 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490606 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-host-run-netns\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.492507 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490629 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-lib-modules\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.492507 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490651 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-var-lib-kubelet\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.492507 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490662 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-ovnkube-config\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.492507 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490674 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twsvp\" (UniqueName: \"kubernetes.io/projected/d4923458-dd09-4ed9-877f-7223efd21713-kube-api-access-twsvp\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.492507 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490703 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-var-lib-openvswitch\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.492507 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490729 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-host-run-k8s-cni-cncf-io\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.492507 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490719 2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Apr 16 16:47:48.492507 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490772 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-host-run-netns\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.492507 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490777 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-systemd-units\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.492507 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490793 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8eb081ed-d37b-42a9-a3c9-8549bcb2da0d-tmp-dir\") pod \"node-resolver-5zftw\" (UID: \"8eb081ed-d37b-42a9-a3c9-8549bcb2da0d\") " pod="openshift-dns/node-resolver-5zftw" Apr 16 16:47:48.492507 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490800 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b7ff6bde-0c70-4148-bec9-b41af34740f1-agent-certs\") pod \"konnectivity-agent-wtvc6\" (UID: \"b7ff6bde-0c70-4148-bec9-b41af34740f1\") " pod="kube-system/konnectivity-agent-wtvc6" Apr 16 16:47:48.492507 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490823 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hfq92\" (UniqueName: \"kubernetes.io/projected/8eb081ed-d37b-42a9-a3c9-8549bcb2da0d-kube-api-access-hfq92\") pod \"node-resolver-5zftw\" (UID: \"8eb081ed-d37b-42a9-a3c9-8549bcb2da0d\") " pod="openshift-dns/node-resolver-5zftw" Apr 16 16:47:48.492507 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490824 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-host-run-k8s-cni-cncf-io\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.492507 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490835 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-var-lib-openvswitch\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.493252 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490849 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-host-cni-netd\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.493252 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490741 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-host-kubelet\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.493252 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490871 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-system-cni-dir\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.493252 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490853 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-cni-binary-copy\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.493252 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490897 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-node-log\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.493252 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490904 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-host-cni-netd\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.493252 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.490976 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-node-log\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.493252 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.491017 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-system-cni-dir\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.494249 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.494233 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-ovn-node-metrics-cert\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.497853 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:48.497831 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 16 16:47:48.497853 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:48.497852 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 16 16:47:48.498004 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:48.497865 2574 projected.go:194] Error preparing data for projected volume kube-api-access-8srq7 for pod openshift-network-diagnostics/network-check-target-6kmtd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 16:47:48.498004 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:48.497916 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d844889-4f54-47ae-8c62-03b86abb6d39-kube-api-access-8srq7 podName:1d844889-4f54-47ae-8c62-03b86abb6d39 nodeName:}" failed. No retries permitted until 2026-04-16 16:47:48.997901248 +0000 UTC m=+3.079635441 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8srq7" (UniqueName: "kubernetes.io/projected/1d844889-4f54-47ae-8c62-03b86abb6d39-kube-api-access-8srq7") pod "network-check-target-6kmtd" (UID: "1d844889-4f54-47ae-8c62-03b86abb6d39") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 16:47:48.498667 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.498640 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx2sj\" (UniqueName: \"kubernetes.io/projected/d2114257-06f1-4070-ad8d-afce484cc205-kube-api-access-dx2sj\") pod \"network-metrics-daemon-kfxzj\" (UID: \"d2114257-06f1-4070-ad8d-afce484cc205\") " pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:47:48.498771 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.498721 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqkjc\" (UniqueName: \"kubernetes.io/projected/5ec5cd46-2ab4-4b8a-961e-22cad6381d66-kube-api-access-rqkjc\") pod \"ovnkube-node-hfbd8\" (UID: \"5ec5cd46-2ab4-4b8a-961e-22cad6381d66\") " pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.498771 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.498754 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqpn2\" (UniqueName: \"kubernetes.io/projected/a63483e4-ee18-41f8-b36c-45b8cb789405-kube-api-access-kqpn2\") pod \"iptables-alerter-66wx7\" (UID: \"a63483e4-ee18-41f8-b36c-45b8cb789405\") " pod="openshift-network-operator/iptables-alerter-66wx7" Apr 16 16:47:48.499110 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.499087 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjtgp\" (UniqueName: \"kubernetes.io/projected/d655ba61-fb76-4e4b-bbef-f9cfca4b8495-kube-api-access-sjtgp\") pod \"multus-4k9v9\" (UID: \"d655ba61-fb76-4e4b-bbef-f9cfca4b8495\") " pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.500250 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.500142 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htnkx\" (UniqueName: \"kubernetes.io/projected/62b3fae0-34f0-4dab-871f-12838879bf70-kube-api-access-htnkx\") pod \"aws-ebs-csi-driver-node-xn9cg\" (UID: \"62b3fae0-34f0-4dab-871f-12838879bf70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xn9cg" Apr 16 16:47:48.500250 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.500214 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnln2\" (UniqueName: \"kubernetes.io/projected/fa3397b8-3244-4c47-bf40-735b4b386fb5-kube-api-access-cnln2\") pod \"multus-additional-cni-plugins-kfh7g\" (UID: \"fa3397b8-3244-4c47-bf40-735b4b386fb5\") " pod="openshift-multus/multus-additional-cni-plugins-kfh7g" Apr 16 16:47:48.500557 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.500540 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfq92\" (UniqueName: \"kubernetes.io/projected/8eb081ed-d37b-42a9-a3c9-8549bcb2da0d-kube-api-access-hfq92\") pod \"node-resolver-5zftw\" (UID: \"8eb081ed-d37b-42a9-a3c9-8549bcb2da0d\") " pod="openshift-dns/node-resolver-5zftw" Apr 16 16:47:48.591443 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.591387 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/29ee980f-ef4e-416f-a7db-2b23ef70873e-host\") pod \"node-ca-mx85v\" (UID: \"29ee980f-ef4e-416f-a7db-2b23ef70873e\") " pod="openshift-image-registry/node-ca-mx85v" Apr 16 16:47:48.591443 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.591416 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xv469\" (UniqueName: \"kubernetes.io/projected/29ee980f-ef4e-416f-a7db-2b23ef70873e-kube-api-access-xv469\") pod \"node-ca-mx85v\" (UID: \"29ee980f-ef4e-416f-a7db-2b23ef70873e\") " pod="openshift-image-registry/node-ca-mx85v" Apr 16 16:47:48.591443 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.591437 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-run\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.591654 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.591461 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-lib-modules\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.591654 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.591483 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/29ee980f-ef4e-416f-a7db-2b23ef70873e-host\") pod \"node-ca-mx85v\" (UID: \"29ee980f-ef4e-416f-a7db-2b23ef70873e\") " pod="openshift-image-registry/node-ca-mx85v" Apr 16 16:47:48.591654 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.591521 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-run\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.591654 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.591569 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-lib-modules\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.591654 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.591639 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-var-lib-kubelet\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.591881 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.591662 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-twsvp\" (UniqueName: \"kubernetes.io/projected/d4923458-dd09-4ed9-877f-7223efd21713-kube-api-access-twsvp\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.591881 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.591679 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b7ff6bde-0c70-4148-bec9-b41af34740f1-agent-certs\") pod \"konnectivity-agent-wtvc6\" (UID: \"b7ff6bde-0c70-4148-bec9-b41af34740f1\") " pod="kube-system/konnectivity-agent-wtvc6" Apr 16 16:47:48.591881 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.591711 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/29ee980f-ef4e-416f-a7db-2b23ef70873e-serviceca\") pod \"node-ca-mx85v\" (UID: \"29ee980f-ef4e-416f-a7db-2b23ef70873e\") " pod="openshift-image-registry/node-ca-mx85v" Apr 16 16:47:48.591881 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.591789 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-var-lib-kubelet\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.591881 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.591824 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-etc-systemd\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.591881 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.591842 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-sys\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.591881 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.591867 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-host\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.592213 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.591910 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-etc-systemd\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.592213 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.591896 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-etc-kubernetes\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.592213 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.591943 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-sys\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.592213 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.591955 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b7ff6bde-0c70-4148-bec9-b41af34740f1-konnectivity-ca\") pod \"konnectivity-agent-wtvc6\" (UID: \"b7ff6bde-0c70-4148-bec9-b41af34740f1\") " pod="kube-system/konnectivity-agent-wtvc6" Apr 16 16:47:48.592213 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.591967 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-host\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.592213 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.592005 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-etc-sysconfig\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.592213 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.592029 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-etc-kubernetes\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.592213 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.592035 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-etc-modprobe-d\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.592213 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.592080 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-etc-sysconfig\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.592213 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.592110 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-etc-sysctl-conf\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.592213 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.592119 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/29ee980f-ef4e-416f-a7db-2b23ef70873e-serviceca\") pod \"node-ca-mx85v\" (UID: \"29ee980f-ef4e-416f-a7db-2b23ef70873e\") " pod="openshift-image-registry/node-ca-mx85v" Apr 16 16:47:48.592213 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.592130 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-etc-modprobe-d\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.592213 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.592140 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d4923458-dd09-4ed9-877f-7223efd21713-etc-tuned\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.592213 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.592164 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4923458-dd09-4ed9-877f-7223efd21713-tmp\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.592949 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.592228 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-etc-sysctl-d\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.592949 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.592229 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-etc-sysctl-conf\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.592949 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.592337 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d4923458-dd09-4ed9-877f-7223efd21713-etc-sysctl-d\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.593108 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.592997 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b7ff6bde-0c70-4148-bec9-b41af34740f1-konnectivity-ca\") pod \"konnectivity-agent-wtvc6\" (UID: \"b7ff6bde-0c70-4148-bec9-b41af34740f1\") " pod="kube-system/konnectivity-agent-wtvc6" Apr 16 16:47:48.594176 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.594152 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b7ff6bde-0c70-4148-bec9-b41af34740f1-agent-certs\") pod \"konnectivity-agent-wtvc6\" (UID: \"b7ff6bde-0c70-4148-bec9-b41af34740f1\") " pod="kube-system/konnectivity-agent-wtvc6" Apr 16 16:47:48.594302 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.594241 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d4923458-dd09-4ed9-877f-7223efd21713-tmp\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.594410 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.594394 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d4923458-dd09-4ed9-877f-7223efd21713-etc-tuned\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.609604 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.609567 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twsvp\" (UniqueName: \"kubernetes.io/projected/d4923458-dd09-4ed9-877f-7223efd21713-kube-api-access-twsvp\") pod \"tuned-2m2qc\" (UID: \"d4923458-dd09-4ed9-877f-7223efd21713\") " pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.610480 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.610459 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv469\" (UniqueName: \"kubernetes.io/projected/29ee980f-ef4e-416f-a7db-2b23ef70873e-kube-api-access-xv469\") pod \"node-ca-mx85v\" (UID: \"29ee980f-ef4e-416f-a7db-2b23ef70873e\") " pod="openshift-image-registry/node-ca-mx85v" Apr 16 16:47:48.678948 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.678914 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5zftw" Apr 16 16:47:48.684630 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.684607 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kfh7g" Apr 16 16:47:48.693253 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.693231 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:47:48.697838 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.697822 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xn9cg" Apr 16 16:47:48.703400 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.703381 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4k9v9" Apr 16 16:47:48.709943 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.709920 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-66wx7" Apr 16 16:47:48.716470 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.716451 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wtvc6" Apr 16 16:47:48.721937 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.721918 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" Apr 16 16:47:48.727434 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.727418 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mx85v" Apr 16 16:47:48.995080 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:48.994999 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs\") pod \"network-metrics-daemon-kfxzj\" (UID: \"d2114257-06f1-4070-ad8d-afce484cc205\") " pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:47:48.995242 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:48.995156 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 16:47:48.995242 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:48.995238 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs podName:d2114257-06f1-4070-ad8d-afce484cc205 nodeName:}" failed. No retries permitted until 2026-04-16 16:47:49.995217301 +0000 UTC m=+4.076951493 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs") pod "network-metrics-daemon-kfxzj" (UID: "d2114257-06f1-4070-ad8d-afce484cc205") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 16:47:49.091313 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:49.091279 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eb081ed_d37b_42a9_a3c9_8549bcb2da0d.slice/crio-0dc3173e63b78072621d7e521395926b3acb66d4d4922f5db7dab732ff384a00 WatchSource:0}: Error finding container 0dc3173e63b78072621d7e521395926b3acb66d4d4922f5db7dab732ff384a00: Status 404 returned error can't find the container with id 0dc3173e63b78072621d7e521395926b3acb66d4d4922f5db7dab732ff384a00 Apr 16 16:47:49.092921 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:49.092891 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4923458_dd09_4ed9_877f_7223efd21713.slice/crio-e269fb84dc06f5491738efc86e6e1abb5e1415812573653361203dd86501ce0d WatchSource:0}: Error finding container e269fb84dc06f5491738efc86e6e1abb5e1415812573653361203dd86501ce0d: Status 404 returned error can't find the container with id e269fb84dc06f5491738efc86e6e1abb5e1415812573653361203dd86501ce0d Apr 16 16:47:49.095534 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:49.095426 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8srq7\" (UniqueName: \"kubernetes.io/projected/1d844889-4f54-47ae-8c62-03b86abb6d39-kube-api-access-8srq7\") pod \"network-check-target-6kmtd\" (UID: \"1d844889-4f54-47ae-8c62-03b86abb6d39\") " pod="openshift-network-diagnostics/network-check-target-6kmtd" Apr 16 16:47:49.095616 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:49.095566 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 16 16:47:49.095616 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:49.095583 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 16 16:47:49.095616 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:49.095595 2574 projected.go:194] Error preparing data for projected volume kube-api-access-8srq7 for pod openshift-network-diagnostics/network-check-target-6kmtd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 16:47:49.095757 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:49.095656 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d844889-4f54-47ae-8c62-03b86abb6d39-kube-api-access-8srq7 podName:1d844889-4f54-47ae-8c62-03b86abb6d39 nodeName:}" failed. No retries permitted until 2026-04-16 16:47:50.095626952 +0000 UTC m=+4.177361145 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8srq7" (UniqueName: "kubernetes.io/projected/1d844889-4f54-47ae-8c62-03b86abb6d39-kube-api-access-8srq7") pod "network-check-target-6kmtd" (UID: "1d844889-4f54-47ae-8c62-03b86abb6d39") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 16:47:49.096355 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:49.096325 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7ff6bde_0c70_4148_bec9_b41af34740f1.slice/crio-f0e6e5bcaa21e05e6b73b109e10c622ceaa42e44571159bec517b32e393a4bb0 WatchSource:0}: Error finding container f0e6e5bcaa21e05e6b73b109e10c622ceaa42e44571159bec517b32e393a4bb0: Status 404 returned error can't find the container with id f0e6e5bcaa21e05e6b73b109e10c622ceaa42e44571159bec517b32e393a4bb0 Apr 16 16:47:49.097384 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:49.097356 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29ee980f_ef4e_416f_a7db_2b23ef70873e.slice/crio-9d8a7ef7dcc553ee84d71340a92a8fe06ff4a32dbea030f933e61a70a9da5390 WatchSource:0}: Error finding container 9d8a7ef7dcc553ee84d71340a92a8fe06ff4a32dbea030f933e61a70a9da5390: Status 404 returned error can't find the container with id 9d8a7ef7dcc553ee84d71340a92a8fe06ff4a32dbea030f933e61a70a9da5390 Apr 16 16:47:49.098128 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:49.098022 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ec5cd46_2ab4_4b8a_961e_22cad6381d66.slice/crio-43459c7c5fdfedb9c598b4f6b09f458ffe981551cc9f39baad99a9036fb398c9 WatchSource:0}: Error finding container 43459c7c5fdfedb9c598b4f6b09f458ffe981551cc9f39baad99a9036fb398c9: Status 404 returned error can't find the container with id 43459c7c5fdfedb9c598b4f6b09f458ffe981551cc9f39baad99a9036fb398c9 Apr 16 16:47:49.098741 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:49.098627 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa3397b8_3244_4c47_bf40_735b4b386fb5.slice/crio-901db54ad06ef9380c544c6e84259e4743f0e31cd9c176afd856d98665c541bc WatchSource:0}: Error finding container 901db54ad06ef9380c544c6e84259e4743f0e31cd9c176afd856d98665c541bc: Status 404 returned error can't find the container with id 901db54ad06ef9380c544c6e84259e4743f0e31cd9c176afd856d98665c541bc Apr 16 16:47:49.099580 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:49.099534 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda63483e4_ee18_41f8_b36c_45b8cb789405.slice/crio-2f18e75a04a9530158d7ee103ab09fa6a10648ca0771ea29ed91a01fe251d6b3 WatchSource:0}: Error finding container 2f18e75a04a9530158d7ee103ab09fa6a10648ca0771ea29ed91a01fe251d6b3: Status 404 returned error can't find the container with id 2f18e75a04a9530158d7ee103ab09fa6a10648ca0771ea29ed91a01fe251d6b3 Apr 16 16:47:49.101444 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:49.101345 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd655ba61_fb76_4e4b_bbef_f9cfca4b8495.slice/crio-93430c2a8a61401eb25f02428d5cad5c31eb0d42021e1734320f2c92017ed1f3 WatchSource:0}: Error finding container 93430c2a8a61401eb25f02428d5cad5c31eb0d42021e1734320f2c92017ed1f3: Status 404 returned error can't find the container with id 93430c2a8a61401eb25f02428d5cad5c31eb0d42021e1734320f2c92017ed1f3 Apr 16 16:47:49.102774 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:47:49.102756 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62b3fae0_34f0_4dab_871f_12838879bf70.slice/crio-2d1cbb88f614891cfecc3779b47f782908bb69e6ad69f0713d3fb659478b0878 WatchSource:0}: Error finding container 2d1cbb88f614891cfecc3779b47f782908bb69e6ad69f0713d3fb659478b0878: Status 404 returned error can't find the container with id 2d1cbb88f614891cfecc3779b47f782908bb69e6ad69f0713d3fb659478b0878 Apr 16 16:47:49.428903 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:49.428535 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-04-15 16:42:47 +0000 UTC" deadline="2027-11-25 15:07:30.756137102 +0000 UTC" Apr 16 16:47:49.428903 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:49.428809 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14110h19m41.327334611s" Apr 16 16:47:49.517628 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:49.516986 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-49.ec2.internal" event={"ID":"c0e6e3b61a3e8cfc6394e27b3b2ac0f3","Type":"ContainerStarted","Data":"71deb2c1bb14783c98e0c2b74b5e1699209ed7c538b53b2e8a3685c0b84ae9e0"} Apr 16 16:47:49.519366 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:49.519339 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xn9cg" event={"ID":"62b3fae0-34f0-4dab-871f-12838879bf70","Type":"ContainerStarted","Data":"2d1cbb88f614891cfecc3779b47f782908bb69e6ad69f0713d3fb659478b0878"} Apr 16 16:47:49.522947 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:49.522925 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4k9v9" event={"ID":"d655ba61-fb76-4e4b-bbef-f9cfca4b8495","Type":"ContainerStarted","Data":"93430c2a8a61401eb25f02428d5cad5c31eb0d42021e1734320f2c92017ed1f3"} Apr 16 16:47:49.524747 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:49.524700 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-66wx7" event={"ID":"a63483e4-ee18-41f8-b36c-45b8cb789405","Type":"ContainerStarted","Data":"2f18e75a04a9530158d7ee103ab09fa6a10648ca0771ea29ed91a01fe251d6b3"} Apr 16 16:47:49.526417 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:49.526372 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kfh7g" event={"ID":"fa3397b8-3244-4c47-bf40-735b4b386fb5","Type":"ContainerStarted","Data":"901db54ad06ef9380c544c6e84259e4743f0e31cd9c176afd856d98665c541bc"} Apr 16 16:47:49.527803 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:49.527752 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mx85v" event={"ID":"29ee980f-ef4e-416f-a7db-2b23ef70873e","Type":"ContainerStarted","Data":"9d8a7ef7dcc553ee84d71340a92a8fe06ff4a32dbea030f933e61a70a9da5390"} Apr 16 16:47:49.529019 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:49.528975 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" event={"ID":"5ec5cd46-2ab4-4b8a-961e-22cad6381d66","Type":"ContainerStarted","Data":"43459c7c5fdfedb9c598b4f6b09f458ffe981551cc9f39baad99a9036fb398c9"} Apr 16 16:47:49.530935 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:49.530889 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wtvc6" event={"ID":"b7ff6bde-0c70-4148-bec9-b41af34740f1","Type":"ContainerStarted","Data":"f0e6e5bcaa21e05e6b73b109e10c622ceaa42e44571159bec517b32e393a4bb0"} Apr 16 16:47:49.532466 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:49.531868 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-49.ec2.internal" podStartSLOduration=2.531856002 podStartE2EDuration="2.531856002s" podCreationTimestamp="2026-04-16 16:47:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 16:47:49.531259152 +0000 UTC m=+3.612993361" watchObservedRunningTime="2026-04-16 16:47:49.531856002 +0000 UTC m=+3.613590219" Apr 16 16:47:49.535745 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:49.535697 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" event={"ID":"d4923458-dd09-4ed9-877f-7223efd21713","Type":"ContainerStarted","Data":"e269fb84dc06f5491738efc86e6e1abb5e1415812573653361203dd86501ce0d"} Apr 16 16:47:49.546684 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:49.546608 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5zftw" event={"ID":"8eb081ed-d37b-42a9-a3c9-8549bcb2da0d","Type":"ContainerStarted","Data":"0dc3173e63b78072621d7e521395926b3acb66d4d4922f5db7dab732ff384a00"} Apr 16 16:47:50.003255 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:50.003229 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs\") pod \"network-metrics-daemon-kfxzj\" (UID: \"d2114257-06f1-4070-ad8d-afce484cc205\") " pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:47:50.003380 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:50.003363 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 16:47:50.003442 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:50.003430 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs podName:d2114257-06f1-4070-ad8d-afce484cc205 nodeName:}" failed. No retries permitted until 2026-04-16 16:47:52.00341015 +0000 UTC m=+6.085144366 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs") pod "network-metrics-daemon-kfxzj" (UID: "d2114257-06f1-4070-ad8d-afce484cc205") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 16:47:50.104825 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:50.104229 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8srq7\" (UniqueName: \"kubernetes.io/projected/1d844889-4f54-47ae-8c62-03b86abb6d39-kube-api-access-8srq7\") pod \"network-check-target-6kmtd\" (UID: \"1d844889-4f54-47ae-8c62-03b86abb6d39\") " pod="openshift-network-diagnostics/network-check-target-6kmtd" Apr 16 16:47:50.104825 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:50.104401 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 16 16:47:50.104825 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:50.104424 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 16 16:47:50.104825 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:50.104436 2574 projected.go:194] Error preparing data for projected volume kube-api-access-8srq7 for pod openshift-network-diagnostics/network-check-target-6kmtd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 16:47:50.104825 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:50.104497 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d844889-4f54-47ae-8c62-03b86abb6d39-kube-api-access-8srq7 podName:1d844889-4f54-47ae-8c62-03b86abb6d39 nodeName:}" failed. No retries permitted until 2026-04-16 16:47:52.104480273 +0000 UTC m=+6.186214488 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8srq7" (UniqueName: "kubernetes.io/projected/1d844889-4f54-47ae-8c62-03b86abb6d39-kube-api-access-8srq7") pod "network-check-target-6kmtd" (UID: "1d844889-4f54-47ae-8c62-03b86abb6d39") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 16:47:50.507956 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:50.507924 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:47:50.508412 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:50.508062 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfxzj" podUID="d2114257-06f1-4070-ad8d-afce484cc205" Apr 16 16:47:50.508497 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:50.508477 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kmtd" Apr 16 16:47:50.508710 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:50.508585 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kmtd" podUID="1d844889-4f54-47ae-8c62-03b86abb6d39" Apr 16 16:47:50.564271 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:50.564135 2574 generic.go:358] "Generic (PLEG): container finished" podID="985f218434988cfff775e487e766fd2a" containerID="d8630197173374058dfb1581e65ea65ad3c3700c96e3c5da334d16070683839f" exitCode=0 Apr 16 16:47:50.564439 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:50.564315 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-49.ec2.internal" event={"ID":"985f218434988cfff775e487e766fd2a","Type":"ContainerDied","Data":"d8630197173374058dfb1581e65ea65ad3c3700c96e3c5da334d16070683839f"} Apr 16 16:47:51.574226 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:51.573677 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-49.ec2.internal" event={"ID":"985f218434988cfff775e487e766fd2a","Type":"ContainerStarted","Data":"7fb3cdab7fc80a125900f350991b19875f51ce6c80b826c2844e2216b8f95a2e"} Apr 16 16:47:52.021515 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:52.021429 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs\") pod \"network-metrics-daemon-kfxzj\" (UID: \"d2114257-06f1-4070-ad8d-afce484cc205\") " pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:47:52.021682 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:52.021577 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 16:47:52.021682 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:52.021648 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs podName:d2114257-06f1-4070-ad8d-afce484cc205 nodeName:}" failed. No retries permitted until 2026-04-16 16:47:56.021626692 +0000 UTC m=+10.103360885 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs") pod "network-metrics-daemon-kfxzj" (UID: "d2114257-06f1-4070-ad8d-afce484cc205") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 16:47:52.122655 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:52.122623 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8srq7\" (UniqueName: \"kubernetes.io/projected/1d844889-4f54-47ae-8c62-03b86abb6d39-kube-api-access-8srq7\") pod \"network-check-target-6kmtd\" (UID: \"1d844889-4f54-47ae-8c62-03b86abb6d39\") " pod="openshift-network-diagnostics/network-check-target-6kmtd" Apr 16 16:47:52.122801 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:52.122768 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 16 16:47:52.122801 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:52.122781 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 16 16:47:52.122801 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:52.122789 2574 projected.go:194] Error preparing data for projected volume kube-api-access-8srq7 for pod openshift-network-diagnostics/network-check-target-6kmtd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 16:47:52.122895 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:52.122837 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d844889-4f54-47ae-8c62-03b86abb6d39-kube-api-access-8srq7 podName:1d844889-4f54-47ae-8c62-03b86abb6d39 nodeName:}" failed. No retries permitted until 2026-04-16 16:47:56.122824864 +0000 UTC m=+10.204559057 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8srq7" (UniqueName: "kubernetes.io/projected/1d844889-4f54-47ae-8c62-03b86abb6d39-kube-api-access-8srq7") pod "network-check-target-6kmtd" (UID: "1d844889-4f54-47ae-8c62-03b86abb6d39") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 16:47:52.511707 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:52.511009 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:47:52.511707 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:52.511145 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfxzj" podUID="d2114257-06f1-4070-ad8d-afce484cc205" Apr 16 16:47:52.511707 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:52.511256 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kmtd" Apr 16 16:47:52.511707 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:52.511330 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kmtd" podUID="1d844889-4f54-47ae-8c62-03b86abb6d39" Apr 16 16:47:54.507376 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:54.507306 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:47:54.507885 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:54.507790 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfxzj" podUID="d2114257-06f1-4070-ad8d-afce484cc205" Apr 16 16:47:54.508318 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:54.508262 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kmtd" Apr 16 16:47:54.508459 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:54.508436 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kmtd" podUID="1d844889-4f54-47ae-8c62-03b86abb6d39" Apr 16 16:47:56.057169 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:56.057132 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs\") pod \"network-metrics-daemon-kfxzj\" (UID: \"d2114257-06f1-4070-ad8d-afce484cc205\") " pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:47:56.057620 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:56.057304 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 16:47:56.057620 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:56.057389 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs podName:d2114257-06f1-4070-ad8d-afce484cc205 nodeName:}" failed. No retries permitted until 2026-04-16 16:48:04.057374562 +0000 UTC m=+18.139108759 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs") pod "network-metrics-daemon-kfxzj" (UID: "d2114257-06f1-4070-ad8d-afce484cc205") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 16:47:56.158133 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:56.158090 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8srq7\" (UniqueName: \"kubernetes.io/projected/1d844889-4f54-47ae-8c62-03b86abb6d39-kube-api-access-8srq7\") pod \"network-check-target-6kmtd\" (UID: \"1d844889-4f54-47ae-8c62-03b86abb6d39\") " pod="openshift-network-diagnostics/network-check-target-6kmtd" Apr 16 16:47:56.158317 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:56.158270 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 16 16:47:56.158317 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:56.158294 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 16 16:47:56.158317 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:56.158307 2574 projected.go:194] Error preparing data for projected volume kube-api-access-8srq7 for pod openshift-network-diagnostics/network-check-target-6kmtd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 16:47:56.158466 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:56.158376 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d844889-4f54-47ae-8c62-03b86abb6d39-kube-api-access-8srq7 podName:1d844889-4f54-47ae-8c62-03b86abb6d39 nodeName:}" failed. No retries permitted until 2026-04-16 16:48:04.158354845 +0000 UTC m=+18.240089039 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8srq7" (UniqueName: "kubernetes.io/projected/1d844889-4f54-47ae-8c62-03b86abb6d39-kube-api-access-8srq7") pod "network-check-target-6kmtd" (UID: "1d844889-4f54-47ae-8c62-03b86abb6d39") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 16:47:56.508392 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:56.508318 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:47:56.508547 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:56.508434 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfxzj" podUID="d2114257-06f1-4070-ad8d-afce484cc205" Apr 16 16:47:56.508547 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:56.508520 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kmtd" Apr 16 16:47:56.508657 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:56.508621 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kmtd" podUID="1d844889-4f54-47ae-8c62-03b86abb6d39" Apr 16 16:47:58.508004 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:58.507968 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kmtd" Apr 16 16:47:58.508004 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:47:58.508003 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:47:58.508531 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:58.508108 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kmtd" podUID="1d844889-4f54-47ae-8c62-03b86abb6d39" Apr 16 16:47:58.508531 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:47:58.508235 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfxzj" podUID="d2114257-06f1-4070-ad8d-afce484cc205" Apr 16 16:48:00.507235 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:00.507192 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:48:00.507692 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:00.507337 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfxzj" podUID="d2114257-06f1-4070-ad8d-afce484cc205" Apr 16 16:48:00.507692 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:00.507391 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kmtd" Apr 16 16:48:00.507692 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:00.507497 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kmtd" podUID="1d844889-4f54-47ae-8c62-03b86abb6d39" Apr 16 16:48:02.507490 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:02.507444 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kmtd" Apr 16 16:48:02.507490 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:02.507471 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:48:02.507983 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:02.507571 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kmtd" podUID="1d844889-4f54-47ae-8c62-03b86abb6d39" Apr 16 16:48:02.507983 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:02.507700 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfxzj" podUID="d2114257-06f1-4070-ad8d-afce484cc205" Apr 16 16:48:04.116850 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:04.116789 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs\") pod \"network-metrics-daemon-kfxzj\" (UID: \"d2114257-06f1-4070-ad8d-afce484cc205\") " pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:48:04.117436 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:04.116913 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 16:48:04.117436 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:04.116979 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs podName:d2114257-06f1-4070-ad8d-afce484cc205 nodeName:}" failed. No retries permitted until 2026-04-16 16:48:20.116960343 +0000 UTC m=+34.198694550 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs") pod "network-metrics-daemon-kfxzj" (UID: "d2114257-06f1-4070-ad8d-afce484cc205") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 16:48:04.217629 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:04.217589 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8srq7\" (UniqueName: \"kubernetes.io/projected/1d844889-4f54-47ae-8c62-03b86abb6d39-kube-api-access-8srq7\") pod \"network-check-target-6kmtd\" (UID: \"1d844889-4f54-47ae-8c62-03b86abb6d39\") " pod="openshift-network-diagnostics/network-check-target-6kmtd" Apr 16 16:48:04.217807 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:04.217741 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 16 16:48:04.217807 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:04.217758 2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 16 16:48:04.217807 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:04.217766 2574 projected.go:194] Error preparing data for projected volume kube-api-access-8srq7 for pod openshift-network-diagnostics/network-check-target-6kmtd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 16:48:04.218009 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:04.217820 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d844889-4f54-47ae-8c62-03b86abb6d39-kube-api-access-8srq7 podName:1d844889-4f54-47ae-8c62-03b86abb6d39 nodeName:}" failed. No retries permitted until 2026-04-16 16:48:20.217801808 +0000 UTC m=+34.299536000 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8srq7" (UniqueName: "kubernetes.io/projected/1d844889-4f54-47ae-8c62-03b86abb6d39-kube-api-access-8srq7") pod "network-check-target-6kmtd" (UID: "1d844889-4f54-47ae-8c62-03b86abb6d39") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 16:48:04.507634 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:04.507516 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kmtd" Apr 16 16:48:04.507634 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:04.507582 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:48:04.507846 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:04.507686 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kmtd" podUID="1d844889-4f54-47ae-8c62-03b86abb6d39" Apr 16 16:48:04.507846 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:04.507817 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfxzj" podUID="d2114257-06f1-4070-ad8d-afce484cc205" Apr 16 16:48:06.509147 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:06.508723 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kmtd" Apr 16 16:48:06.509834 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:06.508790 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:48:06.509834 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:06.509330 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfxzj" podUID="d2114257-06f1-4070-ad8d-afce484cc205" Apr 16 16:48:06.509834 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:06.509218 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kmtd" podUID="1d844889-4f54-47ae-8c62-03b86abb6d39" Apr 16 16:48:06.609738 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:06.609701 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xn9cg" event={"ID":"62b3fae0-34f0-4dab-871f-12838879bf70","Type":"ContainerStarted","Data":"89f907a09a439f81cf78325b2458f5717f2e07b36a302a08a807b16024f7f93a"} Apr 16 16:48:06.611441 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:06.611165 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4k9v9" event={"ID":"d655ba61-fb76-4e4b-bbef-f9cfca4b8495","Type":"ContainerStarted","Data":"540037f30adf747c4b4b7b4964fae9aaad6c83a8ac193f786f238aa72c1fe51e"} Apr 16 16:48:06.612760 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:06.612729 2574 generic.go:358] "Generic (PLEG): container finished" podID="fa3397b8-3244-4c47-bf40-735b4b386fb5" containerID="3a6c5038a9a7e8983f6c11678fca673376046ebd5d8e67a70cb59e1757d927ba" exitCode=0 Apr 16 16:48:06.612910 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:06.612811 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kfh7g" event={"ID":"fa3397b8-3244-4c47-bf40-735b4b386fb5","Type":"ContainerDied","Data":"3a6c5038a9a7e8983f6c11678fca673376046ebd5d8e67a70cb59e1757d927ba"} Apr 16 16:48:06.614408 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:06.614296 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mx85v" event={"ID":"29ee980f-ef4e-416f-a7db-2b23ef70873e","Type":"ContainerStarted","Data":"69ca53c555c6ab68ffb3f21a5fdd72b9ace3859a6a80418e997b83267beb0b14"} Apr 16 16:48:06.617496 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:06.617473 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" event={"ID":"5ec5cd46-2ab4-4b8a-961e-22cad6381d66","Type":"ContainerStarted","Data":"24ba2632d9f0526eb70ccc692625215bc4a1b8ac89f984de9e90f41f34dd5a89"} Apr 16 16:48:06.617605 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:06.617503 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" event={"ID":"5ec5cd46-2ab4-4b8a-961e-22cad6381d66","Type":"ContainerStarted","Data":"dfbde22a05a28697a9e1cc855124f0584713c9c911c4f8f93da29480076c766d"} Apr 16 16:48:06.617605 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:06.617525 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" event={"ID":"5ec5cd46-2ab4-4b8a-961e-22cad6381d66","Type":"ContainerStarted","Data":"262536df66d4a279fd23b9f124d07186d48cec1ac80ac5cdf59f4089fbf9af9c"} Apr 16 16:48:06.617605 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:06.617538 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" event={"ID":"5ec5cd46-2ab4-4b8a-961e-22cad6381d66","Type":"ContainerStarted","Data":"51eef5fc8a0fa2f48fb5d4f254fc7c694af1387352bf95a2e6699b7ba1c3aa46"} Apr 16 16:48:06.617605 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:06.617551 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" event={"ID":"5ec5cd46-2ab4-4b8a-961e-22cad6381d66","Type":"ContainerStarted","Data":"392bddd24242811bdd69210e87d1156c5b65fc7ee0ac1c05b0a2bc40201929b2"} Apr 16 16:48:06.617605 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:06.617563 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" event={"ID":"5ec5cd46-2ab4-4b8a-961e-22cad6381d66","Type":"ContainerStarted","Data":"fe8752e5c67db881f51ddf31d824f1fe5f3d611529610e0814df2c608b9b30ba"} Apr 16 16:48:06.618888 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:06.618864 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wtvc6" event={"ID":"b7ff6bde-0c70-4148-bec9-b41af34740f1","Type":"ContainerStarted","Data":"1fbb998d07158bfc4c4d4de90013efffb15875dc01912cd49d749d97f5eeab1f"} Apr 16 16:48:06.620470 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:06.620447 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" event={"ID":"d4923458-dd09-4ed9-877f-7223efd21713","Type":"ContainerStarted","Data":"52fb495f99671ea4116b4010cb1376aecbc6d44673a1b578b99858b599f8a255"} Apr 16 16:48:06.621845 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:06.621821 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5zftw" event={"ID":"8eb081ed-d37b-42a9-a3c9-8549bcb2da0d","Type":"ContainerStarted","Data":"e17b650b6c58404851d8709d777632d59986fc3fd4884f4a438a93fd68d3e1e4"} Apr 16 16:48:06.628282 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:06.628237 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-49.ec2.internal" podStartSLOduration=19.628223215 podStartE2EDuration="19.628223215s" podCreationTimestamp="2026-04-16 16:47:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 16:47:51.588156933 +0000 UTC m=+5.669891151" watchObservedRunningTime="2026-04-16 16:48:06.628223215 +0000 UTC m=+20.709957427" Apr 16 16:48:06.628375 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:06.628338 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-4k9v9" podStartSLOduration=4.126414171 podStartE2EDuration="20.628330713s" podCreationTimestamp="2026-04-16 16:47:46 +0000 UTC" firstStartedPulling="2026-04-16 16:47:49.103462982 +0000 UTC m=+3.185197174" lastFinishedPulling="2026-04-16 16:48:05.605379523 +0000 UTC m=+19.687113716" observedRunningTime="2026-04-16 16:48:06.628139804 +0000 UTC m=+20.709874018" watchObservedRunningTime="2026-04-16 16:48:06.628330713 +0000 UTC m=+20.710064941" Apr 16 16:48:06.641814 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:06.641779 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-5zftw" podStartSLOduration=4.138365492 podStartE2EDuration="20.641767831s" podCreationTimestamp="2026-04-16 16:47:46 +0000 UTC" firstStartedPulling="2026-04-16 16:47:49.093756252 +0000 UTC m=+3.175490448" lastFinishedPulling="2026-04-16 16:48:05.597158581 +0000 UTC m=+19.678892787" observedRunningTime="2026-04-16 16:48:06.641694173 +0000 UTC m=+20.723428386" watchObservedRunningTime="2026-04-16 16:48:06.641767831 +0000 UTC m=+20.723502044" Apr 16 16:48:06.657977 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:06.657940 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-2m2qc" podStartSLOduration=4.15409789 podStartE2EDuration="20.657929598s" podCreationTimestamp="2026-04-16 16:47:46 +0000 UTC" firstStartedPulling="2026-04-16 16:47:49.095153242 +0000 UTC m=+3.176887438" lastFinishedPulling="2026-04-16 16:48:05.598984939 +0000 UTC m=+19.680719146" observedRunningTime="2026-04-16 16:48:06.657880838 +0000 UTC m=+20.739615051" watchObservedRunningTime="2026-04-16 16:48:06.657929598 +0000 UTC m=+20.739663811" Apr 16 16:48:06.683197 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:06.683181 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wtvc6" Apr 16 16:48:06.683707 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:06.683639 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wtvc6" Apr 16 16:48:06.692120 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:06.692091 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mx85v" podStartSLOduration=4.194102507 podStartE2EDuration="20.692082386s" podCreationTimestamp="2026-04-16 16:47:46 +0000 UTC" firstStartedPulling="2026-04-16 16:47:49.099091026 +0000 UTC m=+3.180825219" lastFinishedPulling="2026-04-16 16:48:05.59707089 +0000 UTC m=+19.678805098" observedRunningTime="2026-04-16 16:48:06.670759903 +0000 UTC m=+20.752494115" watchObservedRunningTime="2026-04-16 16:48:06.692082386 +0000 UTC m=+20.773816598" Apr 16 16:48:06.729193 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:06.729148 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wtvc6" podStartSLOduration=12.143599631 podStartE2EDuration="20.729135512s" podCreationTimestamp="2026-04-16 16:47:46 +0000 UTC" firstStartedPulling="2026-04-16 16:47:49.098577415 +0000 UTC m=+3.180311624" lastFinishedPulling="2026-04-16 16:47:57.6841133 +0000 UTC m=+11.765847505" observedRunningTime="2026-04-16 16:48:06.7088351 +0000 UTC m=+20.790569325" watchObservedRunningTime="2026-04-16 16:48:06.729135512 +0000 UTC m=+20.810869725" Apr 16 16:48:06.794907 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:06.794888 2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Apr 16 16:48:07.437843 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:07.437723 2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-04-16T16:48:06.794903095Z","UUID":"062fd970-2ee7-430e-a531-6bba03f1e006","Handler":null,"Name":"","Endpoint":""} Apr 16 16:48:07.439784 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:07.439751 2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Apr 16 16:48:07.439784 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:07.439784 2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Apr 16 16:48:07.625293 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:07.625267 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xn9cg" event={"ID":"62b3fae0-34f0-4dab-871f-12838879bf70","Type":"ContainerStarted","Data":"d9e11e75994ecf08a1dfa1526476418d769df3378bee538450cacbaf3b28e6e5"} Apr 16 16:48:07.626874 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:07.626673 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-66wx7" event={"ID":"a63483e4-ee18-41f8-b36c-45b8cb789405","Type":"ContainerStarted","Data":"0e0f78be94ba17ef0b3f3974629516c2c32f2527c57efd779eb2631559429e77"} Apr 16 16:48:07.627009 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:07.626990 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wtvc6" Apr 16 16:48:07.627461 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:07.627438 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wtvc6" Apr 16 16:48:07.643319 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:07.643285 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-66wx7" podStartSLOduration=5.147772295 podStartE2EDuration="21.643275206s" podCreationTimestamp="2026-04-16 16:47:46 +0000 UTC" firstStartedPulling="2026-04-16 16:47:49.101866577 +0000 UTC m=+3.183600781" lastFinishedPulling="2026-04-16 16:48:05.59736949 +0000 UTC m=+19.679103692" observedRunningTime="2026-04-16 16:48:07.642869601 +0000 UTC m=+21.724603814" watchObservedRunningTime="2026-04-16 16:48:07.643275206 +0000 UTC m=+21.725009420" Apr 16 16:48:08.507941 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:08.507915 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kmtd" Apr 16 16:48:08.508066 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:08.508038 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kmtd" podUID="1d844889-4f54-47ae-8c62-03b86abb6d39" Apr 16 16:48:08.508135 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:08.508094 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:48:08.508327 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:08.508283 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfxzj" podUID="d2114257-06f1-4070-ad8d-afce484cc205" Apr 16 16:48:08.630952 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:08.630912 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xn9cg" event={"ID":"62b3fae0-34f0-4dab-871f-12838879bf70","Type":"ContainerStarted","Data":"4fcdb00c8c93976cedefb34beaa846d0280dfbec8d902a39f94ee41e84069dd8"} Apr 16 16:48:08.634162 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:08.634135 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" event={"ID":"5ec5cd46-2ab4-4b8a-961e-22cad6381d66","Type":"ContainerStarted","Data":"ec6f2e230b9f354d10fb5e757c772958374b689fc954f1c17287bd6c0f1ec01f"} Apr 16 16:48:08.650039 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:08.649996 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xn9cg" podStartSLOduration=4.236778547 podStartE2EDuration="22.649980764s" podCreationTimestamp="2026-04-16 16:47:46 +0000 UTC" firstStartedPulling="2026-04-16 16:47:49.104855341 +0000 UTC m=+3.186589537" lastFinishedPulling="2026-04-16 16:48:07.518057549 +0000 UTC m=+21.599791754" observedRunningTime="2026-04-16 16:48:08.649724509 +0000 UTC m=+22.731458723" watchObservedRunningTime="2026-04-16 16:48:08.649980764 +0000 UTC m=+22.731714979" Apr 16 16:48:10.507934 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:10.507782 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:48:10.508328 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:10.507790 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kmtd" Apr 16 16:48:10.508328 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:10.508030 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfxzj" podUID="d2114257-06f1-4070-ad8d-afce484cc205" Apr 16 16:48:10.508328 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:10.508109 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kmtd" podUID="1d844889-4f54-47ae-8c62-03b86abb6d39" Apr 16 16:48:10.639262 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:10.639226 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kfh7g" event={"ID":"fa3397b8-3244-4c47-bf40-735b4b386fb5","Type":"ContainerStarted","Data":"8f29fa77a972fff5ec0342e661e7e38c98c1c1042932d79aa0c7f39a6c3f794d"} Apr 16 16:48:11.641948 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:11.641913 2574 generic.go:358] "Generic (PLEG): container finished" podID="fa3397b8-3244-4c47-bf40-735b4b386fb5" containerID="8f29fa77a972fff5ec0342e661e7e38c98c1c1042932d79aa0c7f39a6c3f794d" exitCode=0 Apr 16 16:48:11.642792 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:11.641982 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kfh7g" event={"ID":"fa3397b8-3244-4c47-bf40-735b4b386fb5","Type":"ContainerDied","Data":"8f29fa77a972fff5ec0342e661e7e38c98c1c1042932d79aa0c7f39a6c3f794d"} Apr 16 16:48:11.645181 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:11.645154 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" event={"ID":"5ec5cd46-2ab4-4b8a-961e-22cad6381d66","Type":"ContainerStarted","Data":"45d72ce172312cca025123bd814a0e900e25234a84d691f367b33b1675da8eab"} Apr 16 16:48:11.645520 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:11.645500 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:48:11.659638 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:11.659619 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:48:11.689576 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:11.689533 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" podStartSLOduration=8.902117965 podStartE2EDuration="25.689523578s" podCreationTimestamp="2026-04-16 16:47:46 +0000 UTC" firstStartedPulling="2026-04-16 16:47:49.100402284 +0000 UTC m=+3.182136484" lastFinishedPulling="2026-04-16 16:48:05.887807901 +0000 UTC m=+19.969542097" observedRunningTime="2026-04-16 16:48:11.688737734 +0000 UTC m=+25.770471947" watchObservedRunningTime="2026-04-16 16:48:11.689523578 +0000 UTC m=+25.771257792" Apr 16 16:48:12.507599 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:12.507429 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kmtd" Apr 16 16:48:12.507765 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:12.507482 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:48:12.507765 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:12.507692 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kmtd" podUID="1d844889-4f54-47ae-8c62-03b86abb6d39" Apr 16 16:48:12.507765 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:12.507731 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfxzj" podUID="d2114257-06f1-4070-ad8d-afce484cc205" Apr 16 16:48:12.562782 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:12.562753 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6kmtd"] Apr 16 16:48:12.574848 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:12.574823 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kfxzj"] Apr 16 16:48:12.646606 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:12.646583 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:48:12.647002 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:12.646668 2574 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Apr 16 16:48:12.647002 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:12.646678 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfxzj" podUID="d2114257-06f1-4070-ad8d-afce484cc205" Apr 16 16:48:12.647002 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:12.646591 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kmtd" Apr 16 16:48:12.647119 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:12.646992 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kmtd" podUID="1d844889-4f54-47ae-8c62-03b86abb6d39" Apr 16 16:48:12.647119 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:12.647002 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:48:12.661067 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:12.661043 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:48:13.649417 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:13.649385 2574 generic.go:358] "Generic (PLEG): container finished" podID="fa3397b8-3244-4c47-bf40-735b4b386fb5" containerID="cc2a7937a57aeb3d036dcdfbf865b5eecad04fa70f17f2c9e7358aadbf7ba60b" exitCode=0 Apr 16 16:48:13.649830 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:13.649456 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kfh7g" event={"ID":"fa3397b8-3244-4c47-bf40-735b4b386fb5","Type":"ContainerDied","Data":"cc2a7937a57aeb3d036dcdfbf865b5eecad04fa70f17f2c9e7358aadbf7ba60b"} Apr 16 16:48:13.649830 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:13.649669 2574 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Apr 16 16:48:14.507540 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:14.507506 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:48:14.507686 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:14.507549 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kmtd" Apr 16 16:48:14.507686 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:14.507609 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfxzj" podUID="d2114257-06f1-4070-ad8d-afce484cc205" Apr 16 16:48:14.507767 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:14.507723 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kmtd" podUID="1d844889-4f54-47ae-8c62-03b86abb6d39" Apr 16 16:48:14.651960 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:14.651846 2574 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Apr 16 16:48:15.655214 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:15.655173 2574 generic.go:358] "Generic (PLEG): container finished" podID="fa3397b8-3244-4c47-bf40-735b4b386fb5" containerID="9c50b64349b272f687b7de20bdc6b2347b3ea801fce077ec8c90ee8f0529f760" exitCode=0 Apr 16 16:48:15.655587 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:15.655235 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kfh7g" event={"ID":"fa3397b8-3244-4c47-bf40-735b4b386fb5","Type":"ContainerDied","Data":"9c50b64349b272f687b7de20bdc6b2347b3ea801fce077ec8c90ee8f0529f760"} Apr 16 16:48:15.886262 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:15.886233 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:48:15.886465 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:15.886451 2574 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Apr 16 16:48:15.903793 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:15.903746 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" podUID="5ec5cd46-2ab4-4b8a-961e-22cad6381d66" containerName="ovnkube-controller" probeResult="failure" output="" Apr 16 16:48:15.911897 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:15.911838 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" podUID="5ec5cd46-2ab4-4b8a-961e-22cad6381d66" containerName="ovnkube-controller" probeResult="failure" output="" Apr 16 16:48:16.508368 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:16.508152 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kmtd" Apr 16 16:48:16.508534 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:16.508267 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:48:16.508534 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:16.508436 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kmtd" podUID="1d844889-4f54-47ae-8c62-03b86abb6d39" Apr 16 16:48:16.508534 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:16.508512 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfxzj" podUID="d2114257-06f1-4070-ad8d-afce484cc205" Apr 16 16:48:18.227072 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.227044 2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-140-49.ec2.internal" event="NodeReady" Apr 16 16:48:18.227464 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.227172 2574 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Apr 16 16:48:18.280308 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.280277 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gk5x8"] Apr 16 16:48:18.314682 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.314652 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8kc47"] Apr 16 16:48:18.314845 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.314750 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gk5x8" Apr 16 16:48:18.317574 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.317548 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Apr 16 16:48:18.317705 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.317586 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-99nwp\"" Apr 16 16:48:18.317705 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.317590 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Apr 16 16:48:18.329925 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.329904 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8kc47"] Apr 16 16:48:18.329925 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.329927 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gk5x8"] Apr 16 16:48:18.330064 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.330016 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8kc47" Apr 16 16:48:18.332752 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.332731 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-c25vw\"" Apr 16 16:48:18.332857 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.332794 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Apr 16 16:48:18.332857 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.332801 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Apr 16 16:48:18.332857 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.332816 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Apr 16 16:48:18.425578 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.425499 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c897j\" (UniqueName: \"kubernetes.io/projected/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-kube-api-access-c897j\") pod \"ingress-canary-8kc47\" (UID: \"a5cf9cec-6008-46d4-9c84-74af0d3ff9c7\") " pod="openshift-ingress-canary/ingress-canary-8kc47" Apr 16 16:48:18.425578 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.425543 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bvqj\" (UniqueName: \"kubernetes.io/projected/430f1690-8090-4eaa-903e-a5aab4c24146-kube-api-access-5bvqj\") pod \"dns-default-gk5x8\" (UID: \"430f1690-8090-4eaa-903e-a5aab4c24146\") " pod="openshift-dns/dns-default-gk5x8" Apr 16 16:48:18.425773 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.425587 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-cert\") pod \"ingress-canary-8kc47\" (UID: \"a5cf9cec-6008-46d4-9c84-74af0d3ff9c7\") " pod="openshift-ingress-canary/ingress-canary-8kc47" Apr 16 16:48:18.425773 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.425645 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/430f1690-8090-4eaa-903e-a5aab4c24146-config-volume\") pod \"dns-default-gk5x8\" (UID: \"430f1690-8090-4eaa-903e-a5aab4c24146\") " pod="openshift-dns/dns-default-gk5x8" Apr 16 16:48:18.425773 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.425674 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/430f1690-8090-4eaa-903e-a5aab4c24146-metrics-tls\") pod \"dns-default-gk5x8\" (UID: \"430f1690-8090-4eaa-903e-a5aab4c24146\") " pod="openshift-dns/dns-default-gk5x8" Apr 16 16:48:18.425773 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.425692 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/430f1690-8090-4eaa-903e-a5aab4c24146-tmp-dir\") pod \"dns-default-gk5x8\" (UID: \"430f1690-8090-4eaa-903e-a5aab4c24146\") " pod="openshift-dns/dns-default-gk5x8" Apr 16 16:48:18.507981 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.507950 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:48:18.508258 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.508242 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kmtd" Apr 16 16:48:18.511173 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.511150 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Apr 16 16:48:18.511333 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.511228 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Apr 16 16:48:18.511333 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.511309 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wrbkk\"" Apr 16 16:48:18.511450 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.511232 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Apr 16 16:48:18.511450 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.511230 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-877l2\"" Apr 16 16:48:18.526439 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.526419 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c897j\" (UniqueName: \"kubernetes.io/projected/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-kube-api-access-c897j\") pod \"ingress-canary-8kc47\" (UID: \"a5cf9cec-6008-46d4-9c84-74af0d3ff9c7\") " pod="openshift-ingress-canary/ingress-canary-8kc47" Apr 16 16:48:18.526535 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.526458 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5bvqj\" (UniqueName: \"kubernetes.io/projected/430f1690-8090-4eaa-903e-a5aab4c24146-kube-api-access-5bvqj\") pod \"dns-default-gk5x8\" (UID: \"430f1690-8090-4eaa-903e-a5aab4c24146\") " pod="openshift-dns/dns-default-gk5x8" Apr 16 16:48:18.526674 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.526657 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-cert\") pod \"ingress-canary-8kc47\" (UID: \"a5cf9cec-6008-46d4-9c84-74af0d3ff9c7\") " pod="openshift-ingress-canary/ingress-canary-8kc47" Apr 16 16:48:18.526740 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.526691 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/430f1690-8090-4eaa-903e-a5aab4c24146-config-volume\") pod \"dns-default-gk5x8\" (UID: \"430f1690-8090-4eaa-903e-a5aab4c24146\") " pod="openshift-dns/dns-default-gk5x8" Apr 16 16:48:18.526740 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.526718 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/430f1690-8090-4eaa-903e-a5aab4c24146-metrics-tls\") pod \"dns-default-gk5x8\" (UID: \"430f1690-8090-4eaa-903e-a5aab4c24146\") " pod="openshift-dns/dns-default-gk5x8" Apr 16 16:48:18.526823 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.526743 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/430f1690-8090-4eaa-903e-a5aab4c24146-tmp-dir\") pod \"dns-default-gk5x8\" (UID: \"430f1690-8090-4eaa-903e-a5aab4c24146\") " pod="openshift-dns/dns-default-gk5x8" Apr 16 16:48:18.526823 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:18.526789 2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 16:48:18.526881 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:18.526827 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 16:48:18.526881 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:18.526848 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-cert podName:a5cf9cec-6008-46d4-9c84-74af0d3ff9c7 nodeName:}" failed. No retries permitted until 2026-04-16 16:48:19.026828824 +0000 UTC m=+33.108563032 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-cert") pod "ingress-canary-8kc47" (UID: "a5cf9cec-6008-46d4-9c84-74af0d3ff9c7") : secret "canary-serving-cert" not found Apr 16 16:48:18.526881 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:18.526873 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/430f1690-8090-4eaa-903e-a5aab4c24146-metrics-tls podName:430f1690-8090-4eaa-903e-a5aab4c24146 nodeName:}" failed. No retries permitted until 2026-04-16 16:48:19.026859 +0000 UTC m=+33.108593193 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/430f1690-8090-4eaa-903e-a5aab4c24146-metrics-tls") pod "dns-default-gk5x8" (UID: "430f1690-8090-4eaa-903e-a5aab4c24146") : secret "dns-default-metrics-tls" not found Apr 16 16:48:18.527066 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.527050 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/430f1690-8090-4eaa-903e-a5aab4c24146-tmp-dir\") pod \"dns-default-gk5x8\" (UID: \"430f1690-8090-4eaa-903e-a5aab4c24146\") " pod="openshift-dns/dns-default-gk5x8" Apr 16 16:48:18.527281 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.527264 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/430f1690-8090-4eaa-903e-a5aab4c24146-config-volume\") pod \"dns-default-gk5x8\" (UID: \"430f1690-8090-4eaa-903e-a5aab4c24146\") " pod="openshift-dns/dns-default-gk5x8" Apr 16 16:48:18.536805 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.536782 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bvqj\" (UniqueName: \"kubernetes.io/projected/430f1690-8090-4eaa-903e-a5aab4c24146-kube-api-access-5bvqj\") pod \"dns-default-gk5x8\" (UID: \"430f1690-8090-4eaa-903e-a5aab4c24146\") " pod="openshift-dns/dns-default-gk5x8" Apr 16 16:48:18.536893 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:18.536847 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c897j\" (UniqueName: \"kubernetes.io/projected/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-kube-api-access-c897j\") pod \"ingress-canary-8kc47\" (UID: \"a5cf9cec-6008-46d4-9c84-74af0d3ff9c7\") " pod="openshift-ingress-canary/ingress-canary-8kc47" Apr 16 16:48:19.029990 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:19.029962 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-cert\") pod \"ingress-canary-8kc47\" (UID: \"a5cf9cec-6008-46d4-9c84-74af0d3ff9c7\") " pod="openshift-ingress-canary/ingress-canary-8kc47" Apr 16 16:48:19.030179 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:19.029996 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/430f1690-8090-4eaa-903e-a5aab4c24146-metrics-tls\") pod \"dns-default-gk5x8\" (UID: \"430f1690-8090-4eaa-903e-a5aab4c24146\") " pod="openshift-dns/dns-default-gk5x8" Apr 16 16:48:19.030179 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:19.030101 2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 16:48:19.030179 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:19.030110 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 16:48:19.030179 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:19.030167 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-cert podName:a5cf9cec-6008-46d4-9c84-74af0d3ff9c7 nodeName:}" failed. No retries permitted until 2026-04-16 16:48:20.030152515 +0000 UTC m=+34.111886708 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-cert") pod "ingress-canary-8kc47" (UID: "a5cf9cec-6008-46d4-9c84-74af0d3ff9c7") : secret "canary-serving-cert" not found Apr 16 16:48:19.030179 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:19.030181 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/430f1690-8090-4eaa-903e-a5aab4c24146-metrics-tls podName:430f1690-8090-4eaa-903e-a5aab4c24146 nodeName:}" failed. No retries permitted until 2026-04-16 16:48:20.030175308 +0000 UTC m=+34.111909500 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/430f1690-8090-4eaa-903e-a5aab4c24146-metrics-tls") pod "dns-default-gk5x8" (UID: "430f1690-8090-4eaa-903e-a5aab4c24146") : secret "dns-default-metrics-tls" not found Apr 16 16:48:20.036460 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:20.036423 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-cert\") pod \"ingress-canary-8kc47\" (UID: \"a5cf9cec-6008-46d4-9c84-74af0d3ff9c7\") " pod="openshift-ingress-canary/ingress-canary-8kc47" Apr 16 16:48:20.037133 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:20.036471 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/430f1690-8090-4eaa-903e-a5aab4c24146-metrics-tls\") pod \"dns-default-gk5x8\" (UID: \"430f1690-8090-4eaa-903e-a5aab4c24146\") " pod="openshift-dns/dns-default-gk5x8" Apr 16 16:48:20.037133 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:20.036516 2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 16:48:20.037133 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:20.036587 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 16:48:20.037133 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:20.036606 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-cert podName:a5cf9cec-6008-46d4-9c84-74af0d3ff9c7 nodeName:}" failed. No retries permitted until 2026-04-16 16:48:22.036584752 +0000 UTC m=+36.118318944 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-cert") pod "ingress-canary-8kc47" (UID: "a5cf9cec-6008-46d4-9c84-74af0d3ff9c7") : secret "canary-serving-cert" not found Apr 16 16:48:20.037133 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:20.036630 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/430f1690-8090-4eaa-903e-a5aab4c24146-metrics-tls podName:430f1690-8090-4eaa-903e-a5aab4c24146 nodeName:}" failed. No retries permitted until 2026-04-16 16:48:22.036616412 +0000 UTC m=+36.118350606 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/430f1690-8090-4eaa-903e-a5aab4c24146-metrics-tls") pod "dns-default-gk5x8" (UID: "430f1690-8090-4eaa-903e-a5aab4c24146") : secret "dns-default-metrics-tls" not found Apr 16 16:48:20.136924 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:20.136887 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs\") pod \"network-metrics-daemon-kfxzj\" (UID: \"d2114257-06f1-4070-ad8d-afce484cc205\") " pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:48:20.137113 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:20.137050 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Apr 16 16:48:20.137178 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:20.137125 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs podName:d2114257-06f1-4070-ad8d-afce484cc205 nodeName:}" failed. No retries permitted until 2026-04-16 16:48:52.137105738 +0000 UTC m=+66.218839945 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs") pod "network-metrics-daemon-kfxzj" (UID: "d2114257-06f1-4070-ad8d-afce484cc205") : secret "metrics-daemon-secret" not found Apr 16 16:48:20.237582 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:20.237540 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8srq7\" (UniqueName: \"kubernetes.io/projected/1d844889-4f54-47ae-8c62-03b86abb6d39-kube-api-access-8srq7\") pod \"network-check-target-6kmtd\" (UID: \"1d844889-4f54-47ae-8c62-03b86abb6d39\") " pod="openshift-network-diagnostics/network-check-target-6kmtd" Apr 16 16:48:20.240077 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:20.240053 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8srq7\" (UniqueName: \"kubernetes.io/projected/1d844889-4f54-47ae-8c62-03b86abb6d39-kube-api-access-8srq7\") pod \"network-check-target-6kmtd\" (UID: \"1d844889-4f54-47ae-8c62-03b86abb6d39\") " pod="openshift-network-diagnostics/network-check-target-6kmtd" Apr 16 16:48:20.325364 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:20.325285 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kmtd" Apr 16 16:48:21.306774 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:21.306746 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6kmtd"] Apr 16 16:48:21.424408 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:48:21.424187 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d844889_4f54_47ae_8c62_03b86abb6d39.slice/crio-5554e92bfda16ac82101f886207fb3f05d25a78da22d150630074d7a2d67d4d2 WatchSource:0}: Error finding container 5554e92bfda16ac82101f886207fb3f05d25a78da22d150630074d7a2d67d4d2: Status 404 returned error can't find the container with id 5554e92bfda16ac82101f886207fb3f05d25a78da22d150630074d7a2d67d4d2 Apr 16 16:48:21.673672 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:21.673636 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kfh7g" event={"ID":"fa3397b8-3244-4c47-bf40-735b4b386fb5","Type":"ContainerStarted","Data":"c924f2754cb9aefb1359d0c10f4442c0622494141a9e0ea0ed7548783bbca91d"} Apr 16 16:48:21.674722 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:21.674699 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6kmtd" event={"ID":"1d844889-4f54-47ae-8c62-03b86abb6d39","Type":"ContainerStarted","Data":"5554e92bfda16ac82101f886207fb3f05d25a78da22d150630074d7a2d67d4d2"} Apr 16 16:48:22.053730 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:22.053643 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-cert\") pod \"ingress-canary-8kc47\" (UID: \"a5cf9cec-6008-46d4-9c84-74af0d3ff9c7\") " pod="openshift-ingress-canary/ingress-canary-8kc47" Apr 16 16:48:22.053730 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:22.053691 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/430f1690-8090-4eaa-903e-a5aab4c24146-metrics-tls\") pod \"dns-default-gk5x8\" (UID: \"430f1690-8090-4eaa-903e-a5aab4c24146\") " pod="openshift-dns/dns-default-gk5x8" Apr 16 16:48:22.053923 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:22.053805 2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 16:48:22.053923 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:22.053852 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 16:48:22.053923 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:22.053884 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-cert podName:a5cf9cec-6008-46d4-9c84-74af0d3ff9c7 nodeName:}" failed. No retries permitted until 2026-04-16 16:48:26.05386115 +0000 UTC m=+40.135595344 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-cert") pod "ingress-canary-8kc47" (UID: "a5cf9cec-6008-46d4-9c84-74af0d3ff9c7") : secret "canary-serving-cert" not found Apr 16 16:48:22.053923 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:22.053914 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/430f1690-8090-4eaa-903e-a5aab4c24146-metrics-tls podName:430f1690-8090-4eaa-903e-a5aab4c24146 nodeName:}" failed. No retries permitted until 2026-04-16 16:48:26.053900414 +0000 UTC m=+40.135634606 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/430f1690-8090-4eaa-903e-a5aab4c24146-metrics-tls") pod "dns-default-gk5x8" (UID: "430f1690-8090-4eaa-903e-a5aab4c24146") : secret "dns-default-metrics-tls" not found Apr 16 16:48:22.679658 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:22.679623 2574 generic.go:358] "Generic (PLEG): container finished" podID="fa3397b8-3244-4c47-bf40-735b4b386fb5" containerID="c924f2754cb9aefb1359d0c10f4442c0622494141a9e0ea0ed7548783bbca91d" exitCode=0 Apr 16 16:48:22.680012 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:22.679674 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kfh7g" event={"ID":"fa3397b8-3244-4c47-bf40-735b4b386fb5","Type":"ContainerDied","Data":"c924f2754cb9aefb1359d0c10f4442c0622494141a9e0ea0ed7548783bbca91d"} Apr 16 16:48:23.684134 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:23.684100 2574 generic.go:358] "Generic (PLEG): container finished" podID="fa3397b8-3244-4c47-bf40-735b4b386fb5" containerID="f7aa20aacd18071e3d90687d5613616f1b27d154726b4b288cd7f62d888b5fb7" exitCode=0 Apr 16 16:48:23.684575 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:23.684150 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kfh7g" event={"ID":"fa3397b8-3244-4c47-bf40-735b4b386fb5","Type":"ContainerDied","Data":"f7aa20aacd18071e3d90687d5613616f1b27d154726b4b288cd7f62d888b5fb7"} Apr 16 16:48:24.687077 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:24.687044 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6kmtd" event={"ID":"1d844889-4f54-47ae-8c62-03b86abb6d39","Type":"ContainerStarted","Data":"b9b4b9a3f2590cf688abbb8e96649d7ce4c1dbfa12f99fa7c4c0e642a9a84582"} Apr 16 16:48:24.687523 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:24.687175 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6kmtd" Apr 16 16:48:24.690050 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:24.690026 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kfh7g" event={"ID":"fa3397b8-3244-4c47-bf40-735b4b386fb5","Type":"ContainerStarted","Data":"f3821876268b9307986492d02cef03652659fb0261239aa613b650b9cfa4b91d"} Apr 16 16:48:24.703390 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:24.703348 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6kmtd" podStartSLOduration=35.798184681 podStartE2EDuration="38.703336844s" podCreationTimestamp="2026-04-16 16:47:46 +0000 UTC" firstStartedPulling="2026-04-16 16:48:21.44037665 +0000 UTC m=+35.522110842" lastFinishedPulling="2026-04-16 16:48:24.345528794 +0000 UTC m=+38.427263005" observedRunningTime="2026-04-16 16:48:24.702393038 +0000 UTC m=+38.784127253" watchObservedRunningTime="2026-04-16 16:48:24.703336844 +0000 UTC m=+38.785071057" Apr 16 16:48:24.733182 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:24.733143 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-kfh7g" podStartSLOduration=6.371311617 podStartE2EDuration="38.733131741s" podCreationTimestamp="2026-04-16 16:47:46 +0000 UTC" firstStartedPulling="2026-04-16 16:47:49.101427319 +0000 UTC m=+3.183161515" lastFinishedPulling="2026-04-16 16:48:21.463247442 +0000 UTC m=+35.544981639" observedRunningTime="2026-04-16 16:48:24.732819606 +0000 UTC m=+38.814553819" watchObservedRunningTime="2026-04-16 16:48:24.733131741 +0000 UTC m=+38.814865954" Apr 16 16:48:26.080389 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:26.080356 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-cert\") pod \"ingress-canary-8kc47\" (UID: \"a5cf9cec-6008-46d4-9c84-74af0d3ff9c7\") " pod="openshift-ingress-canary/ingress-canary-8kc47" Apr 16 16:48:26.080389 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:26.080390 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/430f1690-8090-4eaa-903e-a5aab4c24146-metrics-tls\") pod \"dns-default-gk5x8\" (UID: \"430f1690-8090-4eaa-903e-a5aab4c24146\") " pod="openshift-dns/dns-default-gk5x8" Apr 16 16:48:26.080794 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:26.080467 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 16:48:26.080794 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:26.080481 2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 16:48:26.080794 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:26.080511 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/430f1690-8090-4eaa-903e-a5aab4c24146-metrics-tls podName:430f1690-8090-4eaa-903e-a5aab4c24146 nodeName:}" failed. No retries permitted until 2026-04-16 16:48:34.080499304 +0000 UTC m=+48.162233496 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/430f1690-8090-4eaa-903e-a5aab4c24146-metrics-tls") pod "dns-default-gk5x8" (UID: "430f1690-8090-4eaa-903e-a5aab4c24146") : secret "dns-default-metrics-tls" not found Apr 16 16:48:26.080794 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:26.080532 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-cert podName:a5cf9cec-6008-46d4-9c84-74af0d3ff9c7 nodeName:}" failed. No retries permitted until 2026-04-16 16:48:34.080520095 +0000 UTC m=+48.162254287 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-cert") pod "ingress-canary-8kc47" (UID: "a5cf9cec-6008-46d4-9c84-74af0d3ff9c7") : secret "canary-serving-cert" not found Apr 16 16:48:34.131282 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:34.131236 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-cert\") pod \"ingress-canary-8kc47\" (UID: \"a5cf9cec-6008-46d4-9c84-74af0d3ff9c7\") " pod="openshift-ingress-canary/ingress-canary-8kc47" Apr 16 16:48:34.131282 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:34.131286 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/430f1690-8090-4eaa-903e-a5aab4c24146-metrics-tls\") pod \"dns-default-gk5x8\" (UID: \"430f1690-8090-4eaa-903e-a5aab4c24146\") " pod="openshift-dns/dns-default-gk5x8" Apr 16 16:48:34.131818 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:34.131390 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 16:48:34.131818 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:34.131392 2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 16:48:34.131818 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:34.131449 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/430f1690-8090-4eaa-903e-a5aab4c24146-metrics-tls podName:430f1690-8090-4eaa-903e-a5aab4c24146 nodeName:}" failed. No retries permitted until 2026-04-16 16:48:50.131436384 +0000 UTC m=+64.213170576 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/430f1690-8090-4eaa-903e-a5aab4c24146-metrics-tls") pod "dns-default-gk5x8" (UID: "430f1690-8090-4eaa-903e-a5aab4c24146") : secret "dns-default-metrics-tls" not found Apr 16 16:48:34.131818 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:34.131462 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-cert podName:a5cf9cec-6008-46d4-9c84-74af0d3ff9c7 nodeName:}" failed. No retries permitted until 2026-04-16 16:48:50.131456708 +0000 UTC m=+64.213190900 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-cert") pod "ingress-canary-8kc47" (UID: "a5cf9cec-6008-46d4-9c84-74af0d3ff9c7") : secret "canary-serving-cert" not found Apr 16 16:48:45.912671 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:45.912641 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hfbd8" Apr 16 16:48:50.231972 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:50.231936 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-cert\") pod \"ingress-canary-8kc47\" (UID: \"a5cf9cec-6008-46d4-9c84-74af0d3ff9c7\") " pod="openshift-ingress-canary/ingress-canary-8kc47" Apr 16 16:48:50.231972 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:50.231974 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/430f1690-8090-4eaa-903e-a5aab4c24146-metrics-tls\") pod \"dns-default-gk5x8\" (UID: \"430f1690-8090-4eaa-903e-a5aab4c24146\") " pod="openshift-dns/dns-default-gk5x8" Apr 16 16:48:50.232364 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:50.232074 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 16:48:50.232364 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:50.232076 2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 16:48:50.232364 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:50.232130 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-cert podName:a5cf9cec-6008-46d4-9c84-74af0d3ff9c7 nodeName:}" failed. No retries permitted until 2026-04-16 16:49:22.23211627 +0000 UTC m=+96.313850463 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-cert") pod "ingress-canary-8kc47" (UID: "a5cf9cec-6008-46d4-9c84-74af0d3ff9c7") : secret "canary-serving-cert" not found Apr 16 16:48:50.232364 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:50.232144 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/430f1690-8090-4eaa-903e-a5aab4c24146-metrics-tls podName:430f1690-8090-4eaa-903e-a5aab4c24146 nodeName:}" failed. No retries permitted until 2026-04-16 16:49:22.232137889 +0000 UTC m=+96.313872082 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/430f1690-8090-4eaa-903e-a5aab4c24146-metrics-tls") pod "dns-default-gk5x8" (UID: "430f1690-8090-4eaa-903e-a5aab4c24146") : secret "dns-default-metrics-tls" not found Apr 16 16:48:52.144756 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:52.144717 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs\") pod \"network-metrics-daemon-kfxzj\" (UID: \"d2114257-06f1-4070-ad8d-afce484cc205\") " pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:48:52.145156 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:52.144824 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Apr 16 16:48:52.145156 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:48:52.144874 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs podName:d2114257-06f1-4070-ad8d-afce484cc205 nodeName:}" failed. No retries permitted until 2026-04-16 16:49:56.144860868 +0000 UTC m=+130.226595061 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs") pod "network-metrics-daemon-kfxzj" (UID: "d2114257-06f1-4070-ad8d-afce484cc205") : secret "metrics-daemon-secret" not found Apr 16 16:48:55.695110 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:48:55.695076 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6kmtd" Apr 16 16:49:22.332646 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:22.332618 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-cert\") pod \"ingress-canary-8kc47\" (UID: \"a5cf9cec-6008-46d4-9c84-74af0d3ff9c7\") " pod="openshift-ingress-canary/ingress-canary-8kc47" Apr 16 16:49:22.332971 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:22.332650 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/430f1690-8090-4eaa-903e-a5aab4c24146-metrics-tls\") pod \"dns-default-gk5x8\" (UID: \"430f1690-8090-4eaa-903e-a5aab4c24146\") " pod="openshift-dns/dns-default-gk5x8" Apr 16 16:49:22.332971 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:49:22.332746 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 16:49:22.332971 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:49:22.332749 2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 16:49:22.332971 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:49:22.332811 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/430f1690-8090-4eaa-903e-a5aab4c24146-metrics-tls podName:430f1690-8090-4eaa-903e-a5aab4c24146 nodeName:}" failed. No retries permitted until 2026-04-16 16:50:26.332789422 +0000 UTC m=+160.414523615 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/430f1690-8090-4eaa-903e-a5aab4c24146-metrics-tls") pod "dns-default-gk5x8" (UID: "430f1690-8090-4eaa-903e-a5aab4c24146") : secret "dns-default-metrics-tls" not found Apr 16 16:49:22.332971 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:49:22.332824 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-cert podName:a5cf9cec-6008-46d4-9c84-74af0d3ff9c7 nodeName:}" failed. No retries permitted until 2026-04-16 16:50:26.33281862 +0000 UTC m=+160.414552812 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-cert") pod "ingress-canary-8kc47" (UID: "a5cf9cec-6008-46d4-9c84-74af0d3ff9c7") : secret "canary-serving-cert" not found Apr 16 16:49:45.413901 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:45.413868 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5785d4fcdd-tfwxh"] Apr 16 16:49:45.416607 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:45.416589 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5785d4fcdd-tfwxh" Apr 16 16:49:45.419180 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:45.419157 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Apr 16 16:49:45.419180 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:45.419173 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-xsxsz\"" Apr 16 16:49:45.420379 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:45.420351 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Apr 16 16:49:45.420461 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:45.420352 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Apr 16 16:49:45.420461 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:45.420352 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Apr 16 16:49:45.425885 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:45.425862 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5785d4fcdd-tfwxh"] Apr 16 16:49:45.427521 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:45.426352 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Apr 16 16:49:45.487878 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:45.487850 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3ef9470d-4a29-4aa3-aa5d-21af5d523f6f-serving-cert\") pod \"insights-operator-5785d4fcdd-tfwxh\" (UID: \"3ef9470d-4a29-4aa3-aa5d-21af5d523f6f\") " pod="openshift-insights/insights-operator-5785d4fcdd-tfwxh" Apr 16 16:49:45.487981 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:45.487886 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3ef9470d-4a29-4aa3-aa5d-21af5d523f6f-snapshots\") pod \"insights-operator-5785d4fcdd-tfwxh\" (UID: \"3ef9470d-4a29-4aa3-aa5d-21af5d523f6f\") " pod="openshift-insights/insights-operator-5785d4fcdd-tfwxh" Apr 16 16:49:45.487981 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:45.487911 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxm2b\" (UniqueName: \"kubernetes.io/projected/3ef9470d-4a29-4aa3-aa5d-21af5d523f6f-kube-api-access-rxm2b\") pod \"insights-operator-5785d4fcdd-tfwxh\" (UID: \"3ef9470d-4a29-4aa3-aa5d-21af5d523f6f\") " pod="openshift-insights/insights-operator-5785d4fcdd-tfwxh" Apr 16 16:49:45.487981 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:45.487969 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ef9470d-4a29-4aa3-aa5d-21af5d523f6f-tmp\") pod \"insights-operator-5785d4fcdd-tfwxh\" (UID: \"3ef9470d-4a29-4aa3-aa5d-21af5d523f6f\") " pod="openshift-insights/insights-operator-5785d4fcdd-tfwxh" Apr 16 16:49:45.488085 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:45.488026 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ef9470d-4a29-4aa3-aa5d-21af5d523f6f-trusted-ca-bundle\") pod \"insights-operator-5785d4fcdd-tfwxh\" (UID: \"3ef9470d-4a29-4aa3-aa5d-21af5d523f6f\") " pod="openshift-insights/insights-operator-5785d4fcdd-tfwxh" Apr 16 16:49:45.488085 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:45.488045 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ef9470d-4a29-4aa3-aa5d-21af5d523f6f-service-ca-bundle\") pod \"insights-operator-5785d4fcdd-tfwxh\" (UID: \"3ef9470d-4a29-4aa3-aa5d-21af5d523f6f\") " pod="openshift-insights/insights-operator-5785d4fcdd-tfwxh" Apr 16 16:49:45.588438 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:45.588415 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ef9470d-4a29-4aa3-aa5d-21af5d523f6f-trusted-ca-bundle\") pod \"insights-operator-5785d4fcdd-tfwxh\" (UID: \"3ef9470d-4a29-4aa3-aa5d-21af5d523f6f\") " pod="openshift-insights/insights-operator-5785d4fcdd-tfwxh" Apr 16 16:49:45.588509 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:45.588443 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ef9470d-4a29-4aa3-aa5d-21af5d523f6f-service-ca-bundle\") pod \"insights-operator-5785d4fcdd-tfwxh\" (UID: \"3ef9470d-4a29-4aa3-aa5d-21af5d523f6f\") " pod="openshift-insights/insights-operator-5785d4fcdd-tfwxh" Apr 16 16:49:45.588509 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:45.588487 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3ef9470d-4a29-4aa3-aa5d-21af5d523f6f-serving-cert\") pod \"insights-operator-5785d4fcdd-tfwxh\" (UID: \"3ef9470d-4a29-4aa3-aa5d-21af5d523f6f\") " pod="openshift-insights/insights-operator-5785d4fcdd-tfwxh" Apr 16 16:49:45.588591 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:45.588513 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3ef9470d-4a29-4aa3-aa5d-21af5d523f6f-snapshots\") pod \"insights-operator-5785d4fcdd-tfwxh\" (UID: \"3ef9470d-4a29-4aa3-aa5d-21af5d523f6f\") " pod="openshift-insights/insights-operator-5785d4fcdd-tfwxh" Apr 16 16:49:45.588591 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:45.588538 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rxm2b\" (UniqueName: \"kubernetes.io/projected/3ef9470d-4a29-4aa3-aa5d-21af5d523f6f-kube-api-access-rxm2b\") pod \"insights-operator-5785d4fcdd-tfwxh\" (UID: \"3ef9470d-4a29-4aa3-aa5d-21af5d523f6f\") " pod="openshift-insights/insights-operator-5785d4fcdd-tfwxh" Apr 16 16:49:45.588591 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:45.588575 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ef9470d-4a29-4aa3-aa5d-21af5d523f6f-tmp\") pod \"insights-operator-5785d4fcdd-tfwxh\" (UID: \"3ef9470d-4a29-4aa3-aa5d-21af5d523f6f\") " pod="openshift-insights/insights-operator-5785d4fcdd-tfwxh" Apr 16 16:49:45.588878 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:45.588851 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ef9470d-4a29-4aa3-aa5d-21af5d523f6f-tmp\") pod \"insights-operator-5785d4fcdd-tfwxh\" (UID: \"3ef9470d-4a29-4aa3-aa5d-21af5d523f6f\") " pod="openshift-insights/insights-operator-5785d4fcdd-tfwxh" Apr 16 16:49:45.589032 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:45.589013 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ef9470d-4a29-4aa3-aa5d-21af5d523f6f-service-ca-bundle\") pod \"insights-operator-5785d4fcdd-tfwxh\" (UID: \"3ef9470d-4a29-4aa3-aa5d-21af5d523f6f\") " pod="openshift-insights/insights-operator-5785d4fcdd-tfwxh" Apr 16 16:49:45.589163 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:45.589146 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3ef9470d-4a29-4aa3-aa5d-21af5d523f6f-snapshots\") pod \"insights-operator-5785d4fcdd-tfwxh\" (UID: \"3ef9470d-4a29-4aa3-aa5d-21af5d523f6f\") " pod="openshift-insights/insights-operator-5785d4fcdd-tfwxh" Apr 16 16:49:45.589495 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:45.589476 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ef9470d-4a29-4aa3-aa5d-21af5d523f6f-trusted-ca-bundle\") pod \"insights-operator-5785d4fcdd-tfwxh\" (UID: \"3ef9470d-4a29-4aa3-aa5d-21af5d523f6f\") " pod="openshift-insights/insights-operator-5785d4fcdd-tfwxh" Apr 16 16:49:45.592056 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:45.592034 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3ef9470d-4a29-4aa3-aa5d-21af5d523f6f-serving-cert\") pod \"insights-operator-5785d4fcdd-tfwxh\" (UID: \"3ef9470d-4a29-4aa3-aa5d-21af5d523f6f\") " pod="openshift-insights/insights-operator-5785d4fcdd-tfwxh" Apr 16 16:49:45.596547 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:45.596528 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxm2b\" (UniqueName: \"kubernetes.io/projected/3ef9470d-4a29-4aa3-aa5d-21af5d523f6f-kube-api-access-rxm2b\") pod \"insights-operator-5785d4fcdd-tfwxh\" (UID: \"3ef9470d-4a29-4aa3-aa5d-21af5d523f6f\") " pod="openshift-insights/insights-operator-5785d4fcdd-tfwxh" Apr 16 16:49:45.726633 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:45.726584 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5785d4fcdd-tfwxh" Apr 16 16:49:45.832784 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:45.832748 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5785d4fcdd-tfwxh"] Apr 16 16:49:45.836854 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:49:45.836828 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ef9470d_4a29_4aa3_aa5d_21af5d523f6f.slice/crio-66a2d755d37c5441ddadea93b273259e79175efb0c570cf9c09364617bc09aa1 WatchSource:0}: Error finding container 66a2d755d37c5441ddadea93b273259e79175efb0c570cf9c09364617bc09aa1: Status 404 returned error can't find the container with id 66a2d755d37c5441ddadea93b273259e79175efb0c570cf9c09364617bc09aa1 Apr 16 16:49:46.839222 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:46.839173 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5785d4fcdd-tfwxh" event={"ID":"3ef9470d-4a29-4aa3-aa5d-21af5d523f6f","Type":"ContainerStarted","Data":"66a2d755d37c5441ddadea93b273259e79175efb0c570cf9c09364617bc09aa1"} Apr 16 16:49:47.842282 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:47.842191 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5785d4fcdd-tfwxh" event={"ID":"3ef9470d-4a29-4aa3-aa5d-21af5d523f6f","Type":"ContainerStarted","Data":"b378f57c214a293dd8c34065e1d6f8069a828f658b8e9f664b5c382f574425b8"} Apr 16 16:49:47.859101 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:47.859060 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-operator-5785d4fcdd-tfwxh" podStartSLOduration=1.178605088 podStartE2EDuration="2.85904954s" podCreationTimestamp="2026-04-16 16:49:45 +0000 UTC" firstStartedPulling="2026-04-16 16:49:45.838488477 +0000 UTC m=+119.920222670" lastFinishedPulling="2026-04-16 16:49:47.518932926 +0000 UTC m=+121.600667122" observedRunningTime="2026-04-16 16:49:47.8585487 +0000 UTC m=+121.940282913" watchObservedRunningTime="2026-04-16 16:49:47.85904954 +0000 UTC m=+121.940783754" Apr 16 16:49:49.348394 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:49.348363 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-667775844f-l9td2"] Apr 16 16:49:49.351293 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:49.351278 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-667775844f-l9td2" Apr 16 16:49:49.353902 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:49.353879 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Apr 16 16:49:49.354002 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:49.353948 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Apr 16 16:49:49.354002 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:49.353885 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-fdzd2\"" Apr 16 16:49:49.355102 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:49.355085 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Apr 16 16:49:49.361486 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:49.361465 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-667775844f-l9td2"] Apr 16 16:49:49.414413 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:49.414394 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a2bd7515-2313-46a8-8961-28f13c30996b-samples-operator-tls\") pod \"cluster-samples-operator-667775844f-l9td2\" (UID: \"a2bd7515-2313-46a8-8961-28f13c30996b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-667775844f-l9td2" Apr 16 16:49:49.414511 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:49.414420 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrg45\" (UniqueName: \"kubernetes.io/projected/a2bd7515-2313-46a8-8961-28f13c30996b-kube-api-access-wrg45\") pod \"cluster-samples-operator-667775844f-l9td2\" (UID: \"a2bd7515-2313-46a8-8961-28f13c30996b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-667775844f-l9td2" Apr 16 16:49:49.515461 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:49.515441 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a2bd7515-2313-46a8-8961-28f13c30996b-samples-operator-tls\") pod \"cluster-samples-operator-667775844f-l9td2\" (UID: \"a2bd7515-2313-46a8-8961-28f13c30996b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-667775844f-l9td2" Apr 16 16:49:49.515545 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:49.515465 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wrg45\" (UniqueName: \"kubernetes.io/projected/a2bd7515-2313-46a8-8961-28f13c30996b-kube-api-access-wrg45\") pod \"cluster-samples-operator-667775844f-l9td2\" (UID: \"a2bd7515-2313-46a8-8961-28f13c30996b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-667775844f-l9td2" Apr 16 16:49:49.515587 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:49:49.515563 2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Apr 16 16:49:49.515627 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:49:49.515622 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2bd7515-2313-46a8-8961-28f13c30996b-samples-operator-tls podName:a2bd7515-2313-46a8-8961-28f13c30996b nodeName:}" failed. No retries permitted until 2026-04-16 16:49:50.015608475 +0000 UTC m=+124.097342667 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a2bd7515-2313-46a8-8961-28f13c30996b-samples-operator-tls") pod "cluster-samples-operator-667775844f-l9td2" (UID: "a2bd7515-2313-46a8-8961-28f13c30996b") : secret "samples-operator-tls" not found Apr 16 16:49:49.524348 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:49.524321 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrg45\" (UniqueName: \"kubernetes.io/projected/a2bd7515-2313-46a8-8961-28f13c30996b-kube-api-access-wrg45\") pod \"cluster-samples-operator-667775844f-l9td2\" (UID: \"a2bd7515-2313-46a8-8961-28f13c30996b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-667775844f-l9td2" Apr 16 16:49:50.019118 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:50.019084 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a2bd7515-2313-46a8-8961-28f13c30996b-samples-operator-tls\") pod \"cluster-samples-operator-667775844f-l9td2\" (UID: \"a2bd7515-2313-46a8-8961-28f13c30996b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-667775844f-l9td2" Apr 16 16:49:50.019323 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:49:50.019215 2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Apr 16 16:49:50.019323 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:49:50.019289 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2bd7515-2313-46a8-8961-28f13c30996b-samples-operator-tls podName:a2bd7515-2313-46a8-8961-28f13c30996b nodeName:}" failed. No retries permitted until 2026-04-16 16:49:51.019270212 +0000 UTC m=+125.101004417 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a2bd7515-2313-46a8-8961-28f13c30996b-samples-operator-tls") pod "cluster-samples-operator-667775844f-l9td2" (UID: "a2bd7515-2313-46a8-8961-28f13c30996b") : secret "samples-operator-tls" not found Apr 16 16:49:50.344242 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:50.344144 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7d955d5dd4-ss8kp"] Apr 16 16:49:50.346974 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:50.346951 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-7d955d5dd4-ss8kp" Apr 16 16:49:50.349732 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:50.349709 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Apr 16 16:49:50.350961 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:50.350945 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Apr 16 16:49:50.351055 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:50.350947 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-q8s4v\"" Apr 16 16:49:50.353185 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:50.353163 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7d955d5dd4-ss8kp"] Apr 16 16:49:50.421822 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:50.421794 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jmwt\" (UniqueName: \"kubernetes.io/projected/c1529d61-c793-4fc5-977d-5e976290f774-kube-api-access-8jmwt\") pod \"volume-data-source-validator-7d955d5dd4-ss8kp\" (UID: \"c1529d61-c793-4fc5-977d-5e976290f774\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7d955d5dd4-ss8kp" Apr 16 16:49:50.522066 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:50.522037 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8jmwt\" (UniqueName: \"kubernetes.io/projected/c1529d61-c793-4fc5-977d-5e976290f774-kube-api-access-8jmwt\") pod \"volume-data-source-validator-7d955d5dd4-ss8kp\" (UID: \"c1529d61-c793-4fc5-977d-5e976290f774\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7d955d5dd4-ss8kp" Apr 16 16:49:50.531063 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:50.531033 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jmwt\" (UniqueName: \"kubernetes.io/projected/c1529d61-c793-4fc5-977d-5e976290f774-kube-api-access-8jmwt\") pod \"volume-data-source-validator-7d955d5dd4-ss8kp\" (UID: \"c1529d61-c793-4fc5-977d-5e976290f774\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7d955d5dd4-ss8kp" Apr 16 16:49:50.656316 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:50.656281 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-7d955d5dd4-ss8kp" Apr 16 16:49:50.761723 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:50.761695 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7d955d5dd4-ss8kp"] Apr 16 16:49:50.764817 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:49:50.764780 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1529d61_c793_4fc5_977d_5e976290f774.slice/crio-a50a6136a538528216001d7a85be1edae840b72a5c08690ae18a581b55d98288 WatchSource:0}: Error finding container a50a6136a538528216001d7a85be1edae840b72a5c08690ae18a581b55d98288: Status 404 returned error can't find the container with id a50a6136a538528216001d7a85be1edae840b72a5c08690ae18a581b55d98288 Apr 16 16:49:50.848857 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:50.848830 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-7d955d5dd4-ss8kp" event={"ID":"c1529d61-c793-4fc5-977d-5e976290f774","Type":"ContainerStarted","Data":"a50a6136a538528216001d7a85be1edae840b72a5c08690ae18a581b55d98288"} Apr 16 16:49:51.025245 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:51.025152 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a2bd7515-2313-46a8-8961-28f13c30996b-samples-operator-tls\") pod \"cluster-samples-operator-667775844f-l9td2\" (UID: \"a2bd7515-2313-46a8-8961-28f13c30996b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-667775844f-l9td2" Apr 16 16:49:51.025385 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:49:51.025274 2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Apr 16 16:49:51.025385 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:49:51.025329 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2bd7515-2313-46a8-8961-28f13c30996b-samples-operator-tls podName:a2bd7515-2313-46a8-8961-28f13c30996b nodeName:}" failed. No retries permitted until 2026-04-16 16:49:53.025314092 +0000 UTC m=+127.107048284 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a2bd7515-2313-46a8-8961-28f13c30996b-samples-operator-tls") pod "cluster-samples-operator-667775844f-l9td2" (UID: "a2bd7515-2313-46a8-8961-28f13c30996b") : secret "samples-operator-tls" not found Apr 16 16:49:51.116019 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:51.115996 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5zftw_8eb081ed-d37b-42a9-a3c9-8549bcb2da0d/dns-node-resolver/0.log" Apr 16 16:49:51.360537 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:51.360501 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-d87b8d5fc-s6s57"] Apr 16 16:49:51.363583 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:51.363563 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-d87b8d5fc-s6s57" Apr 16 16:49:51.367614 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:51.366745 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Apr 16 16:49:51.367614 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:51.366779 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Apr 16 16:49:51.367614 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:51.367041 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Apr 16 16:49:51.368402 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:51.368376 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Apr 16 16:49:51.368512 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:51.368392 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-xrhw8\"" Apr 16 16:49:51.372539 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:51.372469 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Apr 16 16:49:51.375349 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:51.375326 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-d87b8d5fc-s6s57"] Apr 16 16:49:51.428426 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:51.428398 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3e7b3fa7-ba27-4076-b195-19d14c437d6a-serving-cert\") pod \"console-operator-d87b8d5fc-s6s57\" (UID: \"3e7b3fa7-ba27-4076-b195-19d14c437d6a\") " pod="openshift-console-operator/console-operator-d87b8d5fc-s6s57" Apr 16 16:49:51.428578 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:51.428481 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3e7b3fa7-ba27-4076-b195-19d14c437d6a-config\") pod \"console-operator-d87b8d5fc-s6s57\" (UID: \"3e7b3fa7-ba27-4076-b195-19d14c437d6a\") " pod="openshift-console-operator/console-operator-d87b8d5fc-s6s57" Apr 16 16:49:51.428578 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:51.428511 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgrst\" (UniqueName: \"kubernetes.io/projected/3e7b3fa7-ba27-4076-b195-19d14c437d6a-kube-api-access-dgrst\") pod \"console-operator-d87b8d5fc-s6s57\" (UID: \"3e7b3fa7-ba27-4076-b195-19d14c437d6a\") " pod="openshift-console-operator/console-operator-d87b8d5fc-s6s57" Apr 16 16:49:51.428578 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:51.428539 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3e7b3fa7-ba27-4076-b195-19d14c437d6a-trusted-ca\") pod \"console-operator-d87b8d5fc-s6s57\" (UID: \"3e7b3fa7-ba27-4076-b195-19d14c437d6a\") " pod="openshift-console-operator/console-operator-d87b8d5fc-s6s57" Apr 16 16:49:51.529023 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:51.528990 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3e7b3fa7-ba27-4076-b195-19d14c437d6a-serving-cert\") pod \"console-operator-d87b8d5fc-s6s57\" (UID: \"3e7b3fa7-ba27-4076-b195-19d14c437d6a\") " pod="openshift-console-operator/console-operator-d87b8d5fc-s6s57" Apr 16 16:49:51.529190 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:51.529082 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3e7b3fa7-ba27-4076-b195-19d14c437d6a-config\") pod \"console-operator-d87b8d5fc-s6s57\" (UID: \"3e7b3fa7-ba27-4076-b195-19d14c437d6a\") " pod="openshift-console-operator/console-operator-d87b8d5fc-s6s57" Apr 16 16:49:51.529190 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:51.529115 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dgrst\" (UniqueName: \"kubernetes.io/projected/3e7b3fa7-ba27-4076-b195-19d14c437d6a-kube-api-access-dgrst\") pod \"console-operator-d87b8d5fc-s6s57\" (UID: \"3e7b3fa7-ba27-4076-b195-19d14c437d6a\") " pod="openshift-console-operator/console-operator-d87b8d5fc-s6s57" Apr 16 16:49:51.529190 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:51.529150 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3e7b3fa7-ba27-4076-b195-19d14c437d6a-trusted-ca\") pod \"console-operator-d87b8d5fc-s6s57\" (UID: \"3e7b3fa7-ba27-4076-b195-19d14c437d6a\") " pod="openshift-console-operator/console-operator-d87b8d5fc-s6s57" Apr 16 16:49:51.529842 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:51.529818 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3e7b3fa7-ba27-4076-b195-19d14c437d6a-config\") pod \"console-operator-d87b8d5fc-s6s57\" (UID: \"3e7b3fa7-ba27-4076-b195-19d14c437d6a\") " pod="openshift-console-operator/console-operator-d87b8d5fc-s6s57" Apr 16 16:49:51.529947 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:51.529900 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3e7b3fa7-ba27-4076-b195-19d14c437d6a-trusted-ca\") pod \"console-operator-d87b8d5fc-s6s57\" (UID: \"3e7b3fa7-ba27-4076-b195-19d14c437d6a\") " pod="openshift-console-operator/console-operator-d87b8d5fc-s6s57" Apr 16 16:49:51.532073 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:51.532049 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3e7b3fa7-ba27-4076-b195-19d14c437d6a-serving-cert\") pod \"console-operator-d87b8d5fc-s6s57\" (UID: \"3e7b3fa7-ba27-4076-b195-19d14c437d6a\") " pod="openshift-console-operator/console-operator-d87b8d5fc-s6s57" Apr 16 16:49:51.537738 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:51.537709 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgrst\" (UniqueName: \"kubernetes.io/projected/3e7b3fa7-ba27-4076-b195-19d14c437d6a-kube-api-access-dgrst\") pod \"console-operator-d87b8d5fc-s6s57\" (UID: \"3e7b3fa7-ba27-4076-b195-19d14c437d6a\") " pod="openshift-console-operator/console-operator-d87b8d5fc-s6s57" Apr 16 16:49:51.675808 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:51.675738 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-d87b8d5fc-s6s57" Apr 16 16:49:51.867975 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:51.867947 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-d87b8d5fc-s6s57"] Apr 16 16:49:51.871090 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:49:51.871065 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e7b3fa7_ba27_4076_b195_19d14c437d6a.slice/crio-019684a1230d778c9a6015e1b0a3fb7c43fa4d32da9455eb8a5eb340a2b94177 WatchSource:0}: Error finding container 019684a1230d778c9a6015e1b0a3fb7c43fa4d32da9455eb8a5eb340a2b94177: Status 404 returned error can't find the container with id 019684a1230d778c9a6015e1b0a3fb7c43fa4d32da9455eb8a5eb340a2b94177 Apr 16 16:49:51.913239 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:51.913219 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mx85v_29ee980f-ef4e-416f-a7db-2b23ef70873e/node-ca/0.log" Apr 16 16:49:52.855616 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:52.855574 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-d87b8d5fc-s6s57" event={"ID":"3e7b3fa7-ba27-4076-b195-19d14c437d6a","Type":"ContainerStarted","Data":"019684a1230d778c9a6015e1b0a3fb7c43fa4d32da9455eb8a5eb340a2b94177"} Apr 16 16:49:52.856759 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:52.856738 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-7d955d5dd4-ss8kp" event={"ID":"c1529d61-c793-4fc5-977d-5e976290f774","Type":"ContainerStarted","Data":"566e6b9ad1ac62a0787a2a6862f00e4d690bb8737dd380d5389e8381f2a02c43"} Apr 16 16:49:52.873613 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:52.873573 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-7d955d5dd4-ss8kp" podStartSLOduration=1.84014737 podStartE2EDuration="2.873560286s" podCreationTimestamp="2026-04-16 16:49:50 +0000 UTC" firstStartedPulling="2026-04-16 16:49:50.766694118 +0000 UTC m=+124.848428327" lastFinishedPulling="2026-04-16 16:49:51.800107046 +0000 UTC m=+125.881841243" observedRunningTime="2026-04-16 16:49:52.871784529 +0000 UTC m=+126.953518752" watchObservedRunningTime="2026-04-16 16:49:52.873560286 +0000 UTC m=+126.955294496" Apr 16 16:49:53.040316 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:53.040276 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a2bd7515-2313-46a8-8961-28f13c30996b-samples-operator-tls\") pod \"cluster-samples-operator-667775844f-l9td2\" (UID: \"a2bd7515-2313-46a8-8961-28f13c30996b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-667775844f-l9td2" Apr 16 16:49:53.040503 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:49:53.040400 2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Apr 16 16:49:53.040503 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:49:53.040469 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2bd7515-2313-46a8-8961-28f13c30996b-samples-operator-tls podName:a2bd7515-2313-46a8-8961-28f13c30996b nodeName:}" failed. No retries permitted until 2026-04-16 16:49:57.040449171 +0000 UTC m=+131.122183405 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a2bd7515-2313-46a8-8961-28f13c30996b-samples-operator-tls") pod "cluster-samples-operator-667775844f-l9td2" (UID: "a2bd7515-2313-46a8-8961-28f13c30996b") : secret "samples-operator-tls" not found Apr 16 16:49:53.860404 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:53.860379 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/0.log" Apr 16 16:49:53.860783 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:53.860415 2574 generic.go:358] "Generic (PLEG): container finished" podID="3e7b3fa7-ba27-4076-b195-19d14c437d6a" containerID="24af34d08948606054b0d5fb42b4b3bfcda818185cf15ab91306f0c72d30d924" exitCode=255 Apr 16 16:49:53.860783 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:53.860453 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-d87b8d5fc-s6s57" event={"ID":"3e7b3fa7-ba27-4076-b195-19d14c437d6a","Type":"ContainerDied","Data":"24af34d08948606054b0d5fb42b4b3bfcda818185cf15ab91306f0c72d30d924"} Apr 16 16:49:53.860783 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:53.860662 2574 scope.go:117] "RemoveContainer" containerID="24af34d08948606054b0d5fb42b4b3bfcda818185cf15ab91306f0c72d30d924" Apr 16 16:49:54.863876 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:54.863853 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/1.log" Apr 16 16:49:54.864272 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:54.864215 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/0.log" Apr 16 16:49:54.864272 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:54.864256 2574 generic.go:358] "Generic (PLEG): container finished" podID="3e7b3fa7-ba27-4076-b195-19d14c437d6a" containerID="ff46f9c0ff18a877331694aa374c63acda75f715db4b63c5da0f08c94e611a5c" exitCode=255 Apr 16 16:49:54.864361 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:54.864302 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-d87b8d5fc-s6s57" event={"ID":"3e7b3fa7-ba27-4076-b195-19d14c437d6a","Type":"ContainerDied","Data":"ff46f9c0ff18a877331694aa374c63acda75f715db4b63c5da0f08c94e611a5c"} Apr 16 16:49:54.864361 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:54.864329 2574 scope.go:117] "RemoveContainer" containerID="24af34d08948606054b0d5fb42b4b3bfcda818185cf15ab91306f0c72d30d924" Apr 16 16:49:54.864583 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:54.864557 2574 scope.go:117] "RemoveContainer" containerID="ff46f9c0ff18a877331694aa374c63acda75f715db4b63c5da0f08c94e611a5c" Apr 16 16:49:54.864743 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:49:54.864725 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-d87b8d5fc-s6s57_openshift-console-operator(3e7b3fa7-ba27-4076-b195-19d14c437d6a)\"" pod="openshift-console-operator/console-operator-d87b8d5fc-s6s57" podUID="3e7b3fa7-ba27-4076-b195-19d14c437d6a" Apr 16 16:49:55.506900 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:55.506871 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-69965bb79d-2vh6b"] Apr 16 16:49:55.509986 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:55.509971 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-2vh6b" Apr 16 16:49:55.512459 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:55.512443 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Apr 16 16:49:55.512532 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:55.512442 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-zfffj\"" Apr 16 16:49:55.512532 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:55.512449 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Apr 16 16:49:55.513764 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:55.513751 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Apr 16 16:49:55.513836 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:55.513822 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Apr 16 16:49:55.516114 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:55.516097 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-69965bb79d-2vh6b"] Apr 16 16:49:55.557901 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:55.557880 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d403ed6e-d30e-4524-8802-f98b4817628b-serving-cert\") pod \"service-ca-operator-69965bb79d-2vh6b\" (UID: \"d403ed6e-d30e-4524-8802-f98b4817628b\") " pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-2vh6b" Apr 16 16:49:55.557998 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:55.557914 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58rwj\" (UniqueName: \"kubernetes.io/projected/d403ed6e-d30e-4524-8802-f98b4817628b-kube-api-access-58rwj\") pod \"service-ca-operator-69965bb79d-2vh6b\" (UID: \"d403ed6e-d30e-4524-8802-f98b4817628b\") " pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-2vh6b" Apr 16 16:49:55.557998 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:55.557939 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d403ed6e-d30e-4524-8802-f98b4817628b-config\") pod \"service-ca-operator-69965bb79d-2vh6b\" (UID: \"d403ed6e-d30e-4524-8802-f98b4817628b\") " pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-2vh6b" Apr 16 16:49:55.659161 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:55.659132 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d403ed6e-d30e-4524-8802-f98b4817628b-serving-cert\") pod \"service-ca-operator-69965bb79d-2vh6b\" (UID: \"d403ed6e-d30e-4524-8802-f98b4817628b\") " pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-2vh6b" Apr 16 16:49:55.659260 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:55.659176 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-58rwj\" (UniqueName: \"kubernetes.io/projected/d403ed6e-d30e-4524-8802-f98b4817628b-kube-api-access-58rwj\") pod \"service-ca-operator-69965bb79d-2vh6b\" (UID: \"d403ed6e-d30e-4524-8802-f98b4817628b\") " pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-2vh6b" Apr 16 16:49:55.659260 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:55.659219 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d403ed6e-d30e-4524-8802-f98b4817628b-config\") pod \"service-ca-operator-69965bb79d-2vh6b\" (UID: \"d403ed6e-d30e-4524-8802-f98b4817628b\") " pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-2vh6b" Apr 16 16:49:55.659750 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:55.659730 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d403ed6e-d30e-4524-8802-f98b4817628b-config\") pod \"service-ca-operator-69965bb79d-2vh6b\" (UID: \"d403ed6e-d30e-4524-8802-f98b4817628b\") " pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-2vh6b" Apr 16 16:49:55.661312 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:55.661291 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d403ed6e-d30e-4524-8802-f98b4817628b-serving-cert\") pod \"service-ca-operator-69965bb79d-2vh6b\" (UID: \"d403ed6e-d30e-4524-8802-f98b4817628b\") " pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-2vh6b" Apr 16 16:49:55.667155 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:55.667132 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58rwj\" (UniqueName: \"kubernetes.io/projected/d403ed6e-d30e-4524-8802-f98b4817628b-kube-api-access-58rwj\") pod \"service-ca-operator-69965bb79d-2vh6b\" (UID: \"d403ed6e-d30e-4524-8802-f98b4817628b\") " pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-2vh6b" Apr 16 16:49:55.818665 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:55.818591 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-2vh6b" Apr 16 16:49:55.867767 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:55.867746 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/1.log" Apr 16 16:49:55.868153 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:55.868132 2574 scope.go:117] "RemoveContainer" containerID="ff46f9c0ff18a877331694aa374c63acda75f715db4b63c5da0f08c94e611a5c" Apr 16 16:49:55.868370 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:49:55.868351 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-d87b8d5fc-s6s57_openshift-console-operator(3e7b3fa7-ba27-4076-b195-19d14c437d6a)\"" pod="openshift-console-operator/console-operator-d87b8d5fc-s6s57" podUID="3e7b3fa7-ba27-4076-b195-19d14c437d6a" Apr 16 16:49:55.927778 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:55.927751 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-69965bb79d-2vh6b"] Apr 16 16:49:55.930691 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:49:55.930661 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd403ed6e_d30e_4524_8802_f98b4817628b.slice/crio-8200766b25106ad848c2e96a42f9f238118ee24202be265fef10ccfd4a7e84c5 WatchSource:0}: Error finding container 8200766b25106ad848c2e96a42f9f238118ee24202be265fef10ccfd4a7e84c5: Status 404 returned error can't find the container with id 8200766b25106ad848c2e96a42f9f238118ee24202be265fef10ccfd4a7e84c5 Apr 16 16:49:56.163085 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:56.163048 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs\") pod \"network-metrics-daemon-kfxzj\" (UID: \"d2114257-06f1-4070-ad8d-afce484cc205\") " pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:49:56.163234 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:49:56.163196 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Apr 16 16:49:56.163287 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:49:56.163277 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs podName:d2114257-06f1-4070-ad8d-afce484cc205 nodeName:}" failed. No retries permitted until 2026-04-16 16:51:58.163258987 +0000 UTC m=+252.244993200 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs") pod "network-metrics-daemon-kfxzj" (UID: "d2114257-06f1-4070-ad8d-afce484cc205") : secret "metrics-daemon-secret" not found Apr 16 16:49:56.870497 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:56.870452 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-2vh6b" event={"ID":"d403ed6e-d30e-4524-8802-f98b4817628b","Type":"ContainerStarted","Data":"8200766b25106ad848c2e96a42f9f238118ee24202be265fef10ccfd4a7e84c5"} Apr 16 16:49:57.069988 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:57.069950 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a2bd7515-2313-46a8-8961-28f13c30996b-samples-operator-tls\") pod \"cluster-samples-operator-667775844f-l9td2\" (UID: \"a2bd7515-2313-46a8-8961-28f13c30996b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-667775844f-l9td2" Apr 16 16:49:57.070167 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:49:57.070122 2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Apr 16 16:49:57.070243 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:49:57.070213 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2bd7515-2313-46a8-8961-28f13c30996b-samples-operator-tls podName:a2bd7515-2313-46a8-8961-28f13c30996b nodeName:}" failed. No retries permitted until 2026-04-16 16:50:05.070183334 +0000 UTC m=+139.151917526 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a2bd7515-2313-46a8-8961-28f13c30996b-samples-operator-tls") pod "cluster-samples-operator-667775844f-l9td2" (UID: "a2bd7515-2313-46a8-8961-28f13c30996b") : secret "samples-operator-tls" not found Apr 16 16:49:57.873697 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:57.873665 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-2vh6b" event={"ID":"d403ed6e-d30e-4524-8802-f98b4817628b","Type":"ContainerStarted","Data":"a1057fd812b3b752f80c4813a9c6680965382ad96532e48fdca58781dd718023"} Apr 16 16:49:57.889402 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:57.889363 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-2vh6b" podStartSLOduration=1.3240441029999999 podStartE2EDuration="2.889353633s" podCreationTimestamp="2026-04-16 16:49:55 +0000 UTC" firstStartedPulling="2026-04-16 16:49:55.932442216 +0000 UTC m=+130.014176408" lastFinishedPulling="2026-04-16 16:49:57.497751745 +0000 UTC m=+131.579485938" observedRunningTime="2026-04-16 16:49:57.888975513 +0000 UTC m=+131.970709728" watchObservedRunningTime="2026-04-16 16:49:57.889353633 +0000 UTC m=+131.971087847" Apr 16 16:49:58.646986 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:58.646949 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-64d4d94569-66kjg"] Apr 16 16:49:58.650163 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:58.650147 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-64d4d94569-66kjg" Apr 16 16:49:58.652569 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:58.652548 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Apr 16 16:49:58.653713 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:58.653697 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-kn884\"" Apr 16 16:49:58.653812 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:58.653750 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Apr 16 16:49:58.657758 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:58.657738 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-64d4d94569-66kjg"] Apr 16 16:49:58.783096 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:58.783068 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjttv\" (UniqueName: \"kubernetes.io/projected/4013146b-f68d-4de5-a76e-dc6d4bba1a81-kube-api-access-jjttv\") pod \"migrator-64d4d94569-66kjg\" (UID: \"4013146b-f68d-4de5-a76e-dc6d4bba1a81\") " pod="openshift-kube-storage-version-migrator/migrator-64d4d94569-66kjg" Apr 16 16:49:58.883439 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:58.883418 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jjttv\" (UniqueName: \"kubernetes.io/projected/4013146b-f68d-4de5-a76e-dc6d4bba1a81-kube-api-access-jjttv\") pod \"migrator-64d4d94569-66kjg\" (UID: \"4013146b-f68d-4de5-a76e-dc6d4bba1a81\") " pod="openshift-kube-storage-version-migrator/migrator-64d4d94569-66kjg" Apr 16 16:49:58.891289 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:58.891265 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjttv\" (UniqueName: \"kubernetes.io/projected/4013146b-f68d-4de5-a76e-dc6d4bba1a81-kube-api-access-jjttv\") pod \"migrator-64d4d94569-66kjg\" (UID: \"4013146b-f68d-4de5-a76e-dc6d4bba1a81\") " pod="openshift-kube-storage-version-migrator/migrator-64d4d94569-66kjg" Apr 16 16:49:58.958867 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:58.958817 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-64d4d94569-66kjg" Apr 16 16:49:59.066861 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:59.066833 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-64d4d94569-66kjg"] Apr 16 16:49:59.069382 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:49:59.069353 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4013146b_f68d_4de5_a76e_dc6d4bba1a81.slice/crio-199670f36932869d2d09b163995165ad2e70dd18a28ca4bd064d6c7458b29da8 WatchSource:0}: Error finding container 199670f36932869d2d09b163995165ad2e70dd18a28ca4bd064d6c7458b29da8: Status 404 returned error can't find the container with id 199670f36932869d2d09b163995165ad2e70dd18a28ca4bd064d6c7458b29da8 Apr 16 16:49:59.879148 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:49:59.879111 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-64d4d94569-66kjg" event={"ID":"4013146b-f68d-4de5-a76e-dc6d4bba1a81","Type":"ContainerStarted","Data":"199670f36932869d2d09b163995165ad2e70dd18a28ca4bd064d6c7458b29da8"} Apr 16 16:50:00.883683 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:00.883646 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-64d4d94569-66kjg" event={"ID":"4013146b-f68d-4de5-a76e-dc6d4bba1a81","Type":"ContainerStarted","Data":"156a1779384b026b363acc171f1b4a7ccebb66392a0d321308eb21d926a9df5f"} Apr 16 16:50:00.883683 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:00.883687 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-64d4d94569-66kjg" event={"ID":"4013146b-f68d-4de5-a76e-dc6d4bba1a81","Type":"ContainerStarted","Data":"96f721c93bbbd1947ea4b549260130eb66c6dd4d9dae1ca2894df474b436b7a7"} Apr 16 16:50:00.897300 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:00.897274 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-bfc587fb7-wjc5x"] Apr 16 16:50:00.900160 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:00.900145 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-bfc587fb7-wjc5x" Apr 16 16:50:00.902900 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:00.902880 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Apr 16 16:50:00.903008 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:00.902883 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Apr 16 16:50:00.903249 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:00.903183 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-64d4d94569-66kjg" podStartSLOduration=1.5662701129999999 podStartE2EDuration="2.903169627s" podCreationTimestamp="2026-04-16 16:49:58 +0000 UTC" firstStartedPulling="2026-04-16 16:49:59.07107031 +0000 UTC m=+133.152804503" lastFinishedPulling="2026-04-16 16:50:00.407969823 +0000 UTC m=+134.489704017" observedRunningTime="2026-04-16 16:50:00.902940477 +0000 UTC m=+134.984674715" watchObservedRunningTime="2026-04-16 16:50:00.903169627 +0000 UTC m=+134.984903844" Apr 16 16:50:00.903577 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:00.903555 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Apr 16 16:50:00.903678 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:00.903617 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Apr 16 16:50:00.903724 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:00.903688 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-7pplm\"" Apr 16 16:50:00.906652 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:00.906632 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-bfc587fb7-wjc5x"] Apr 16 16:50:00.999797 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:00.999772 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/46198ecc-9f1b-47dc-a801-80ecf9e1c3f9-signing-key\") pod \"service-ca-bfc587fb7-wjc5x\" (UID: \"46198ecc-9f1b-47dc-a801-80ecf9e1c3f9\") " pod="openshift-service-ca/service-ca-bfc587fb7-wjc5x" Apr 16 16:50:00.999907 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:00.999828 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/46198ecc-9f1b-47dc-a801-80ecf9e1c3f9-signing-cabundle\") pod \"service-ca-bfc587fb7-wjc5x\" (UID: \"46198ecc-9f1b-47dc-a801-80ecf9e1c3f9\") " pod="openshift-service-ca/service-ca-bfc587fb7-wjc5x" Apr 16 16:50:00.999907 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:00.999858 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntlfj\" (UniqueName: \"kubernetes.io/projected/46198ecc-9f1b-47dc-a801-80ecf9e1c3f9-kube-api-access-ntlfj\") pod \"service-ca-bfc587fb7-wjc5x\" (UID: \"46198ecc-9f1b-47dc-a801-80ecf9e1c3f9\") " pod="openshift-service-ca/service-ca-bfc587fb7-wjc5x" Apr 16 16:50:01.100639 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:01.100613 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/46198ecc-9f1b-47dc-a801-80ecf9e1c3f9-signing-key\") pod \"service-ca-bfc587fb7-wjc5x\" (UID: \"46198ecc-9f1b-47dc-a801-80ecf9e1c3f9\") " pod="openshift-service-ca/service-ca-bfc587fb7-wjc5x" Apr 16 16:50:01.100750 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:01.100664 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/46198ecc-9f1b-47dc-a801-80ecf9e1c3f9-signing-cabundle\") pod \"service-ca-bfc587fb7-wjc5x\" (UID: \"46198ecc-9f1b-47dc-a801-80ecf9e1c3f9\") " pod="openshift-service-ca/service-ca-bfc587fb7-wjc5x" Apr 16 16:50:01.100819 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:01.100791 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ntlfj\" (UniqueName: \"kubernetes.io/projected/46198ecc-9f1b-47dc-a801-80ecf9e1c3f9-kube-api-access-ntlfj\") pod \"service-ca-bfc587fb7-wjc5x\" (UID: \"46198ecc-9f1b-47dc-a801-80ecf9e1c3f9\") " pod="openshift-service-ca/service-ca-bfc587fb7-wjc5x" Apr 16 16:50:01.101166 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:01.101150 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/46198ecc-9f1b-47dc-a801-80ecf9e1c3f9-signing-cabundle\") pod \"service-ca-bfc587fb7-wjc5x\" (UID: \"46198ecc-9f1b-47dc-a801-80ecf9e1c3f9\") " pod="openshift-service-ca/service-ca-bfc587fb7-wjc5x" Apr 16 16:50:01.103043 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:01.103024 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/46198ecc-9f1b-47dc-a801-80ecf9e1c3f9-signing-key\") pod \"service-ca-bfc587fb7-wjc5x\" (UID: \"46198ecc-9f1b-47dc-a801-80ecf9e1c3f9\") " pod="openshift-service-ca/service-ca-bfc587fb7-wjc5x" Apr 16 16:50:01.108926 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:01.108907 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntlfj\" (UniqueName: \"kubernetes.io/projected/46198ecc-9f1b-47dc-a801-80ecf9e1c3f9-kube-api-access-ntlfj\") pod \"service-ca-bfc587fb7-wjc5x\" (UID: \"46198ecc-9f1b-47dc-a801-80ecf9e1c3f9\") " pod="openshift-service-ca/service-ca-bfc587fb7-wjc5x" Apr 16 16:50:01.209739 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:01.209694 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-bfc587fb7-wjc5x" Apr 16 16:50:01.313804 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:01.313776 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-bfc587fb7-wjc5x"] Apr 16 16:50:01.316539 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:50:01.316513 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46198ecc_9f1b_47dc_a801_80ecf9e1c3f9.slice/crio-d2f7978546be6a6ba9363ee334d06b1b411e32f4b32832c91e0afba168585e47 WatchSource:0}: Error finding container d2f7978546be6a6ba9363ee334d06b1b411e32f4b32832c91e0afba168585e47: Status 404 returned error can't find the container with id d2f7978546be6a6ba9363ee334d06b1b411e32f4b32832c91e0afba168585e47 Apr 16 16:50:01.676895 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:01.676863 2574 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-d87b8d5fc-s6s57" Apr 16 16:50:01.676895 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:01.676903 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-d87b8d5fc-s6s57" Apr 16 16:50:01.677255 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:01.677242 2574 scope.go:117] "RemoveContainer" containerID="ff46f9c0ff18a877331694aa374c63acda75f715db4b63c5da0f08c94e611a5c" Apr 16 16:50:01.677405 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:50:01.677389 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-d87b8d5fc-s6s57_openshift-console-operator(3e7b3fa7-ba27-4076-b195-19d14c437d6a)\"" pod="openshift-console-operator/console-operator-d87b8d5fc-s6s57" podUID="3e7b3fa7-ba27-4076-b195-19d14c437d6a" Apr 16 16:50:01.887388 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:01.887356 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-bfc587fb7-wjc5x" event={"ID":"46198ecc-9f1b-47dc-a801-80ecf9e1c3f9","Type":"ContainerStarted","Data":"e9cf058c70f307607f5a2db77a039308f039bbb63df917aac5830445f02ef5c6"} Apr 16 16:50:01.887388 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:01.887390 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-bfc587fb7-wjc5x" event={"ID":"46198ecc-9f1b-47dc-a801-80ecf9e1c3f9","Type":"ContainerStarted","Data":"d2f7978546be6a6ba9363ee334d06b1b411e32f4b32832c91e0afba168585e47"} Apr 16 16:50:01.903611 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:01.903570 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-bfc587fb7-wjc5x" podStartSLOduration=1.903557135 podStartE2EDuration="1.903557135s" podCreationTimestamp="2026-04-16 16:50:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 16:50:01.902807669 +0000 UTC m=+135.984541885" watchObservedRunningTime="2026-04-16 16:50:01.903557135 +0000 UTC m=+135.985291348" Apr 16 16:50:05.133170 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:05.133137 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a2bd7515-2313-46a8-8961-28f13c30996b-samples-operator-tls\") pod \"cluster-samples-operator-667775844f-l9td2\" (UID: \"a2bd7515-2313-46a8-8961-28f13c30996b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-667775844f-l9td2" Apr 16 16:50:05.135490 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:05.135466 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a2bd7515-2313-46a8-8961-28f13c30996b-samples-operator-tls\") pod \"cluster-samples-operator-667775844f-l9td2\" (UID: \"a2bd7515-2313-46a8-8961-28f13c30996b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-667775844f-l9td2" Apr 16 16:50:05.259782 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:05.259756 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-667775844f-l9td2" Apr 16 16:50:05.373321 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:05.373291 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-667775844f-l9td2"] Apr 16 16:50:05.898017 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:05.897976 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-667775844f-l9td2" event={"ID":"a2bd7515-2313-46a8-8961-28f13c30996b","Type":"ContainerStarted","Data":"7ae495e195e5d69b5e396449495340cb0d8ddb5ab892d71d40f7ed2a061e4009"} Apr 16 16:50:07.903636 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:07.903598 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-667775844f-l9td2" event={"ID":"a2bd7515-2313-46a8-8961-28f13c30996b","Type":"ContainerStarted","Data":"9d36efcc46ec23eac7374877d0aff0292d4b6a91874e0e79a4ed69b1aa94d1e9"} Apr 16 16:50:07.903636 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:07.903637 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-667775844f-l9td2" event={"ID":"a2bd7515-2313-46a8-8961-28f13c30996b","Type":"ContainerStarted","Data":"d7291527a963c08fd952e3e373c45d3225ecd3bd04d950c467cae41611dfafe5"} Apr 16 16:50:07.923059 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:07.923011 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-667775844f-l9td2" podStartSLOduration=17.382376873 podStartE2EDuration="18.922998846s" podCreationTimestamp="2026-04-16 16:49:49 +0000 UTC" firstStartedPulling="2026-04-16 16:50:05.413378264 +0000 UTC m=+139.495112461" lastFinishedPulling="2026-04-16 16:50:06.954000238 +0000 UTC m=+141.035734434" observedRunningTime="2026-04-16 16:50:07.922304093 +0000 UTC m=+142.004038308" watchObservedRunningTime="2026-04-16 16:50:07.922998846 +0000 UTC m=+142.004733112" Apr 16 16:50:13.507999 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:13.507968 2574 scope.go:117] "RemoveContainer" containerID="ff46f9c0ff18a877331694aa374c63acda75f715db4b63c5da0f08c94e611a5c" Apr 16 16:50:13.919362 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:13.919332 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/1.log" Apr 16 16:50:13.919511 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:13.919413 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-d87b8d5fc-s6s57" event={"ID":"3e7b3fa7-ba27-4076-b195-19d14c437d6a","Type":"ContainerStarted","Data":"74e72a12bf2ed81d5f647555d872ee395b9a3ad02fb11dfd03dec314b5c59f9f"} Apr 16 16:50:13.919694 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:13.919676 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-d87b8d5fc-s6s57" Apr 16 16:50:13.923973 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:13.923953 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-d87b8d5fc-s6s57" Apr 16 16:50:13.936728 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:13.936683 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-d87b8d5fc-s6s57" podStartSLOduration=21.434206645 podStartE2EDuration="22.936672262s" podCreationTimestamp="2026-04-16 16:49:51 +0000 UTC" firstStartedPulling="2026-04-16 16:49:51.873011948 +0000 UTC m=+125.954746140" lastFinishedPulling="2026-04-16 16:49:53.375477564 +0000 UTC m=+127.457211757" observedRunningTime="2026-04-16 16:50:13.936342775 +0000 UTC m=+148.018076989" watchObservedRunningTime="2026-04-16 16:50:13.936672262 +0000 UTC m=+148.018406475" Apr 16 16:50:20.308688 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.308649 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-586b57c7b4-4pm8j"] Apr 16 16:50:20.311559 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.311537 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-69848b55b9-5mtcx"] Apr 16 16:50:20.311693 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.311675 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-586b57c7b4-4pm8j" Apr 16 16:50:20.313435 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.313418 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69848b55b9-5mtcx" Apr 16 16:50:20.314568 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.314543 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Apr 16 16:50:20.314652 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.314574 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Apr 16 16:50:20.317005 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.316988 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Apr 16 16:50:20.317285 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.317265 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Apr 16 16:50:20.317368 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.317277 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Apr 16 16:50:20.318271 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.318151 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Apr 16 16:50:20.318271 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.318165 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Apr 16 16:50:20.318392 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.318171 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-82slw\"" Apr 16 16:50:20.318441 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.318425 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-q9pts\"" Apr 16 16:50:20.326646 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.326625 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4ljtp"] Apr 16 16:50:20.328513 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.328500 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4ljtp" Apr 16 16:50:20.331312 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.331295 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Apr 16 16:50:20.331606 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.331589 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nbldh\"" Apr 16 16:50:20.331927 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.331907 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Apr 16 16:50:20.332026 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.332009 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-586b57c7b4-4pm8j"] Apr 16 16:50:20.337619 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.337602 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-67db985cf-skv8z"] Apr 16 16:50:20.339261 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.339245 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69848b55b9-5mtcx"] Apr 16 16:50:20.339340 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.339328 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67db985cf-skv8z" Apr 16 16:50:20.343908 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.343885 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Apr 16 16:50:20.344191 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.344173 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-djqlx\"" Apr 16 16:50:20.344305 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.344281 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Apr 16 16:50:20.344452 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.344436 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Apr 16 16:50:20.346489 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.346161 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4ljtp"] Apr 16 16:50:20.349318 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.349301 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Apr 16 16:50:20.354795 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.354774 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67db985cf-skv8z"] Apr 16 16:50:20.448973 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.448949 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dc5c6566-f0a7-42ab-ae2f-8a5b0846b043-installation-pull-secrets\") pod \"image-registry-67db985cf-skv8z\" (UID: \"dc5c6566-f0a7-42ab-ae2f-8a5b0846b043\") " pod="openshift-image-registry/image-registry-67db985cf-skv8z" Apr 16 16:50:20.449109 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.448982 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c916378f-19b0-4da1-a0b8-7d1a88286786-service-ca\") pod \"console-69848b55b9-5mtcx\" (UID: \"c916378f-19b0-4da1-a0b8-7d1a88286786\") " pod="openshift-console/console-69848b55b9-5mtcx" Apr 16 16:50:20.449109 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.449007 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4ljtp\" (UID: \"cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7\") " pod="openshift-insights/insights-runtime-extractor-4ljtp" Apr 16 16:50:20.449109 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.449052 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c916378f-19b0-4da1-a0b8-7d1a88286786-console-oauth-config\") pod \"console-69848b55b9-5mtcx\" (UID: \"c916378f-19b0-4da1-a0b8-7d1a88286786\") " pod="openshift-console/console-69848b55b9-5mtcx" Apr 16 16:50:20.449109 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.449097 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dc5c6566-f0a7-42ab-ae2f-8a5b0846b043-ca-trust-extracted\") pod \"image-registry-67db985cf-skv8z\" (UID: \"dc5c6566-f0a7-42ab-ae2f-8a5b0846b043\") " pod="openshift-image-registry/image-registry-67db985cf-skv8z" Apr 16 16:50:20.449355 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.449127 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dc5c6566-f0a7-42ab-ae2f-8a5b0846b043-registry-certificates\") pod \"image-registry-67db985cf-skv8z\" (UID: \"dc5c6566-f0a7-42ab-ae2f-8a5b0846b043\") " pod="openshift-image-registry/image-registry-67db985cf-skv8z" Apr 16 16:50:20.449355 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.449175 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dc5c6566-f0a7-42ab-ae2f-8a5b0846b043-image-registry-private-configuration\") pod \"image-registry-67db985cf-skv8z\" (UID: \"dc5c6566-f0a7-42ab-ae2f-8a5b0846b043\") " pod="openshift-image-registry/image-registry-67db985cf-skv8z" Apr 16 16:50:20.449355 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.449238 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z8hk\" (UniqueName: \"kubernetes.io/projected/cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7-kube-api-access-6z8hk\") pod \"insights-runtime-extractor-4ljtp\" (UID: \"cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7\") " pod="openshift-insights/insights-runtime-extractor-4ljtp" Apr 16 16:50:20.449355 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.449259 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w5n2\" (UniqueName: \"kubernetes.io/projected/5a9d593a-f4f5-4786-849a-1330254f9b1c-kube-api-access-6w5n2\") pod \"downloads-586b57c7b4-4pm8j\" (UID: \"5a9d593a-f4f5-4786-849a-1330254f9b1c\") " pod="openshift-console/downloads-586b57c7b4-4pm8j" Apr 16 16:50:20.449355 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.449277 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dc5c6566-f0a7-42ab-ae2f-8a5b0846b043-trusted-ca\") pod \"image-registry-67db985cf-skv8z\" (UID: \"dc5c6566-f0a7-42ab-ae2f-8a5b0846b043\") " pod="openshift-image-registry/image-registry-67db985cf-skv8z" Apr 16 16:50:20.449355 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.449292 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq69v\" (UniqueName: \"kubernetes.io/projected/dc5c6566-f0a7-42ab-ae2f-8a5b0846b043-kube-api-access-hq69v\") pod \"image-registry-67db985cf-skv8z\" (UID: \"dc5c6566-f0a7-42ab-ae2f-8a5b0846b043\") " pod="openshift-image-registry/image-registry-67db985cf-skv8z" Apr 16 16:50:20.449355 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.449319 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c916378f-19b0-4da1-a0b8-7d1a88286786-oauth-serving-cert\") pod \"console-69848b55b9-5mtcx\" (UID: \"c916378f-19b0-4da1-a0b8-7d1a88286786\") " pod="openshift-console/console-69848b55b9-5mtcx" Apr 16 16:50:20.449355 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.449346 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7-data-volume\") pod \"insights-runtime-extractor-4ljtp\" (UID: \"cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7\") " pod="openshift-insights/insights-runtime-extractor-4ljtp" Apr 16 16:50:20.449732 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.449364 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4ljtp\" (UID: \"cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7\") " pod="openshift-insights/insights-runtime-extractor-4ljtp" Apr 16 16:50:20.449732 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.449377 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c916378f-19b0-4da1-a0b8-7d1a88286786-console-config\") pod \"console-69848b55b9-5mtcx\" (UID: \"c916378f-19b0-4da1-a0b8-7d1a88286786\") " pod="openshift-console/console-69848b55b9-5mtcx" Apr 16 16:50:20.449732 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.449438 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dc5c6566-f0a7-42ab-ae2f-8a5b0846b043-registry-tls\") pod \"image-registry-67db985cf-skv8z\" (UID: \"dc5c6566-f0a7-42ab-ae2f-8a5b0846b043\") " pod="openshift-image-registry/image-registry-67db985cf-skv8z" Apr 16 16:50:20.449732 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.449463 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dc5c6566-f0a7-42ab-ae2f-8a5b0846b043-bound-sa-token\") pod \"image-registry-67db985cf-skv8z\" (UID: \"dc5c6566-f0a7-42ab-ae2f-8a5b0846b043\") " pod="openshift-image-registry/image-registry-67db985cf-skv8z" Apr 16 16:50:20.449732 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.449493 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c916378f-19b0-4da1-a0b8-7d1a88286786-console-serving-cert\") pod \"console-69848b55b9-5mtcx\" (UID: \"c916378f-19b0-4da1-a0b8-7d1a88286786\") " pod="openshift-console/console-69848b55b9-5mtcx" Apr 16 16:50:20.449732 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.449513 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7-crio-socket\") pod \"insights-runtime-extractor-4ljtp\" (UID: \"cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7\") " pod="openshift-insights/insights-runtime-extractor-4ljtp" Apr 16 16:50:20.449732 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.449526 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stjqv\" (UniqueName: \"kubernetes.io/projected/c916378f-19b0-4da1-a0b8-7d1a88286786-kube-api-access-stjqv\") pod \"console-69848b55b9-5mtcx\" (UID: \"c916378f-19b0-4da1-a0b8-7d1a88286786\") " pod="openshift-console/console-69848b55b9-5mtcx" Apr 16 16:50:20.549757 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.549734 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dc5c6566-f0a7-42ab-ae2f-8a5b0846b043-image-registry-private-configuration\") pod \"image-registry-67db985cf-skv8z\" (UID: \"dc5c6566-f0a7-42ab-ae2f-8a5b0846b043\") " pod="openshift-image-registry/image-registry-67db985cf-skv8z" Apr 16 16:50:20.549853 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.549778 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6z8hk\" (UniqueName: \"kubernetes.io/projected/cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7-kube-api-access-6z8hk\") pod \"insights-runtime-extractor-4ljtp\" (UID: \"cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7\") " pod="openshift-insights/insights-runtime-extractor-4ljtp" Apr 16 16:50:20.549853 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.549801 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6w5n2\" (UniqueName: \"kubernetes.io/projected/5a9d593a-f4f5-4786-849a-1330254f9b1c-kube-api-access-6w5n2\") pod \"downloads-586b57c7b4-4pm8j\" (UID: \"5a9d593a-f4f5-4786-849a-1330254f9b1c\") " pod="openshift-console/downloads-586b57c7b4-4pm8j" Apr 16 16:50:20.549853 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.549826 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dc5c6566-f0a7-42ab-ae2f-8a5b0846b043-trusted-ca\") pod \"image-registry-67db985cf-skv8z\" (UID: \"dc5c6566-f0a7-42ab-ae2f-8a5b0846b043\") " pod="openshift-image-registry/image-registry-67db985cf-skv8z" Apr 16 16:50:20.549853 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.549848 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hq69v\" (UniqueName: \"kubernetes.io/projected/dc5c6566-f0a7-42ab-ae2f-8a5b0846b043-kube-api-access-hq69v\") pod \"image-registry-67db985cf-skv8z\" (UID: \"dc5c6566-f0a7-42ab-ae2f-8a5b0846b043\") " pod="openshift-image-registry/image-registry-67db985cf-skv8z" Apr 16 16:50:20.550054 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.549874 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c916378f-19b0-4da1-a0b8-7d1a88286786-oauth-serving-cert\") pod \"console-69848b55b9-5mtcx\" (UID: \"c916378f-19b0-4da1-a0b8-7d1a88286786\") " pod="openshift-console/console-69848b55b9-5mtcx" Apr 16 16:50:20.550054 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.550033 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7-data-volume\") pod \"insights-runtime-extractor-4ljtp\" (UID: \"cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7\") " pod="openshift-insights/insights-runtime-extractor-4ljtp" Apr 16 16:50:20.550151 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.550081 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4ljtp\" (UID: \"cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7\") " pod="openshift-insights/insights-runtime-extractor-4ljtp" Apr 16 16:50:20.550151 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.550107 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c916378f-19b0-4da1-a0b8-7d1a88286786-console-config\") pod \"console-69848b55b9-5mtcx\" (UID: \"c916378f-19b0-4da1-a0b8-7d1a88286786\") " pod="openshift-console/console-69848b55b9-5mtcx" Apr 16 16:50:20.550271 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.550154 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dc5c6566-f0a7-42ab-ae2f-8a5b0846b043-registry-tls\") pod \"image-registry-67db985cf-skv8z\" (UID: \"dc5c6566-f0a7-42ab-ae2f-8a5b0846b043\") " pod="openshift-image-registry/image-registry-67db985cf-skv8z" Apr 16 16:50:20.550271 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.550186 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dc5c6566-f0a7-42ab-ae2f-8a5b0846b043-bound-sa-token\") pod \"image-registry-67db985cf-skv8z\" (UID: \"dc5c6566-f0a7-42ab-ae2f-8a5b0846b043\") " pod="openshift-image-registry/image-registry-67db985cf-skv8z" Apr 16 16:50:20.550271 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.550252 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c916378f-19b0-4da1-a0b8-7d1a88286786-console-serving-cert\") pod \"console-69848b55b9-5mtcx\" (UID: \"c916378f-19b0-4da1-a0b8-7d1a88286786\") " pod="openshift-console/console-69848b55b9-5mtcx" Apr 16 16:50:20.550411 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.550281 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7-crio-socket\") pod \"insights-runtime-extractor-4ljtp\" (UID: \"cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7\") " pod="openshift-insights/insights-runtime-extractor-4ljtp" Apr 16 16:50:20.550411 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.550309 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-stjqv\" (UniqueName: \"kubernetes.io/projected/c916378f-19b0-4da1-a0b8-7d1a88286786-kube-api-access-stjqv\") pod \"console-69848b55b9-5mtcx\" (UID: \"c916378f-19b0-4da1-a0b8-7d1a88286786\") " pod="openshift-console/console-69848b55b9-5mtcx" Apr 16 16:50:20.550411 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.550348 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dc5c6566-f0a7-42ab-ae2f-8a5b0846b043-installation-pull-secrets\") pod \"image-registry-67db985cf-skv8z\" (UID: \"dc5c6566-f0a7-42ab-ae2f-8a5b0846b043\") " pod="openshift-image-registry/image-registry-67db985cf-skv8z" Apr 16 16:50:20.550411 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.550379 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c916378f-19b0-4da1-a0b8-7d1a88286786-service-ca\") pod \"console-69848b55b9-5mtcx\" (UID: \"c916378f-19b0-4da1-a0b8-7d1a88286786\") " pod="openshift-console/console-69848b55b9-5mtcx" Apr 16 16:50:20.550411 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.550404 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4ljtp\" (UID: \"cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7\") " pod="openshift-insights/insights-runtime-extractor-4ljtp" Apr 16 16:50:20.550647 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.550433 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c916378f-19b0-4da1-a0b8-7d1a88286786-console-oauth-config\") pod \"console-69848b55b9-5mtcx\" (UID: \"c916378f-19b0-4da1-a0b8-7d1a88286786\") " pod="openshift-console/console-69848b55b9-5mtcx" Apr 16 16:50:20.550647 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.550451 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7-data-volume\") pod \"insights-runtime-extractor-4ljtp\" (UID: \"cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7\") " pod="openshift-insights/insights-runtime-extractor-4ljtp" Apr 16 16:50:20.550647 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.550459 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dc5c6566-f0a7-42ab-ae2f-8a5b0846b043-ca-trust-extracted\") pod \"image-registry-67db985cf-skv8z\" (UID: \"dc5c6566-f0a7-42ab-ae2f-8a5b0846b043\") " pod="openshift-image-registry/image-registry-67db985cf-skv8z" Apr 16 16:50:20.550647 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.550483 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dc5c6566-f0a7-42ab-ae2f-8a5b0846b043-registry-certificates\") pod \"image-registry-67db985cf-skv8z\" (UID: \"dc5c6566-f0a7-42ab-ae2f-8a5b0846b043\") " pod="openshift-image-registry/image-registry-67db985cf-skv8z" Apr 16 16:50:20.550647 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.550612 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c916378f-19b0-4da1-a0b8-7d1a88286786-oauth-serving-cert\") pod \"console-69848b55b9-5mtcx\" (UID: \"c916378f-19b0-4da1-a0b8-7d1a88286786\") " pod="openshift-console/console-69848b55b9-5mtcx" Apr 16 16:50:20.550875 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.550733 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c916378f-19b0-4da1-a0b8-7d1a88286786-console-config\") pod \"console-69848b55b9-5mtcx\" (UID: \"c916378f-19b0-4da1-a0b8-7d1a88286786\") " pod="openshift-console/console-69848b55b9-5mtcx" Apr 16 16:50:20.551078 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.551055 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dc5c6566-f0a7-42ab-ae2f-8a5b0846b043-trusted-ca\") pod \"image-registry-67db985cf-skv8z\" (UID: \"dc5c6566-f0a7-42ab-ae2f-8a5b0846b043\") " pod="openshift-image-registry/image-registry-67db985cf-skv8z" Apr 16 16:50:20.551267 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.551245 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4ljtp\" (UID: \"cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7\") " pod="openshift-insights/insights-runtime-extractor-4ljtp" Apr 16 16:50:20.551352 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.551314 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7-crio-socket\") pod \"insights-runtime-extractor-4ljtp\" (UID: \"cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7\") " pod="openshift-insights/insights-runtime-extractor-4ljtp" Apr 16 16:50:20.551352 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.551343 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dc5c6566-f0a7-42ab-ae2f-8a5b0846b043-registry-certificates\") pod \"image-registry-67db985cf-skv8z\" (UID: \"dc5c6566-f0a7-42ab-ae2f-8a5b0846b043\") " pod="openshift-image-registry/image-registry-67db985cf-skv8z" Apr 16 16:50:20.551626 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.551588 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dc5c6566-f0a7-42ab-ae2f-8a5b0846b043-ca-trust-extracted\") pod \"image-registry-67db985cf-skv8z\" (UID: \"dc5c6566-f0a7-42ab-ae2f-8a5b0846b043\") " pod="openshift-image-registry/image-registry-67db985cf-skv8z" Apr 16 16:50:20.551730 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.551635 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c916378f-19b0-4da1-a0b8-7d1a88286786-service-ca\") pod \"console-69848b55b9-5mtcx\" (UID: \"c916378f-19b0-4da1-a0b8-7d1a88286786\") " pod="openshift-console/console-69848b55b9-5mtcx" Apr 16 16:50:20.552811 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.552783 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dc5c6566-f0a7-42ab-ae2f-8a5b0846b043-image-registry-private-configuration\") pod \"image-registry-67db985cf-skv8z\" (UID: \"dc5c6566-f0a7-42ab-ae2f-8a5b0846b043\") " pod="openshift-image-registry/image-registry-67db985cf-skv8z" Apr 16 16:50:20.553362 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.553340 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dc5c6566-f0a7-42ab-ae2f-8a5b0846b043-registry-tls\") pod \"image-registry-67db985cf-skv8z\" (UID: \"dc5c6566-f0a7-42ab-ae2f-8a5b0846b043\") " pod="openshift-image-registry/image-registry-67db985cf-skv8z" Apr 16 16:50:20.553466 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.553403 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4ljtp\" (UID: \"cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7\") " pod="openshift-insights/insights-runtime-extractor-4ljtp" Apr 16 16:50:20.553466 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.553437 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c916378f-19b0-4da1-a0b8-7d1a88286786-console-serving-cert\") pod \"console-69848b55b9-5mtcx\" (UID: \"c916378f-19b0-4da1-a0b8-7d1a88286786\") " pod="openshift-console/console-69848b55b9-5mtcx" Apr 16 16:50:20.553751 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.553730 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dc5c6566-f0a7-42ab-ae2f-8a5b0846b043-installation-pull-secrets\") pod \"image-registry-67db985cf-skv8z\" (UID: \"dc5c6566-f0a7-42ab-ae2f-8a5b0846b043\") " pod="openshift-image-registry/image-registry-67db985cf-skv8z" Apr 16 16:50:20.553849 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.553829 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c916378f-19b0-4da1-a0b8-7d1a88286786-console-oauth-config\") pod \"console-69848b55b9-5mtcx\" (UID: \"c916378f-19b0-4da1-a0b8-7d1a88286786\") " pod="openshift-console/console-69848b55b9-5mtcx" Apr 16 16:50:20.561930 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.561880 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dc5c6566-f0a7-42ab-ae2f-8a5b0846b043-bound-sa-token\") pod \"image-registry-67db985cf-skv8z\" (UID: \"dc5c6566-f0a7-42ab-ae2f-8a5b0846b043\") " pod="openshift-image-registry/image-registry-67db985cf-skv8z" Apr 16 16:50:20.562226 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.562185 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w5n2\" (UniqueName: \"kubernetes.io/projected/5a9d593a-f4f5-4786-849a-1330254f9b1c-kube-api-access-6w5n2\") pod \"downloads-586b57c7b4-4pm8j\" (UID: \"5a9d593a-f4f5-4786-849a-1330254f9b1c\") " pod="openshift-console/downloads-586b57c7b4-4pm8j" Apr 16 16:50:20.562317 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.562260 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z8hk\" (UniqueName: \"kubernetes.io/projected/cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7-kube-api-access-6z8hk\") pod \"insights-runtime-extractor-4ljtp\" (UID: \"cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7\") " pod="openshift-insights/insights-runtime-extractor-4ljtp" Apr 16 16:50:20.562317 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.562305 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq69v\" (UniqueName: \"kubernetes.io/projected/dc5c6566-f0a7-42ab-ae2f-8a5b0846b043-kube-api-access-hq69v\") pod \"image-registry-67db985cf-skv8z\" (UID: \"dc5c6566-f0a7-42ab-ae2f-8a5b0846b043\") " pod="openshift-image-registry/image-registry-67db985cf-skv8z" Apr 16 16:50:20.562594 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.562575 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stjqv\" (UniqueName: \"kubernetes.io/projected/c916378f-19b0-4da1-a0b8-7d1a88286786-kube-api-access-stjqv\") pod \"console-69848b55b9-5mtcx\" (UID: \"c916378f-19b0-4da1-a0b8-7d1a88286786\") " pod="openshift-console/console-69848b55b9-5mtcx" Apr 16 16:50:20.621755 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.621732 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-586b57c7b4-4pm8j" Apr 16 16:50:20.626303 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.626283 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69848b55b9-5mtcx" Apr 16 16:50:20.638088 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.638068 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4ljtp" Apr 16 16:50:20.649439 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.649377 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67db985cf-skv8z" Apr 16 16:50:20.788545 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.788514 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-586b57c7b4-4pm8j"] Apr 16 16:50:20.791214 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:50:20.791169 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a9d593a_f4f5_4786_849a_1330254f9b1c.slice/crio-3da4bd104b1d2867b8f9e2c8e202280311dfe70a292c106271982be891f09738 WatchSource:0}: Error finding container 3da4bd104b1d2867b8f9e2c8e202280311dfe70a292c106271982be891f09738: Status 404 returned error can't find the container with id 3da4bd104b1d2867b8f9e2c8e202280311dfe70a292c106271982be891f09738 Apr 16 16:50:20.806012 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.805991 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69848b55b9-5mtcx"] Apr 16 16:50:20.808088 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:50:20.808059 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc916378f_19b0_4da1_a0b8_7d1a88286786.slice/crio-cca8f917e0db3aa060b3f1adac8857fcd4a57798592bf191f98036bc75240b53 WatchSource:0}: Error finding container cca8f917e0db3aa060b3f1adac8857fcd4a57798592bf191f98036bc75240b53: Status 404 returned error can't find the container with id cca8f917e0db3aa060b3f1adac8857fcd4a57798592bf191f98036bc75240b53 Apr 16 16:50:20.941675 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.941639 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-586b57c7b4-4pm8j" event={"ID":"5a9d593a-f4f5-4786-849a-1330254f9b1c","Type":"ContainerStarted","Data":"3da4bd104b1d2867b8f9e2c8e202280311dfe70a292c106271982be891f09738"} Apr 16 16:50:20.942627 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:20.942603 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69848b55b9-5mtcx" event={"ID":"c916378f-19b0-4da1-a0b8-7d1a88286786","Type":"ContainerStarted","Data":"cca8f917e0db3aa060b3f1adac8857fcd4a57798592bf191f98036bc75240b53"} Apr 16 16:50:21.028708 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:21.028680 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67db985cf-skv8z"] Apr 16 16:50:21.031716 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:21.031695 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4ljtp"] Apr 16 16:50:21.032704 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:50:21.032674 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc5c6566_f0a7_42ab_ae2f_8a5b0846b043.slice/crio-74574293a132e8bcf608c556d384d046090b1fd2eb10eb78532807c6e0ed56bb WatchSource:0}: Error finding container 74574293a132e8bcf608c556d384d046090b1fd2eb10eb78532807c6e0ed56bb: Status 404 returned error can't find the container with id 74574293a132e8bcf608c556d384d046090b1fd2eb10eb78532807c6e0ed56bb Apr 16 16:50:21.034144 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:50:21.034125 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbbcedd1_7dfb_4f57_b692_6fb3ccd071a7.slice/crio-706b121755fb6ef174076d963dcf9957527a23815d6eb651977c633f647d3309 WatchSource:0}: Error finding container 706b121755fb6ef174076d963dcf9957527a23815d6eb651977c633f647d3309: Status 404 returned error can't find the container with id 706b121755fb6ef174076d963dcf9957527a23815d6eb651977c633f647d3309 Apr 16 16:50:21.328501 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:50:21.328410 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-gk5x8" podUID="430f1690-8090-4eaa-903e-a5aab4c24146" Apr 16 16:50:21.339510 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:50:21.339465 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-8kc47" podUID="a5cf9cec-6008-46d4-9c84-74af0d3ff9c7" Apr 16 16:50:21.519586 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:50:21.519543 2574 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-kfxzj" podUID="d2114257-06f1-4070-ad8d-afce484cc205" Apr 16 16:50:21.947975 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:21.947866 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67db985cf-skv8z" event={"ID":"dc5c6566-f0a7-42ab-ae2f-8a5b0846b043","Type":"ContainerStarted","Data":"6351bf5cc7349dfc6b372ef5188b67e3c472deccad10482ab47d87fd72b97f52"} Apr 16 16:50:21.947975 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:21.947906 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67db985cf-skv8z" event={"ID":"dc5c6566-f0a7-42ab-ae2f-8a5b0846b043","Type":"ContainerStarted","Data":"74574293a132e8bcf608c556d384d046090b1fd2eb10eb78532807c6e0ed56bb"} Apr 16 16:50:21.947975 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:21.947951 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-67db985cf-skv8z" Apr 16 16:50:21.949717 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:21.949692 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4ljtp" event={"ID":"cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7","Type":"ContainerStarted","Data":"1e405205c76ad34abffa8722b5f57b8c70a7945459a98ce7ddc41c41075b6a43"} Apr 16 16:50:21.949842 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:21.949724 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4ljtp" event={"ID":"cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7","Type":"ContainerStarted","Data":"7f53e38457372584db9108d2234ecadb186c48399857992cde555e3509bc64c4"} Apr 16 16:50:21.949842 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:21.949740 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4ljtp" event={"ID":"cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7","Type":"ContainerStarted","Data":"706b121755fb6ef174076d963dcf9957527a23815d6eb651977c633f647d3309"} Apr 16 16:50:21.949842 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:21.949742 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gk5x8" Apr 16 16:50:21.968759 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:21.968714 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-67db985cf-skv8z" podStartSLOduration=1.968700883 podStartE2EDuration="1.968700883s" podCreationTimestamp="2026-04-16 16:50:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 16:50:21.967066144 +0000 UTC m=+156.048800383" watchObservedRunningTime="2026-04-16 16:50:21.968700883 +0000 UTC m=+156.050435119" Apr 16 16:50:24.962281 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:24.962243 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69848b55b9-5mtcx" event={"ID":"c916378f-19b0-4da1-a0b8-7d1a88286786","Type":"ContainerStarted","Data":"2fbac51b0db9788e9fb7db1c57336a7bbfb2bace31fe5f7cbe85042300960b24"} Apr 16 16:50:24.964472 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:24.964398 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4ljtp" event={"ID":"cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7","Type":"ContainerStarted","Data":"2df35ed784285b98c8eb3c9b3da3da3bc4fbaedb37d81840206aa7214bf45d56"} Apr 16 16:50:24.980872 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:24.980824 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-69848b55b9-5mtcx" podStartSLOduration=1.066239545 podStartE2EDuration="4.980812784s" podCreationTimestamp="2026-04-16 16:50:20 +0000 UTC" firstStartedPulling="2026-04-16 16:50:20.810428468 +0000 UTC m=+154.892162663" lastFinishedPulling="2026-04-16 16:50:24.72500171 +0000 UTC m=+158.806735902" observedRunningTime="2026-04-16 16:50:24.979754609 +0000 UTC m=+159.061488823" watchObservedRunningTime="2026-04-16 16:50:24.980812784 +0000 UTC m=+159.062546992" Apr 16 16:50:24.996372 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:24.996335 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4ljtp" podStartSLOduration=1.364954034 podStartE2EDuration="4.996323041s" podCreationTimestamp="2026-04-16 16:50:20 +0000 UTC" firstStartedPulling="2026-04-16 16:50:21.096513569 +0000 UTC m=+155.178247762" lastFinishedPulling="2026-04-16 16:50:24.727882574 +0000 UTC m=+158.809616769" observedRunningTime="2026-04-16 16:50:24.995862002 +0000 UTC m=+159.077596217" watchObservedRunningTime="2026-04-16 16:50:24.996323041 +0000 UTC m=+159.078057256" Apr 16 16:50:26.402567 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:26.402479 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/430f1690-8090-4eaa-903e-a5aab4c24146-metrics-tls\") pod \"dns-default-gk5x8\" (UID: \"430f1690-8090-4eaa-903e-a5aab4c24146\") " pod="openshift-dns/dns-default-gk5x8" Apr 16 16:50:26.402567 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:26.402551 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-cert\") pod \"ingress-canary-8kc47\" (UID: \"a5cf9cec-6008-46d4-9c84-74af0d3ff9c7\") " pod="openshift-ingress-canary/ingress-canary-8kc47" Apr 16 16:50:26.404900 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:26.404868 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/430f1690-8090-4eaa-903e-a5aab4c24146-metrics-tls\") pod \"dns-default-gk5x8\" (UID: \"430f1690-8090-4eaa-903e-a5aab4c24146\") " pod="openshift-dns/dns-default-gk5x8" Apr 16 16:50:26.405048 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:26.405028 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5cf9cec-6008-46d4-9c84-74af0d3ff9c7-cert\") pod \"ingress-canary-8kc47\" (UID: \"a5cf9cec-6008-46d4-9c84-74af0d3ff9c7\") " pod="openshift-ingress-canary/ingress-canary-8kc47" Apr 16 16:50:26.452892 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:26.452868 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-99nwp\"" Apr 16 16:50:26.460194 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:26.460172 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gk5x8" Apr 16 16:50:26.587335 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:26.587305 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gk5x8"] Apr 16 16:50:26.589872 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:50:26.589837 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod430f1690_8090_4eaa_903e_a5aab4c24146.slice/crio-dc9a8ea815952fc606c5dc01b1e45275d66bb6c7219e390201a19e7910099f9e WatchSource:0}: Error finding container dc9a8ea815952fc606c5dc01b1e45275d66bb6c7219e390201a19e7910099f9e: Status 404 returned error can't find the container with id dc9a8ea815952fc606c5dc01b1e45275d66bb6c7219e390201a19e7910099f9e Apr 16 16:50:26.971483 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:26.971452 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gk5x8" event={"ID":"430f1690-8090-4eaa-903e-a5aab4c24146","Type":"ContainerStarted","Data":"dc9a8ea815952fc606c5dc01b1e45275d66bb6c7219e390201a19e7910099f9e"} Apr 16 16:50:28.980126 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:28.980089 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gk5x8" event={"ID":"430f1690-8090-4eaa-903e-a5aab4c24146","Type":"ContainerStarted","Data":"f5eb06211817fb1608d9027b526269a4d3bb1ecbc5db8cb691139620c9b3a4bd"} Apr 16 16:50:28.980126 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:28.980130 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gk5x8" event={"ID":"430f1690-8090-4eaa-903e-a5aab4c24146","Type":"ContainerStarted","Data":"9560dd9919c347197dd1df44cf4ec22e75a711cc4d8d71591e39714b9d110bb7"} Apr 16 16:50:28.980627 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:28.980210 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gk5x8" Apr 16 16:50:29.003020 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.002973 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-gk5x8" podStartSLOduration=129.637548905 podStartE2EDuration="2m11.00295736s" podCreationTimestamp="2026-04-16 16:48:18 +0000 UTC" firstStartedPulling="2026-04-16 16:50:26.592286042 +0000 UTC m=+160.674020235" lastFinishedPulling="2026-04-16 16:50:27.957694495 +0000 UTC m=+162.039428690" observedRunningTime="2026-04-16 16:50:29.001827724 +0000 UTC m=+163.083561938" watchObservedRunningTime="2026-04-16 16:50:29.00295736 +0000 UTC m=+163.084691557" Apr 16 16:50:29.164414 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.163006 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-5669946b84-dzpt2"] Apr 16 16:50:29.167309 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.166728 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-5669946b84-dzpt2" Apr 16 16:50:29.169494 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.169446 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Apr 16 16:50:29.170893 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.170094 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Apr 16 16:50:29.170893 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.170182 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Apr 16 16:50:29.170893 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.170240 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Apr 16 16:50:29.170893 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.170318 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-485bk\"" Apr 16 16:50:29.170893 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.170469 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Apr 16 16:50:29.178078 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.175946 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-5669946b84-dzpt2"] Apr 16 16:50:29.200941 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.200914 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-q9hmc"] Apr 16 16:50:29.205630 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.204887 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:29.209789 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.209683 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Apr 16 16:50:29.209789 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.209765 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Apr 16 16:50:29.209942 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.209870 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Apr 16 16:50:29.210005 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.209992 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-66nmg\"" Apr 16 16:50:29.229853 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.228718 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnd4r\" (UniqueName: \"kubernetes.io/projected/29fd6fae-df7e-47f5-b4cb-d7102b1b13ac-kube-api-access-nnd4r\") pod \"openshift-state-metrics-5669946b84-dzpt2\" (UID: \"29fd6fae-df7e-47f5-b4cb-d7102b1b13ac\") " pod="openshift-monitoring/openshift-state-metrics-5669946b84-dzpt2" Apr 16 16:50:29.229853 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.228774 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-node-exporter-textfile\") pod \"node-exporter-q9hmc\" (UID: \"196eabc2-d9ba-4b5e-987e-d3ba87cc1037\") " pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:29.229853 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.228802 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-node-exporter-accelerators-collector-config\") pod \"node-exporter-q9hmc\" (UID: \"196eabc2-d9ba-4b5e-987e-d3ba87cc1037\") " pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:29.229853 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.228858 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29fd6fae-df7e-47f5-b4cb-d7102b1b13ac-metrics-client-ca\") pod \"openshift-state-metrics-5669946b84-dzpt2\" (UID: \"29fd6fae-df7e-47f5-b4cb-d7102b1b13ac\") " pod="openshift-monitoring/openshift-state-metrics-5669946b84-dzpt2" Apr 16 16:50:29.229853 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.228905 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q9hmc\" (UID: \"196eabc2-d9ba-4b5e-987e-d3ba87cc1037\") " pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:29.229853 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.228938 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/29fd6fae-df7e-47f5-b4cb-d7102b1b13ac-openshift-state-metrics-tls\") pod \"openshift-state-metrics-5669946b84-dzpt2\" (UID: \"29fd6fae-df7e-47f5-b4cb-d7102b1b13ac\") " pod="openshift-monitoring/openshift-state-metrics-5669946b84-dzpt2" Apr 16 16:50:29.229853 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.228966 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-sys\") pod \"node-exporter-q9hmc\" (UID: \"196eabc2-d9ba-4b5e-987e-d3ba87cc1037\") " pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:29.229853 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.228994 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/29fd6fae-df7e-47f5-b4cb-d7102b1b13ac-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-5669946b84-dzpt2\" (UID: \"29fd6fae-df7e-47f5-b4cb-d7102b1b13ac\") " pod="openshift-monitoring/openshift-state-metrics-5669946b84-dzpt2" Apr 16 16:50:29.229853 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.229064 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-node-exporter-tls\") pod \"node-exporter-q9hmc\" (UID: \"196eabc2-d9ba-4b5e-987e-d3ba87cc1037\") " pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:29.229853 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.229097 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-metrics-client-ca\") pod \"node-exporter-q9hmc\" (UID: \"196eabc2-d9ba-4b5e-987e-d3ba87cc1037\") " pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:29.229853 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.229125 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-root\") pod \"node-exporter-q9hmc\" (UID: \"196eabc2-d9ba-4b5e-987e-d3ba87cc1037\") " pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:29.229853 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.229151 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-node-exporter-wtmp\") pod \"node-exporter-q9hmc\" (UID: \"196eabc2-d9ba-4b5e-987e-d3ba87cc1037\") " pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:29.229853 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.229187 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gntbv\" (UniqueName: \"kubernetes.io/projected/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-kube-api-access-gntbv\") pod \"node-exporter-q9hmc\" (UID: \"196eabc2-d9ba-4b5e-987e-d3ba87cc1037\") " pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:29.330289 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.330218 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nnd4r\" (UniqueName: \"kubernetes.io/projected/29fd6fae-df7e-47f5-b4cb-d7102b1b13ac-kube-api-access-nnd4r\") pod \"openshift-state-metrics-5669946b84-dzpt2\" (UID: \"29fd6fae-df7e-47f5-b4cb-d7102b1b13ac\") " pod="openshift-monitoring/openshift-state-metrics-5669946b84-dzpt2" Apr 16 16:50:29.330289 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.330272 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-node-exporter-textfile\") pod \"node-exporter-q9hmc\" (UID: \"196eabc2-d9ba-4b5e-987e-d3ba87cc1037\") " pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:29.330463 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.330299 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-node-exporter-accelerators-collector-config\") pod \"node-exporter-q9hmc\" (UID: \"196eabc2-d9ba-4b5e-987e-d3ba87cc1037\") " pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:29.330463 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.330340 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29fd6fae-df7e-47f5-b4cb-d7102b1b13ac-metrics-client-ca\") pod \"openshift-state-metrics-5669946b84-dzpt2\" (UID: \"29fd6fae-df7e-47f5-b4cb-d7102b1b13ac\") " pod="openshift-monitoring/openshift-state-metrics-5669946b84-dzpt2" Apr 16 16:50:29.330463 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.330365 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q9hmc\" (UID: \"196eabc2-d9ba-4b5e-987e-d3ba87cc1037\") " pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:29.330463 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.330392 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/29fd6fae-df7e-47f5-b4cb-d7102b1b13ac-openshift-state-metrics-tls\") pod \"openshift-state-metrics-5669946b84-dzpt2\" (UID: \"29fd6fae-df7e-47f5-b4cb-d7102b1b13ac\") " pod="openshift-monitoring/openshift-state-metrics-5669946b84-dzpt2" Apr 16 16:50:29.330463 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.330413 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-sys\") pod \"node-exporter-q9hmc\" (UID: \"196eabc2-d9ba-4b5e-987e-d3ba87cc1037\") " pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:29.330463 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.330441 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/29fd6fae-df7e-47f5-b4cb-d7102b1b13ac-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-5669946b84-dzpt2\" (UID: \"29fd6fae-df7e-47f5-b4cb-d7102b1b13ac\") " pod="openshift-monitoring/openshift-state-metrics-5669946b84-dzpt2" Apr 16 16:50:29.330737 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.330496 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-node-exporter-tls\") pod \"node-exporter-q9hmc\" (UID: \"196eabc2-d9ba-4b5e-987e-d3ba87cc1037\") " pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:29.330737 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.330525 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-metrics-client-ca\") pod \"node-exporter-q9hmc\" (UID: \"196eabc2-d9ba-4b5e-987e-d3ba87cc1037\") " pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:29.330737 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.330548 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-root\") pod \"node-exporter-q9hmc\" (UID: \"196eabc2-d9ba-4b5e-987e-d3ba87cc1037\") " pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:29.330737 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.330573 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-node-exporter-wtmp\") pod \"node-exporter-q9hmc\" (UID: \"196eabc2-d9ba-4b5e-987e-d3ba87cc1037\") " pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:29.330737 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.330626 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gntbv\" (UniqueName: \"kubernetes.io/projected/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-kube-api-access-gntbv\") pod \"node-exporter-q9hmc\" (UID: \"196eabc2-d9ba-4b5e-987e-d3ba87cc1037\") " pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:29.331014 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.330777 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-node-exporter-textfile\") pod \"node-exporter-q9hmc\" (UID: \"196eabc2-d9ba-4b5e-987e-d3ba87cc1037\") " pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:29.331160 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.331135 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29fd6fae-df7e-47f5-b4cb-d7102b1b13ac-metrics-client-ca\") pod \"openshift-state-metrics-5669946b84-dzpt2\" (UID: \"29fd6fae-df7e-47f5-b4cb-d7102b1b13ac\") " pod="openshift-monitoring/openshift-state-metrics-5669946b84-dzpt2" Apr 16 16:50:29.331839 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.331292 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-node-exporter-accelerators-collector-config\") pod \"node-exporter-q9hmc\" (UID: \"196eabc2-d9ba-4b5e-987e-d3ba87cc1037\") " pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:29.331839 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.331297 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-root\") pod \"node-exporter-q9hmc\" (UID: \"196eabc2-d9ba-4b5e-987e-d3ba87cc1037\") " pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:29.331839 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:50:29.331301 2574 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Apr 16 16:50:29.331839 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:50:29.331386 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-node-exporter-tls podName:196eabc2-d9ba-4b5e-987e-d3ba87cc1037 nodeName:}" failed. No retries permitted until 2026-04-16 16:50:29.831366411 +0000 UTC m=+163.913100618 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-node-exporter-tls") pod "node-exporter-q9hmc" (UID: "196eabc2-d9ba-4b5e-987e-d3ba87cc1037") : secret "node-exporter-tls" not found Apr 16 16:50:29.331839 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.331440 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-node-exporter-wtmp\") pod \"node-exporter-q9hmc\" (UID: \"196eabc2-d9ba-4b5e-987e-d3ba87cc1037\") " pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:29.331839 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:50:29.331520 2574 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Apr 16 16:50:29.331839 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:50:29.331576 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29fd6fae-df7e-47f5-b4cb-d7102b1b13ac-openshift-state-metrics-tls podName:29fd6fae-df7e-47f5-b4cb-d7102b1b13ac nodeName:}" failed. No retries permitted until 2026-04-16 16:50:29.831560841 +0000 UTC m=+163.913295054 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/29fd6fae-df7e-47f5-b4cb-d7102b1b13ac-openshift-state-metrics-tls") pod "openshift-state-metrics-5669946b84-dzpt2" (UID: "29fd6fae-df7e-47f5-b4cb-d7102b1b13ac") : secret "openshift-state-metrics-tls" not found Apr 16 16:50:29.331839 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.331614 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-sys\") pod \"node-exporter-q9hmc\" (UID: \"196eabc2-d9ba-4b5e-987e-d3ba87cc1037\") " pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:29.331839 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.331777 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-metrics-client-ca\") pod \"node-exporter-q9hmc\" (UID: \"196eabc2-d9ba-4b5e-987e-d3ba87cc1037\") " pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:29.333633 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.333614 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q9hmc\" (UID: \"196eabc2-d9ba-4b5e-987e-d3ba87cc1037\") " pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:29.334129 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.334097 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/29fd6fae-df7e-47f5-b4cb-d7102b1b13ac-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-5669946b84-dzpt2\" (UID: \"29fd6fae-df7e-47f5-b4cb-d7102b1b13ac\") " pod="openshift-monitoring/openshift-state-metrics-5669946b84-dzpt2" Apr 16 16:50:29.339930 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.339892 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gntbv\" (UniqueName: \"kubernetes.io/projected/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-kube-api-access-gntbv\") pod \"node-exporter-q9hmc\" (UID: \"196eabc2-d9ba-4b5e-987e-d3ba87cc1037\") " pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:29.341453 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.341430 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnd4r\" (UniqueName: \"kubernetes.io/projected/29fd6fae-df7e-47f5-b4cb-d7102b1b13ac-kube-api-access-nnd4r\") pod \"openshift-state-metrics-5669946b84-dzpt2\" (UID: \"29fd6fae-df7e-47f5-b4cb-d7102b1b13ac\") " pod="openshift-monitoring/openshift-state-metrics-5669946b84-dzpt2" Apr 16 16:50:29.835978 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.835945 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-node-exporter-tls\") pod \"node-exporter-q9hmc\" (UID: \"196eabc2-d9ba-4b5e-987e-d3ba87cc1037\") " pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:29.836135 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.836036 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/29fd6fae-df7e-47f5-b4cb-d7102b1b13ac-openshift-state-metrics-tls\") pod \"openshift-state-metrics-5669946b84-dzpt2\" (UID: \"29fd6fae-df7e-47f5-b4cb-d7102b1b13ac\") " pod="openshift-monitoring/openshift-state-metrics-5669946b84-dzpt2" Apr 16 16:50:29.838533 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.838506 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/196eabc2-d9ba-4b5e-987e-d3ba87cc1037-node-exporter-tls\") pod \"node-exporter-q9hmc\" (UID: \"196eabc2-d9ba-4b5e-987e-d3ba87cc1037\") " pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:29.838533 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:29.838526 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/29fd6fae-df7e-47f5-b4cb-d7102b1b13ac-openshift-state-metrics-tls\") pod \"openshift-state-metrics-5669946b84-dzpt2\" (UID: \"29fd6fae-df7e-47f5-b4cb-d7102b1b13ac\") " pod="openshift-monitoring/openshift-state-metrics-5669946b84-dzpt2" Apr 16 16:50:30.081895 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:30.081864 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-5669946b84-dzpt2" Apr 16 16:50:30.130580 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:30.130215 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q9hmc" Apr 16 16:50:30.626441 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:30.626414 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-69848b55b9-5mtcx" Apr 16 16:50:30.626441 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:30.626457 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-69848b55b9-5mtcx" Apr 16 16:50:30.627785 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:30.627761 2574 patch_prober.go:28] interesting pod/console-69848b55b9-5mtcx container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.15:8443/health\": dial tcp 10.134.0.15:8443: connect: connection refused" start-of-body= Apr 16 16:50:30.627910 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:30.627828 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-69848b55b9-5mtcx" podUID="c916378f-19b0-4da1-a0b8-7d1a88286786" containerName="console" probeResult="failure" output="Get \"https://10.134.0.15:8443/health\": dial tcp 10.134.0.15:8443: connect: connection refused" Apr 16 16:50:33.486502 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.486467 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-54997c8455-sxv9m"] Apr 16 16:50:33.490537 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.490510 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" Apr 16 16:50:33.494756 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.494733 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Apr 16 16:50:33.494756 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.494745 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Apr 16 16:50:33.494919 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.494741 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Apr 16 16:50:33.494919 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.494851 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4qgusdbjop99o\"" Apr 16 16:50:33.495040 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.494967 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-slgxd\"" Apr 16 16:50:33.495040 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.495030 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Apr 16 16:50:33.497546 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.497524 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-54997c8455-sxv9m"] Apr 16 16:50:33.569386 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.569352 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/616efe3e-e6dd-427e-b9ee-db6ac7365ae8-audit-log\") pod \"metrics-server-54997c8455-sxv9m\" (UID: \"616efe3e-e6dd-427e-b9ee-db6ac7365ae8\") " pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" Apr 16 16:50:33.569505 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.569399 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnlrl\" (UniqueName: \"kubernetes.io/projected/616efe3e-e6dd-427e-b9ee-db6ac7365ae8-kube-api-access-nnlrl\") pod \"metrics-server-54997c8455-sxv9m\" (UID: \"616efe3e-e6dd-427e-b9ee-db6ac7365ae8\") " pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" Apr 16 16:50:33.569505 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.569437 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/616efe3e-e6dd-427e-b9ee-db6ac7365ae8-secret-metrics-server-client-certs\") pod \"metrics-server-54997c8455-sxv9m\" (UID: \"616efe3e-e6dd-427e-b9ee-db6ac7365ae8\") " pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" Apr 16 16:50:33.569505 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.569472 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/616efe3e-e6dd-427e-b9ee-db6ac7365ae8-client-ca-bundle\") pod \"metrics-server-54997c8455-sxv9m\" (UID: \"616efe3e-e6dd-427e-b9ee-db6ac7365ae8\") " pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" Apr 16 16:50:33.569654 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.569504 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/616efe3e-e6dd-427e-b9ee-db6ac7365ae8-metrics-server-audit-profiles\") pod \"metrics-server-54997c8455-sxv9m\" (UID: \"616efe3e-e6dd-427e-b9ee-db6ac7365ae8\") " pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" Apr 16 16:50:33.569654 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.569554 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/616efe3e-e6dd-427e-b9ee-db6ac7365ae8-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-54997c8455-sxv9m\" (UID: \"616efe3e-e6dd-427e-b9ee-db6ac7365ae8\") " pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" Apr 16 16:50:33.569746 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.569648 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/616efe3e-e6dd-427e-b9ee-db6ac7365ae8-secret-metrics-server-tls\") pod \"metrics-server-54997c8455-sxv9m\" (UID: \"616efe3e-e6dd-427e-b9ee-db6ac7365ae8\") " pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" Apr 16 16:50:33.670322 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.670287 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/616efe3e-e6dd-427e-b9ee-db6ac7365ae8-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-54997c8455-sxv9m\" (UID: \"616efe3e-e6dd-427e-b9ee-db6ac7365ae8\") " pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" Apr 16 16:50:33.670458 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.670358 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/616efe3e-e6dd-427e-b9ee-db6ac7365ae8-secret-metrics-server-tls\") pod \"metrics-server-54997c8455-sxv9m\" (UID: \"616efe3e-e6dd-427e-b9ee-db6ac7365ae8\") " pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" Apr 16 16:50:33.670458 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.670413 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/616efe3e-e6dd-427e-b9ee-db6ac7365ae8-audit-log\") pod \"metrics-server-54997c8455-sxv9m\" (UID: \"616efe3e-e6dd-427e-b9ee-db6ac7365ae8\") " pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" Apr 16 16:50:33.670458 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.670445 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nnlrl\" (UniqueName: \"kubernetes.io/projected/616efe3e-e6dd-427e-b9ee-db6ac7365ae8-kube-api-access-nnlrl\") pod \"metrics-server-54997c8455-sxv9m\" (UID: \"616efe3e-e6dd-427e-b9ee-db6ac7365ae8\") " pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" Apr 16 16:50:33.670615 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.670474 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/616efe3e-e6dd-427e-b9ee-db6ac7365ae8-secret-metrics-server-client-certs\") pod \"metrics-server-54997c8455-sxv9m\" (UID: \"616efe3e-e6dd-427e-b9ee-db6ac7365ae8\") " pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" Apr 16 16:50:33.670615 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.670512 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/616efe3e-e6dd-427e-b9ee-db6ac7365ae8-client-ca-bundle\") pod \"metrics-server-54997c8455-sxv9m\" (UID: \"616efe3e-e6dd-427e-b9ee-db6ac7365ae8\") " pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" Apr 16 16:50:33.670615 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.670539 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/616efe3e-e6dd-427e-b9ee-db6ac7365ae8-metrics-server-audit-profiles\") pod \"metrics-server-54997c8455-sxv9m\" (UID: \"616efe3e-e6dd-427e-b9ee-db6ac7365ae8\") " pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" Apr 16 16:50:33.670989 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.670953 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/616efe3e-e6dd-427e-b9ee-db6ac7365ae8-audit-log\") pod \"metrics-server-54997c8455-sxv9m\" (UID: \"616efe3e-e6dd-427e-b9ee-db6ac7365ae8\") " pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" Apr 16 16:50:33.671234 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.671165 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/616efe3e-e6dd-427e-b9ee-db6ac7365ae8-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-54997c8455-sxv9m\" (UID: \"616efe3e-e6dd-427e-b9ee-db6ac7365ae8\") " pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" Apr 16 16:50:33.671467 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.671423 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/616efe3e-e6dd-427e-b9ee-db6ac7365ae8-metrics-server-audit-profiles\") pod \"metrics-server-54997c8455-sxv9m\" (UID: \"616efe3e-e6dd-427e-b9ee-db6ac7365ae8\") " pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" Apr 16 16:50:33.673324 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.673299 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/616efe3e-e6dd-427e-b9ee-db6ac7365ae8-secret-metrics-server-client-certs\") pod \"metrics-server-54997c8455-sxv9m\" (UID: \"616efe3e-e6dd-427e-b9ee-db6ac7365ae8\") " pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" Apr 16 16:50:33.673602 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.673582 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/616efe3e-e6dd-427e-b9ee-db6ac7365ae8-client-ca-bundle\") pod \"metrics-server-54997c8455-sxv9m\" (UID: \"616efe3e-e6dd-427e-b9ee-db6ac7365ae8\") " pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" Apr 16 16:50:33.673695 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.673587 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/616efe3e-e6dd-427e-b9ee-db6ac7365ae8-secret-metrics-server-tls\") pod \"metrics-server-54997c8455-sxv9m\" (UID: \"616efe3e-e6dd-427e-b9ee-db6ac7365ae8\") " pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" Apr 16 16:50:33.678958 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.678937 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnlrl\" (UniqueName: \"kubernetes.io/projected/616efe3e-e6dd-427e-b9ee-db6ac7365ae8-kube-api-access-nnlrl\") pod \"metrics-server-54997c8455-sxv9m\" (UID: \"616efe3e-e6dd-427e-b9ee-db6ac7365ae8\") " pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" Apr 16 16:50:33.803138 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:33.803052 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" Apr 16 16:50:34.372625 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.372593 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69848b55b9-5mtcx"] Apr 16 16:50:34.400111 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.400088 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-576956df6b-6gsqb"] Apr 16 16:50:34.403870 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.403850 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-576956df6b-6gsqb" Apr 16 16:50:34.416012 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.413821 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-576956df6b-6gsqb"] Apr 16 16:50:34.416012 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.415770 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Apr 16 16:50:34.478500 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.478471 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd30943e-c1ef-4b0f-b447-4a73893f1364-oauth-serving-cert\") pod \"console-576956df6b-6gsqb\" (UID: \"fd30943e-c1ef-4b0f-b447-4a73893f1364\") " pod="openshift-console/console-576956df6b-6gsqb" Apr 16 16:50:34.478500 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.478503 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd30943e-c1ef-4b0f-b447-4a73893f1364-console-oauth-config\") pod \"console-576956df6b-6gsqb\" (UID: \"fd30943e-c1ef-4b0f-b447-4a73893f1364\") " pod="openshift-console/console-576956df6b-6gsqb" Apr 16 16:50:34.478693 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.478531 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd30943e-c1ef-4b0f-b447-4a73893f1364-console-serving-cert\") pod \"console-576956df6b-6gsqb\" (UID: \"fd30943e-c1ef-4b0f-b447-4a73893f1364\") " pod="openshift-console/console-576956df6b-6gsqb" Apr 16 16:50:34.478693 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.478564 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd30943e-c1ef-4b0f-b447-4a73893f1364-service-ca\") pod \"console-576956df6b-6gsqb\" (UID: \"fd30943e-c1ef-4b0f-b447-4a73893f1364\") " pod="openshift-console/console-576956df6b-6gsqb" Apr 16 16:50:34.478693 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.478595 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd30943e-c1ef-4b0f-b447-4a73893f1364-trusted-ca-bundle\") pod \"console-576956df6b-6gsqb\" (UID: \"fd30943e-c1ef-4b0f-b447-4a73893f1364\") " pod="openshift-console/console-576956df6b-6gsqb" Apr 16 16:50:34.478693 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.478618 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdl64\" (UniqueName: \"kubernetes.io/projected/fd30943e-c1ef-4b0f-b447-4a73893f1364-kube-api-access-tdl64\") pod \"console-576956df6b-6gsqb\" (UID: \"fd30943e-c1ef-4b0f-b447-4a73893f1364\") " pod="openshift-console/console-576956df6b-6gsqb" Apr 16 16:50:34.478897 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.478742 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd30943e-c1ef-4b0f-b447-4a73893f1364-console-config\") pod \"console-576956df6b-6gsqb\" (UID: \"fd30943e-c1ef-4b0f-b447-4a73893f1364\") " pod="openshift-console/console-576956df6b-6gsqb" Apr 16 16:50:34.507512 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.507473 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8kc47" Apr 16 16:50:34.507888 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.507660 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:50:34.510364 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.510346 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-c25vw\"" Apr 16 16:50:34.518519 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.518502 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8kc47" Apr 16 16:50:34.579878 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.579852 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd30943e-c1ef-4b0f-b447-4a73893f1364-console-config\") pod \"console-576956df6b-6gsqb\" (UID: \"fd30943e-c1ef-4b0f-b447-4a73893f1364\") " pod="openshift-console/console-576956df6b-6gsqb" Apr 16 16:50:34.579989 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.579917 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd30943e-c1ef-4b0f-b447-4a73893f1364-oauth-serving-cert\") pod \"console-576956df6b-6gsqb\" (UID: \"fd30943e-c1ef-4b0f-b447-4a73893f1364\") " pod="openshift-console/console-576956df6b-6gsqb" Apr 16 16:50:34.579989 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.579945 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd30943e-c1ef-4b0f-b447-4a73893f1364-console-oauth-config\") pod \"console-576956df6b-6gsqb\" (UID: \"fd30943e-c1ef-4b0f-b447-4a73893f1364\") " pod="openshift-console/console-576956df6b-6gsqb" Apr 16 16:50:34.579989 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.579969 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd30943e-c1ef-4b0f-b447-4a73893f1364-console-serving-cert\") pod \"console-576956df6b-6gsqb\" (UID: \"fd30943e-c1ef-4b0f-b447-4a73893f1364\") " pod="openshift-console/console-576956df6b-6gsqb" Apr 16 16:50:34.580168 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.579994 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd30943e-c1ef-4b0f-b447-4a73893f1364-service-ca\") pod \"console-576956df6b-6gsqb\" (UID: \"fd30943e-c1ef-4b0f-b447-4a73893f1364\") " pod="openshift-console/console-576956df6b-6gsqb" Apr 16 16:50:34.580168 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.580032 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd30943e-c1ef-4b0f-b447-4a73893f1364-trusted-ca-bundle\") pod \"console-576956df6b-6gsqb\" (UID: \"fd30943e-c1ef-4b0f-b447-4a73893f1364\") " pod="openshift-console/console-576956df6b-6gsqb" Apr 16 16:50:34.580915 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.580705 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd30943e-c1ef-4b0f-b447-4a73893f1364-oauth-serving-cert\") pod \"console-576956df6b-6gsqb\" (UID: \"fd30943e-c1ef-4b0f-b447-4a73893f1364\") " pod="openshift-console/console-576956df6b-6gsqb" Apr 16 16:50:34.580915 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.580764 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tdl64\" (UniqueName: \"kubernetes.io/projected/fd30943e-c1ef-4b0f-b447-4a73893f1364-kube-api-access-tdl64\") pod \"console-576956df6b-6gsqb\" (UID: \"fd30943e-c1ef-4b0f-b447-4a73893f1364\") " pod="openshift-console/console-576956df6b-6gsqb" Apr 16 16:50:34.580915 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.580771 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd30943e-c1ef-4b0f-b447-4a73893f1364-console-config\") pod \"console-576956df6b-6gsqb\" (UID: \"fd30943e-c1ef-4b0f-b447-4a73893f1364\") " pod="openshift-console/console-576956df6b-6gsqb" Apr 16 16:50:34.581137 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.580926 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd30943e-c1ef-4b0f-b447-4a73893f1364-service-ca\") pod \"console-576956df6b-6gsqb\" (UID: \"fd30943e-c1ef-4b0f-b447-4a73893f1364\") " pod="openshift-console/console-576956df6b-6gsqb" Apr 16 16:50:34.581332 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.581309 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd30943e-c1ef-4b0f-b447-4a73893f1364-trusted-ca-bundle\") pod \"console-576956df6b-6gsqb\" (UID: \"fd30943e-c1ef-4b0f-b447-4a73893f1364\") " pod="openshift-console/console-576956df6b-6gsqb" Apr 16 16:50:34.583538 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.583514 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd30943e-c1ef-4b0f-b447-4a73893f1364-console-oauth-config\") pod \"console-576956df6b-6gsqb\" (UID: \"fd30943e-c1ef-4b0f-b447-4a73893f1364\") " pod="openshift-console/console-576956df6b-6gsqb" Apr 16 16:50:34.583642 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.583521 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd30943e-c1ef-4b0f-b447-4a73893f1364-console-serving-cert\") pod \"console-576956df6b-6gsqb\" (UID: \"fd30943e-c1ef-4b0f-b447-4a73893f1364\") " pod="openshift-console/console-576956df6b-6gsqb" Apr 16 16:50:34.588864 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.588832 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdl64\" (UniqueName: \"kubernetes.io/projected/fd30943e-c1ef-4b0f-b447-4a73893f1364-kube-api-access-tdl64\") pod \"console-576956df6b-6gsqb\" (UID: \"fd30943e-c1ef-4b0f-b447-4a73893f1364\") " pod="openshift-console/console-576956df6b-6gsqb" Apr 16 16:50:34.719699 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:34.719615 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-576956df6b-6gsqb" Apr 16 16:50:36.636546 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:36.636454 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-5669946b84-dzpt2"] Apr 16 16:50:36.663392 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:50:36.663356 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29fd6fae_df7e_47f5_b4cb_d7102b1b13ac.slice/crio-c7335a50e17996e1e1aa6017abe8d5cbe0dc30019a3fb1ac468a9d4ce7d20d4f WatchSource:0}: Error finding container c7335a50e17996e1e1aa6017abe8d5cbe0dc30019a3fb1ac468a9d4ce7d20d4f: Status 404 returned error can't find the container with id c7335a50e17996e1e1aa6017abe8d5cbe0dc30019a3fb1ac468a9d4ce7d20d4f Apr 16 16:50:36.860524 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:36.860436 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8kc47"] Apr 16 16:50:36.862970 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:50:36.862928 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5cf9cec_6008_46d4_9c84_74af0d3ff9c7.slice/crio-10649874a2563168e55d0606feb5b85f4ec25359812045a1ef45b08e4a84b07d WatchSource:0}: Error finding container 10649874a2563168e55d0606feb5b85f4ec25359812045a1ef45b08e4a84b07d: Status 404 returned error can't find the container with id 10649874a2563168e55d0606feb5b85f4ec25359812045a1ef45b08e4a84b07d Apr 16 16:50:36.866002 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:36.865905 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-576956df6b-6gsqb"] Apr 16 16:50:36.868134 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:50:36.868076 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd30943e_c1ef_4b0f_b447_4a73893f1364.slice/crio-0548e575060a29ff47c71ae6b83f9b534d8d255a941690943e7e6c59b3b89d90 WatchSource:0}: Error finding container 0548e575060a29ff47c71ae6b83f9b534d8d255a941690943e7e6c59b3b89d90: Status 404 returned error can't find the container with id 0548e575060a29ff47c71ae6b83f9b534d8d255a941690943e7e6c59b3b89d90 Apr 16 16:50:36.868819 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:36.868802 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-54997c8455-sxv9m"] Apr 16 16:50:36.871759 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:50:36.871734 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod616efe3e_e6dd_427e_b9ee_db6ac7365ae8.slice/crio-6c15e59c7dab8d53ae929a34d2a6f70516b0d2de38f83486cbf1ab781149fcff WatchSource:0}: Error finding container 6c15e59c7dab8d53ae929a34d2a6f70516b0d2de38f83486cbf1ab781149fcff: Status 404 returned error can't find the container with id 6c15e59c7dab8d53ae929a34d2a6f70516b0d2de38f83486cbf1ab781149fcff Apr 16 16:50:37.007562 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:37.007431 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-576956df6b-6gsqb" event={"ID":"fd30943e-c1ef-4b0f-b447-4a73893f1364","Type":"ContainerStarted","Data":"91ad670148e2b2ffeabe93da38c45f6478f1afae90be59cb169e00d1bf2c30ec"} Apr 16 16:50:37.007562 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:37.007476 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-576956df6b-6gsqb" event={"ID":"fd30943e-c1ef-4b0f-b447-4a73893f1364","Type":"ContainerStarted","Data":"0548e575060a29ff47c71ae6b83f9b534d8d255a941690943e7e6c59b3b89d90"} Apr 16 16:50:37.009392 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:37.009330 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8kc47" event={"ID":"a5cf9cec-6008-46d4-9c84-74af0d3ff9c7","Type":"ContainerStarted","Data":"10649874a2563168e55d0606feb5b85f4ec25359812045a1ef45b08e4a84b07d"} Apr 16 16:50:37.010542 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:37.010520 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" event={"ID":"616efe3e-e6dd-427e-b9ee-db6ac7365ae8","Type":"ContainerStarted","Data":"6c15e59c7dab8d53ae929a34d2a6f70516b0d2de38f83486cbf1ab781149fcff"} Apr 16 16:50:37.013193 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:37.013132 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-5669946b84-dzpt2" event={"ID":"29fd6fae-df7e-47f5-b4cb-d7102b1b13ac","Type":"ContainerStarted","Data":"89d8ef01b2962d9e4d1f8648393b40fb6884e28a653a8b132148f668dd82bb74"} Apr 16 16:50:37.013193 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:37.013162 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-5669946b84-dzpt2" event={"ID":"29fd6fae-df7e-47f5-b4cb-d7102b1b13ac","Type":"ContainerStarted","Data":"971eace4160d73d2dc4fbd10d6374fa6a2aae9aa0e943be89edb95c2c2619c0b"} Apr 16 16:50:37.013193 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:37.013175 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-5669946b84-dzpt2" event={"ID":"29fd6fae-df7e-47f5-b4cb-d7102b1b13ac","Type":"ContainerStarted","Data":"c7335a50e17996e1e1aa6017abe8d5cbe0dc30019a3fb1ac468a9d4ce7d20d4f"} Apr 16 16:50:37.014489 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:37.014454 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q9hmc" event={"ID":"196eabc2-d9ba-4b5e-987e-d3ba87cc1037","Type":"ContainerStarted","Data":"31d09dd1de156b709b24ad7313cf6bb32af3774cab79556903d4bc3de267cc0c"} Apr 16 16:50:37.017028 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:37.017006 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-586b57c7b4-4pm8j" event={"ID":"5a9d593a-f4f5-4786-849a-1330254f9b1c","Type":"ContainerStarted","Data":"0f2cc8c3356c8f34686e1551687ec7e8c0e5531cb7739b74ef58e3afd73346cd"} Apr 16 16:50:37.017690 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:37.017562 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-586b57c7b4-4pm8j" Apr 16 16:50:37.025634 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:37.025598 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-576956df6b-6gsqb" podStartSLOduration=3.025587193 podStartE2EDuration="3.025587193s" podCreationTimestamp="2026-04-16 16:50:34 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 16:50:37.023808419 +0000 UTC m=+171.105542664" watchObservedRunningTime="2026-04-16 16:50:37.025587193 +0000 UTC m=+171.107321385" Apr 16 16:50:37.033454 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:37.033425 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-586b57c7b4-4pm8j" Apr 16 16:50:37.040631 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:37.040598 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/downloads-586b57c7b4-4pm8j" podStartSLOduration=1.233978761 podStartE2EDuration="17.040587195s" podCreationTimestamp="2026-04-16 16:50:20 +0000 UTC" firstStartedPulling="2026-04-16 16:50:20.793248397 +0000 UTC m=+154.874982590" lastFinishedPulling="2026-04-16 16:50:36.599856814 +0000 UTC m=+170.681591024" observedRunningTime="2026-04-16 16:50:37.04014112 +0000 UTC m=+171.121875336" watchObservedRunningTime="2026-04-16 16:50:37.040587195 +0000 UTC m=+171.122321409" Apr 16 16:50:38.023530 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:38.023281 2574 generic.go:358] "Generic (PLEG): container finished" podID="196eabc2-d9ba-4b5e-987e-d3ba87cc1037" containerID="38e703e010e63a563fc37ff80f2d1fbe94d638836b9d8b9118d1783566bc2e8b" exitCode=0 Apr 16 16:50:38.023530 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:38.023449 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q9hmc" event={"ID":"196eabc2-d9ba-4b5e-987e-d3ba87cc1037","Type":"ContainerDied","Data":"38e703e010e63a563fc37ff80f2d1fbe94d638836b9d8b9118d1783566bc2e8b"} Apr 16 16:50:38.985368 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:38.985252 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gk5x8" Apr 16 16:50:40.653672 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:40.653628 2574 patch_prober.go:28] interesting pod/image-registry-67db985cf-skv8z container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Apr 16 16:50:40.654079 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:40.653690 2574 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-67db985cf-skv8z" podUID="dc5c6566-f0a7-42ab-ae2f-8a5b0846b043" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Apr 16 16:50:41.034394 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:41.034308 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8kc47" event={"ID":"a5cf9cec-6008-46d4-9c84-74af0d3ff9c7","Type":"ContainerStarted","Data":"2ee6be7c7e53cbd556686f4b5908972034c151d39754364e9a4f26b51edf8fe1"} Apr 16 16:50:41.035843 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:41.035809 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" event={"ID":"616efe3e-e6dd-427e-b9ee-db6ac7365ae8","Type":"ContainerStarted","Data":"318f1a6545dfcca1c017e85a1f0e611351a3478bf256a922a9c17f985233cd85"} Apr 16 16:50:41.037945 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:41.037905 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-5669946b84-dzpt2" event={"ID":"29fd6fae-df7e-47f5-b4cb-d7102b1b13ac","Type":"ContainerStarted","Data":"6f8652048c8d712fd9a498a772ae71ee43ec206a2df0cd4d475894a41f97799b"} Apr 16 16:50:41.039970 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:41.039938 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q9hmc" event={"ID":"196eabc2-d9ba-4b5e-987e-d3ba87cc1037","Type":"ContainerStarted","Data":"909b75d9034925588909e76779d53f6ffda11343cd3cbccf7e3e3ba0fb861c2e"} Apr 16 16:50:41.040064 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:41.039972 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q9hmc" event={"ID":"196eabc2-d9ba-4b5e-987e-d3ba87cc1037","Type":"ContainerStarted","Data":"69f98aa1518adbff5c122154b4fa8a0225499fadea240df2a188b018139a2f22"} Apr 16 16:50:41.052015 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:41.051967 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8kc47" podStartSLOduration=139.904324523 podStartE2EDuration="2m23.051952076s" podCreationTimestamp="2026-04-16 16:48:18 +0000 UTC" firstStartedPulling="2026-04-16 16:50:36.865346493 +0000 UTC m=+170.947080686" lastFinishedPulling="2026-04-16 16:50:40.012974036 +0000 UTC m=+174.094708239" observedRunningTime="2026-04-16 16:50:41.051583453 +0000 UTC m=+175.133317681" watchObservedRunningTime="2026-04-16 16:50:41.051952076 +0000 UTC m=+175.133686291" Apr 16 16:50:41.069092 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:41.069032 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-q9hmc" podStartSLOduration=11.296037755 podStartE2EDuration="12.069015282s" podCreationTimestamp="2026-04-16 16:50:29 +0000 UTC" firstStartedPulling="2026-04-16 16:50:36.524499879 +0000 UTC m=+170.606234079" lastFinishedPulling="2026-04-16 16:50:37.297477401 +0000 UTC m=+171.379211606" observedRunningTime="2026-04-16 16:50:41.068462802 +0000 UTC m=+175.150197018" watchObservedRunningTime="2026-04-16 16:50:41.069015282 +0000 UTC m=+175.150749496" Apr 16 16:50:41.085328 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:41.085285 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" podStartSLOduration=4.950921331 podStartE2EDuration="8.08527312s" podCreationTimestamp="2026-04-16 16:50:33 +0000 UTC" firstStartedPulling="2026-04-16 16:50:36.873730202 +0000 UTC m=+170.955464399" lastFinishedPulling="2026-04-16 16:50:40.00808199 +0000 UTC m=+174.089816188" observedRunningTime="2026-04-16 16:50:41.083843654 +0000 UTC m=+175.165577891" watchObservedRunningTime="2026-04-16 16:50:41.08527312 +0000 UTC m=+175.167007370" Apr 16 16:50:41.100636 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:41.100591 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-5669946b84-dzpt2" podStartSLOduration=8.871661279 podStartE2EDuration="12.100579512s" podCreationTimestamp="2026-04-16 16:50:29 +0000 UTC" firstStartedPulling="2026-04-16 16:50:36.779162373 +0000 UTC m=+170.860896565" lastFinishedPulling="2026-04-16 16:50:40.008080592 +0000 UTC m=+174.089814798" observedRunningTime="2026-04-16 16:50:41.100101694 +0000 UTC m=+175.181835899" watchObservedRunningTime="2026-04-16 16:50:41.100579512 +0000 UTC m=+175.182313727" Apr 16 16:50:42.959881 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:42.959849 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-67db985cf-skv8z" Apr 16 16:50:44.720516 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:44.720481 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-576956df6b-6gsqb" Apr 16 16:50:44.720516 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:44.720519 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-576956df6b-6gsqb" Apr 16 16:50:44.725778 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:44.725754 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-576956df6b-6gsqb" Apr 16 16:50:45.055619 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:45.055544 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-576956df6b-6gsqb" Apr 16 16:50:53.803875 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:53.803841 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" Apr 16 16:50:53.803875 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:53.803881 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" Apr 16 16:50:59.392609 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:59.392538 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-69848b55b9-5mtcx" podUID="c916378f-19b0-4da1-a0b8-7d1a88286786" containerName="console" containerID="cri-o://2fbac51b0db9788e9fb7db1c57336a7bbfb2bace31fe5f7cbe85042300960b24" gracePeriod=15 Apr 16 16:50:59.635249 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:59.635229 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-69848b55b9-5mtcx_c916378f-19b0-4da1-a0b8-7d1a88286786/console/0.log" Apr 16 16:50:59.635351 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:59.635292 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69848b55b9-5mtcx" Apr 16 16:50:59.796569 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:59.796504 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c916378f-19b0-4da1-a0b8-7d1a88286786-console-oauth-config\") pod \"c916378f-19b0-4da1-a0b8-7d1a88286786\" (UID: \"c916378f-19b0-4da1-a0b8-7d1a88286786\") " Apr 16 16:50:59.796569 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:59.796541 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c916378f-19b0-4da1-a0b8-7d1a88286786-oauth-serving-cert\") pod \"c916378f-19b0-4da1-a0b8-7d1a88286786\" (UID: \"c916378f-19b0-4da1-a0b8-7d1a88286786\") " Apr 16 16:50:59.796758 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:59.796587 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c916378f-19b0-4da1-a0b8-7d1a88286786-console-config\") pod \"c916378f-19b0-4da1-a0b8-7d1a88286786\" (UID: \"c916378f-19b0-4da1-a0b8-7d1a88286786\") " Apr 16 16:50:59.796758 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:59.796610 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c916378f-19b0-4da1-a0b8-7d1a88286786-console-serving-cert\") pod \"c916378f-19b0-4da1-a0b8-7d1a88286786\" (UID: \"c916378f-19b0-4da1-a0b8-7d1a88286786\") " Apr 16 16:50:59.796758 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:59.796635 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c916378f-19b0-4da1-a0b8-7d1a88286786-service-ca\") pod \"c916378f-19b0-4da1-a0b8-7d1a88286786\" (UID: \"c916378f-19b0-4da1-a0b8-7d1a88286786\") " Apr 16 16:50:59.796758 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:59.796682 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-stjqv\" (UniqueName: \"kubernetes.io/projected/c916378f-19b0-4da1-a0b8-7d1a88286786-kube-api-access-stjqv\") pod \"c916378f-19b0-4da1-a0b8-7d1a88286786\" (UID: \"c916378f-19b0-4da1-a0b8-7d1a88286786\") " Apr 16 16:50:59.797024 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:59.797003 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c916378f-19b0-4da1-a0b8-7d1a88286786-console-config" (OuterVolumeSpecName: "console-config") pod "c916378f-19b0-4da1-a0b8-7d1a88286786" (UID: "c916378f-19b0-4da1-a0b8-7d1a88286786"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 16:50:59.797113 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:59.797088 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c916378f-19b0-4da1-a0b8-7d1a88286786-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c916378f-19b0-4da1-a0b8-7d1a88286786" (UID: "c916378f-19b0-4da1-a0b8-7d1a88286786"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 16:50:59.797113 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:59.797098 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c916378f-19b0-4da1-a0b8-7d1a88286786-service-ca" (OuterVolumeSpecName: "service-ca") pod "c916378f-19b0-4da1-a0b8-7d1a88286786" (UID: "c916378f-19b0-4da1-a0b8-7d1a88286786"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 16:50:59.798800 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:59.798778 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c916378f-19b0-4da1-a0b8-7d1a88286786-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c916378f-19b0-4da1-a0b8-7d1a88286786" (UID: "c916378f-19b0-4da1-a0b8-7d1a88286786"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 16:50:59.798890 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:59.798869 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c916378f-19b0-4da1-a0b8-7d1a88286786-kube-api-access-stjqv" (OuterVolumeSpecName: "kube-api-access-stjqv") pod "c916378f-19b0-4da1-a0b8-7d1a88286786" (UID: "c916378f-19b0-4da1-a0b8-7d1a88286786"). InnerVolumeSpecName "kube-api-access-stjqv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 16:50:59.798890 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:59.798883 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c916378f-19b0-4da1-a0b8-7d1a88286786-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c916378f-19b0-4da1-a0b8-7d1a88286786" (UID: "c916378f-19b0-4da1-a0b8-7d1a88286786"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 16:50:59.897660 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:59.897639 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c916378f-19b0-4da1-a0b8-7d1a88286786-console-serving-cert\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:50:59.897660 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:59.897661 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c916378f-19b0-4da1-a0b8-7d1a88286786-service-ca\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:50:59.897783 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:59.897671 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stjqv\" (UniqueName: \"kubernetes.io/projected/c916378f-19b0-4da1-a0b8-7d1a88286786-kube-api-access-stjqv\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:50:59.897783 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:59.897680 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c916378f-19b0-4da1-a0b8-7d1a88286786-console-oauth-config\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:50:59.897783 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:59.897689 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c916378f-19b0-4da1-a0b8-7d1a88286786-oauth-serving-cert\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:50:59.897783 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:50:59.897697 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c916378f-19b0-4da1-a0b8-7d1a88286786-console-config\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:51:00.091442 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:00.091384 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-69848b55b9-5mtcx_c916378f-19b0-4da1-a0b8-7d1a88286786/console/0.log" Apr 16 16:51:00.091442 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:00.091427 2574 generic.go:358] "Generic (PLEG): container finished" podID="c916378f-19b0-4da1-a0b8-7d1a88286786" containerID="2fbac51b0db9788e9fb7db1c57336a7bbfb2bace31fe5f7cbe85042300960b24" exitCode=2 Apr 16 16:51:00.091593 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:00.091464 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69848b55b9-5mtcx" event={"ID":"c916378f-19b0-4da1-a0b8-7d1a88286786","Type":"ContainerDied","Data":"2fbac51b0db9788e9fb7db1c57336a7bbfb2bace31fe5f7cbe85042300960b24"} Apr 16 16:51:00.091593 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:00.091481 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69848b55b9-5mtcx" Apr 16 16:51:00.091593 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:00.091493 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69848b55b9-5mtcx" event={"ID":"c916378f-19b0-4da1-a0b8-7d1a88286786","Type":"ContainerDied","Data":"cca8f917e0db3aa060b3f1adac8857fcd4a57798592bf191f98036bc75240b53"} Apr 16 16:51:00.091593 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:00.091514 2574 scope.go:117] "RemoveContainer" containerID="2fbac51b0db9788e9fb7db1c57336a7bbfb2bace31fe5f7cbe85042300960b24" Apr 16 16:51:00.105518 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:00.105499 2574 scope.go:117] "RemoveContainer" containerID="2fbac51b0db9788e9fb7db1c57336a7bbfb2bace31fe5f7cbe85042300960b24" Apr 16 16:51:00.105891 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:51:00.105870 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fbac51b0db9788e9fb7db1c57336a7bbfb2bace31fe5f7cbe85042300960b24\": container with ID starting with 2fbac51b0db9788e9fb7db1c57336a7bbfb2bace31fe5f7cbe85042300960b24 not found: ID does not exist" containerID="2fbac51b0db9788e9fb7db1c57336a7bbfb2bace31fe5f7cbe85042300960b24" Apr 16 16:51:00.105967 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:00.105903 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fbac51b0db9788e9fb7db1c57336a7bbfb2bace31fe5f7cbe85042300960b24"} err="failed to get container status \"2fbac51b0db9788e9fb7db1c57336a7bbfb2bace31fe5f7cbe85042300960b24\": rpc error: code = NotFound desc = could not find container \"2fbac51b0db9788e9fb7db1c57336a7bbfb2bace31fe5f7cbe85042300960b24\": container with ID starting with 2fbac51b0db9788e9fb7db1c57336a7bbfb2bace31fe5f7cbe85042300960b24 not found: ID does not exist" Apr 16 16:51:00.122178 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:00.122156 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69848b55b9-5mtcx"] Apr 16 16:51:00.126546 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:00.126526 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-69848b55b9-5mtcx"] Apr 16 16:51:00.510975 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:00.510944 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c916378f-19b0-4da1-a0b8-7d1a88286786" path="/var/lib/kubelet/pods/c916378f-19b0-4da1-a0b8-7d1a88286786/volumes" Apr 16 16:51:13.126541 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:13.126512 2574 generic.go:358] "Generic (PLEG): container finished" podID="d403ed6e-d30e-4524-8802-f98b4817628b" containerID="a1057fd812b3b752f80c4813a9c6680965382ad96532e48fdca58781dd718023" exitCode=0 Apr 16 16:51:13.126978 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:13.126585 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-2vh6b" event={"ID":"d403ed6e-d30e-4524-8802-f98b4817628b","Type":"ContainerDied","Data":"a1057fd812b3b752f80c4813a9c6680965382ad96532e48fdca58781dd718023"} Apr 16 16:51:13.126978 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:13.126892 2574 scope.go:117] "RemoveContainer" containerID="a1057fd812b3b752f80c4813a9c6680965382ad96532e48fdca58781dd718023" Apr 16 16:51:13.808462 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:13.808440 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" Apr 16 16:51:13.812484 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:13.812461 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-54997c8455-sxv9m" Apr 16 16:51:14.130755 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:14.130727 2574 generic.go:358] "Generic (PLEG): container finished" podID="3ef9470d-4a29-4aa3-aa5d-21af5d523f6f" containerID="b378f57c214a293dd8c34065e1d6f8069a828f658b8e9f664b5c382f574425b8" exitCode=0 Apr 16 16:51:14.131124 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:14.130803 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5785d4fcdd-tfwxh" event={"ID":"3ef9470d-4a29-4aa3-aa5d-21af5d523f6f","Type":"ContainerDied","Data":"b378f57c214a293dd8c34065e1d6f8069a828f658b8e9f664b5c382f574425b8"} Apr 16 16:51:14.131194 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:14.131144 2574 scope.go:117] "RemoveContainer" containerID="b378f57c214a293dd8c34065e1d6f8069a828f658b8e9f664b5c382f574425b8" Apr 16 16:51:14.132410 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:14.132387 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-69965bb79d-2vh6b" event={"ID":"d403ed6e-d30e-4524-8802-f98b4817628b","Type":"ContainerStarted","Data":"143e4ec1aba57942119bd28505a62e5349583a2abfad3d8ba82466afad65b395"} Apr 16 16:51:15.136984 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:15.136942 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5785d4fcdd-tfwxh" event={"ID":"3ef9470d-4a29-4aa3-aa5d-21af5d523f6f","Type":"ContainerStarted","Data":"0d72c37a77a93dc5092a1eb9cce250528b3392648880c7663c14c71d3ffa725c"} Apr 16 16:51:56.590899 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:56.590819 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b8bf4f7bc-sc9pn"] Apr 16 16:51:56.591341 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:56.591076 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c916378f-19b0-4da1-a0b8-7d1a88286786" containerName="console" Apr 16 16:51:56.591341 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:56.591087 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c916378f-19b0-4da1-a0b8-7d1a88286786" containerName="console" Apr 16 16:51:56.591341 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:56.591137 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="c916378f-19b0-4da1-a0b8-7d1a88286786" containerName="console" Apr 16 16:51:56.593755 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:56.593740 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b8bf4f7bc-sc9pn" Apr 16 16:51:56.608495 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:56.608470 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b8bf4f7bc-sc9pn"] Apr 16 16:51:56.709181 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:56.709153 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-service-ca\") pod \"console-6b8bf4f7bc-sc9pn\" (UID: \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\") " pod="openshift-console/console-6b8bf4f7bc-sc9pn" Apr 16 16:51:56.709315 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:56.709187 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-trusted-ca-bundle\") pod \"console-6b8bf4f7bc-sc9pn\" (UID: \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\") " pod="openshift-console/console-6b8bf4f7bc-sc9pn" Apr 16 16:51:56.709315 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:56.709294 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppqsm\" (UniqueName: \"kubernetes.io/projected/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-kube-api-access-ppqsm\") pod \"console-6b8bf4f7bc-sc9pn\" (UID: \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\") " pod="openshift-console/console-6b8bf4f7bc-sc9pn" Apr 16 16:51:56.709384 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:56.709339 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-console-oauth-config\") pod \"console-6b8bf4f7bc-sc9pn\" (UID: \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\") " pod="openshift-console/console-6b8bf4f7bc-sc9pn" Apr 16 16:51:56.709384 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:56.709364 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-console-config\") pod \"console-6b8bf4f7bc-sc9pn\" (UID: \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\") " pod="openshift-console/console-6b8bf4f7bc-sc9pn" Apr 16 16:51:56.709384 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:56.709381 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-console-serving-cert\") pod \"console-6b8bf4f7bc-sc9pn\" (UID: \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\") " pod="openshift-console/console-6b8bf4f7bc-sc9pn" Apr 16 16:51:56.709473 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:56.709406 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-oauth-serving-cert\") pod \"console-6b8bf4f7bc-sc9pn\" (UID: \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\") " pod="openshift-console/console-6b8bf4f7bc-sc9pn" Apr 16 16:51:56.810405 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:56.810373 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-console-config\") pod \"console-6b8bf4f7bc-sc9pn\" (UID: \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\") " pod="openshift-console/console-6b8bf4f7bc-sc9pn" Apr 16 16:51:56.810405 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:56.810413 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-console-serving-cert\") pod \"console-6b8bf4f7bc-sc9pn\" (UID: \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\") " pod="openshift-console/console-6b8bf4f7bc-sc9pn" Apr 16 16:51:56.810558 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:56.810464 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-oauth-serving-cert\") pod \"console-6b8bf4f7bc-sc9pn\" (UID: \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\") " pod="openshift-console/console-6b8bf4f7bc-sc9pn" Apr 16 16:51:56.810558 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:56.810514 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-service-ca\") pod \"console-6b8bf4f7bc-sc9pn\" (UID: \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\") " pod="openshift-console/console-6b8bf4f7bc-sc9pn" Apr 16 16:51:56.810558 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:56.810542 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-trusted-ca-bundle\") pod \"console-6b8bf4f7bc-sc9pn\" (UID: \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\") " pod="openshift-console/console-6b8bf4f7bc-sc9pn" Apr 16 16:51:56.810710 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:56.810580 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ppqsm\" (UniqueName: \"kubernetes.io/projected/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-kube-api-access-ppqsm\") pod \"console-6b8bf4f7bc-sc9pn\" (UID: \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\") " pod="openshift-console/console-6b8bf4f7bc-sc9pn" Apr 16 16:51:56.810767 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:56.810727 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-console-oauth-config\") pod \"console-6b8bf4f7bc-sc9pn\" (UID: \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\") " pod="openshift-console/console-6b8bf4f7bc-sc9pn" Apr 16 16:51:56.811175 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:56.811151 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-service-ca\") pod \"console-6b8bf4f7bc-sc9pn\" (UID: \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\") " pod="openshift-console/console-6b8bf4f7bc-sc9pn" Apr 16 16:51:56.811175 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:56.811157 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-console-config\") pod \"console-6b8bf4f7bc-sc9pn\" (UID: \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\") " pod="openshift-console/console-6b8bf4f7bc-sc9pn" Apr 16 16:51:56.811328 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:56.811231 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-oauth-serving-cert\") pod \"console-6b8bf4f7bc-sc9pn\" (UID: \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\") " pod="openshift-console/console-6b8bf4f7bc-sc9pn" Apr 16 16:51:56.811533 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:56.811510 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-trusted-ca-bundle\") pod \"console-6b8bf4f7bc-sc9pn\" (UID: \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\") " pod="openshift-console/console-6b8bf4f7bc-sc9pn" Apr 16 16:51:56.812946 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:56.812925 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-console-oauth-config\") pod \"console-6b8bf4f7bc-sc9pn\" (UID: \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\") " pod="openshift-console/console-6b8bf4f7bc-sc9pn" Apr 16 16:51:56.813151 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:56.813132 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-console-serving-cert\") pod \"console-6b8bf4f7bc-sc9pn\" (UID: \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\") " pod="openshift-console/console-6b8bf4f7bc-sc9pn" Apr 16 16:51:56.818327 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:56.818307 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppqsm\" (UniqueName: \"kubernetes.io/projected/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-kube-api-access-ppqsm\") pod \"console-6b8bf4f7bc-sc9pn\" (UID: \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\") " pod="openshift-console/console-6b8bf4f7bc-sc9pn" Apr 16 16:51:56.903391 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:56.903376 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b8bf4f7bc-sc9pn" Apr 16 16:51:57.019100 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:57.019062 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b8bf4f7bc-sc9pn"] Apr 16 16:51:57.024740 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:51:57.024712 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacdb3fea_cf64_42b0_9a16_f2ab5c2d674a.slice/crio-f03bcd0c068e6f8887e15e4962af8bf610da0dfdeef82dd6f3679dbc184be611 WatchSource:0}: Error finding container f03bcd0c068e6f8887e15e4962af8bf610da0dfdeef82dd6f3679dbc184be611: Status 404 returned error can't find the container with id f03bcd0c068e6f8887e15e4962af8bf610da0dfdeef82dd6f3679dbc184be611 Apr 16 16:51:57.257373 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:57.257295 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b8bf4f7bc-sc9pn" event={"ID":"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a","Type":"ContainerStarted","Data":"e0cccbfe063a928025f435525bb20e7e48945ca1824143773231c2d360216a64"} Apr 16 16:51:57.257373 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:57.257329 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b8bf4f7bc-sc9pn" event={"ID":"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a","Type":"ContainerStarted","Data":"f03bcd0c068e6f8887e15e4962af8bf610da0dfdeef82dd6f3679dbc184be611"} Apr 16 16:51:57.274825 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:57.274784 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-6b8bf4f7bc-sc9pn" podStartSLOduration=1.2747709839999999 podStartE2EDuration="1.274770984s" podCreationTimestamp="2026-04-16 16:51:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 16:51:57.273998019 +0000 UTC m=+251.355732234" watchObservedRunningTime="2026-04-16 16:51:57.274770984 +0000 UTC m=+251.356505198" Apr 16 16:51:58.221658 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:58.221623 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs\") pod \"network-metrics-daemon-kfxzj\" (UID: \"d2114257-06f1-4070-ad8d-afce484cc205\") " pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:51:58.223813 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:58.223785 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d2114257-06f1-4070-ad8d-afce484cc205-metrics-certs\") pod \"network-metrics-daemon-kfxzj\" (UID: \"d2114257-06f1-4070-ad8d-afce484cc205\") " pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:51:58.511356 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:58.511285 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-877l2\"" Apr 16 16:51:58.519747 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:58.519733 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfxzj" Apr 16 16:51:58.634273 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:58.634250 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kfxzj"] Apr 16 16:51:58.638686 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:51:58.638570 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2114257_06f1_4070_ad8d_afce484cc205.slice/crio-214e26297c25246b9b17f2e63fc95b6ab389d0af068cbbd94e1ad8d4b49d2b88 WatchSource:0}: Error finding container 214e26297c25246b9b17f2e63fc95b6ab389d0af068cbbd94e1ad8d4b49d2b88: Status 404 returned error can't find the container with id 214e26297c25246b9b17f2e63fc95b6ab389d0af068cbbd94e1ad8d4b49d2b88 Apr 16 16:51:59.263573 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:51:59.263540 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kfxzj" event={"ID":"d2114257-06f1-4070-ad8d-afce484cc205","Type":"ContainerStarted","Data":"214e26297c25246b9b17f2e63fc95b6ab389d0af068cbbd94e1ad8d4b49d2b88"} Apr 16 16:52:00.268123 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:00.268090 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kfxzj" event={"ID":"d2114257-06f1-4070-ad8d-afce484cc205","Type":"ContainerStarted","Data":"49c13e72253441759a8a7f155316182084cb4406ed0c09fa93b97162fe3c7af2"} Apr 16 16:52:00.268123 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:00.268124 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kfxzj" event={"ID":"d2114257-06f1-4070-ad8d-afce484cc205","Type":"ContainerStarted","Data":"25f148929ebc26dbdf960a431fca2fd750b544e36d8580cb11272e2143b98b5c"} Apr 16 16:52:00.284613 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:00.284565 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kfxzj" podStartSLOduration=253.364699159 podStartE2EDuration="4m14.284550454s" podCreationTimestamp="2026-04-16 16:47:46 +0000 UTC" firstStartedPulling="2026-04-16 16:51:58.640959969 +0000 UTC m=+252.722694183" lastFinishedPulling="2026-04-16 16:51:59.560811286 +0000 UTC m=+253.642545478" observedRunningTime="2026-04-16 16:52:00.284035107 +0000 UTC m=+254.365769332" watchObservedRunningTime="2026-04-16 16:52:00.284550454 +0000 UTC m=+254.366284668" Apr 16 16:52:06.904428 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:06.904394 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b8bf4f7bc-sc9pn" Apr 16 16:52:06.904883 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:06.904445 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b8bf4f7bc-sc9pn" Apr 16 16:52:06.908611 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:06.908589 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b8bf4f7bc-sc9pn" Apr 16 16:52:07.292844 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:07.292770 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b8bf4f7bc-sc9pn" Apr 16 16:52:07.340801 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:07.340771 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-576956df6b-6gsqb"] Apr 16 16:52:07.491454 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:07.491421 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d9c66dbb8-vpcjt"] Apr 16 16:52:07.495636 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:07.495619 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d9c66dbb8-vpcjt" Apr 16 16:52:07.507837 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:07.507816 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d9c66dbb8-vpcjt"] Apr 16 16:52:07.582844 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:07.582788 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-console-serving-cert\") pod \"console-6d9c66dbb8-vpcjt\" (UID: \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\") " pod="openshift-console/console-6d9c66dbb8-vpcjt" Apr 16 16:52:07.582844 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:07.582818 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-service-ca\") pod \"console-6d9c66dbb8-vpcjt\" (UID: \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\") " pod="openshift-console/console-6d9c66dbb8-vpcjt" Apr 16 16:52:07.582976 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:07.582874 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-oauth-serving-cert\") pod \"console-6d9c66dbb8-vpcjt\" (UID: \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\") " pod="openshift-console/console-6d9c66dbb8-vpcjt" Apr 16 16:52:07.582976 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:07.582894 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-trusted-ca-bundle\") pod \"console-6d9c66dbb8-vpcjt\" (UID: \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\") " pod="openshift-console/console-6d9c66dbb8-vpcjt" Apr 16 16:52:07.582976 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:07.582944 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrg4b\" (UniqueName: \"kubernetes.io/projected/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-kube-api-access-nrg4b\") pod \"console-6d9c66dbb8-vpcjt\" (UID: \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\") " pod="openshift-console/console-6d9c66dbb8-vpcjt" Apr 16 16:52:07.582976 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:07.582960 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-console-oauth-config\") pod \"console-6d9c66dbb8-vpcjt\" (UID: \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\") " pod="openshift-console/console-6d9c66dbb8-vpcjt" Apr 16 16:52:07.583116 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:07.582980 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-console-config\") pod \"console-6d9c66dbb8-vpcjt\" (UID: \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\") " pod="openshift-console/console-6d9c66dbb8-vpcjt" Apr 16 16:52:07.683839 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:07.683814 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-oauth-serving-cert\") pod \"console-6d9c66dbb8-vpcjt\" (UID: \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\") " pod="openshift-console/console-6d9c66dbb8-vpcjt" Apr 16 16:52:07.683967 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:07.683845 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-trusted-ca-bundle\") pod \"console-6d9c66dbb8-vpcjt\" (UID: \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\") " pod="openshift-console/console-6d9c66dbb8-vpcjt" Apr 16 16:52:07.683967 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:07.683886 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nrg4b\" (UniqueName: \"kubernetes.io/projected/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-kube-api-access-nrg4b\") pod \"console-6d9c66dbb8-vpcjt\" (UID: \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\") " pod="openshift-console/console-6d9c66dbb8-vpcjt" Apr 16 16:52:07.683967 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:07.683912 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-console-oauth-config\") pod \"console-6d9c66dbb8-vpcjt\" (UID: \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\") " pod="openshift-console/console-6d9c66dbb8-vpcjt" Apr 16 16:52:07.683967 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:07.683942 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-console-config\") pod \"console-6d9c66dbb8-vpcjt\" (UID: \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\") " pod="openshift-console/console-6d9c66dbb8-vpcjt" Apr 16 16:52:07.684553 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:07.684524 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-console-serving-cert\") pod \"console-6d9c66dbb8-vpcjt\" (UID: \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\") " pod="openshift-console/console-6d9c66dbb8-vpcjt" Apr 16 16:52:07.684632 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:07.684586 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-service-ca\") pod \"console-6d9c66dbb8-vpcjt\" (UID: \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\") " pod="openshift-console/console-6d9c66dbb8-vpcjt" Apr 16 16:52:07.685675 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:07.685651 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-console-config\") pod \"console-6d9c66dbb8-vpcjt\" (UID: \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\") " pod="openshift-console/console-6d9c66dbb8-vpcjt" Apr 16 16:52:07.687304 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:07.687281 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-service-ca\") pod \"console-6d9c66dbb8-vpcjt\" (UID: \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\") " pod="openshift-console/console-6d9c66dbb8-vpcjt" Apr 16 16:52:07.687439 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:07.687331 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-oauth-serving-cert\") pod \"console-6d9c66dbb8-vpcjt\" (UID: \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\") " pod="openshift-console/console-6d9c66dbb8-vpcjt" Apr 16 16:52:07.690561 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:07.690538 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-console-oauth-config\") pod \"console-6d9c66dbb8-vpcjt\" (UID: \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\") " pod="openshift-console/console-6d9c66dbb8-vpcjt" Apr 16 16:52:07.690642 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:07.690577 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-console-serving-cert\") pod \"console-6d9c66dbb8-vpcjt\" (UID: \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\") " pod="openshift-console/console-6d9c66dbb8-vpcjt" Apr 16 16:52:07.690683 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:07.690663 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-trusted-ca-bundle\") pod \"console-6d9c66dbb8-vpcjt\" (UID: \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\") " pod="openshift-console/console-6d9c66dbb8-vpcjt" Apr 16 16:52:07.692488 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:07.692470 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrg4b\" (UniqueName: \"kubernetes.io/projected/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-kube-api-access-nrg4b\") pod \"console-6d9c66dbb8-vpcjt\" (UID: \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\") " pod="openshift-console/console-6d9c66dbb8-vpcjt" Apr 16 16:52:07.805458 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:07.805438 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d9c66dbb8-vpcjt" Apr 16 16:52:07.918323 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:07.918299 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d9c66dbb8-vpcjt"] Apr 16 16:52:07.920903 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:52:07.920876 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11eb1f23_be6d_4f81_b5ac_752e3c7fc46c.slice/crio-87c70900666d3118c3dc0307ed07d0cccf9bd308c0c856bb7f8ca321796b05cb WatchSource:0}: Error finding container 87c70900666d3118c3dc0307ed07d0cccf9bd308c0c856bb7f8ca321796b05cb: Status 404 returned error can't find the container with id 87c70900666d3118c3dc0307ed07d0cccf9bd308c0c856bb7f8ca321796b05cb Apr 16 16:52:08.292826 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:08.292791 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d9c66dbb8-vpcjt" event={"ID":"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c","Type":"ContainerStarted","Data":"13b3a06d861b14e6efd1e6c9a9167b9e1b097349a0c7bd65e2c19d16e16e6008"} Apr 16 16:52:08.292826 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:08.292824 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d9c66dbb8-vpcjt" event={"ID":"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c","Type":"ContainerStarted","Data":"87c70900666d3118c3dc0307ed07d0cccf9bd308c0c856bb7f8ca321796b05cb"} Apr 16 16:52:08.311836 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:08.311787 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-6d9c66dbb8-vpcjt" podStartSLOduration=1.311775521 podStartE2EDuration="1.311775521s" podCreationTimestamp="2026-04-16 16:52:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 16:52:08.309385467 +0000 UTC m=+262.391119692" watchObservedRunningTime="2026-04-16 16:52:08.311775521 +0000 UTC m=+262.393509731" Apr 16 16:52:17.805728 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:17.805694 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d9c66dbb8-vpcjt" Apr 16 16:52:17.806169 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:17.805766 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d9c66dbb8-vpcjt" Apr 16 16:52:17.810273 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:17.810257 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d9c66dbb8-vpcjt" Apr 16 16:52:18.322849 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:18.322826 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d9c66dbb8-vpcjt" Apr 16 16:52:18.377874 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:18.377844 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b8bf4f7bc-sc9pn"] Apr 16 16:52:32.358912 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:32.358857 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-576956df6b-6gsqb" podUID="fd30943e-c1ef-4b0f-b447-4a73893f1364" containerName="console" containerID="cri-o://91ad670148e2b2ffeabe93da38c45f6478f1afae90be59cb169e00d1bf2c30ec" gracePeriod=15 Apr 16 16:52:32.593387 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:32.593367 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-576956df6b-6gsqb_fd30943e-c1ef-4b0f-b447-4a73893f1364/console/0.log" Apr 16 16:52:32.593485 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:32.593424 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-576956df6b-6gsqb" Apr 16 16:52:32.769121 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:32.769047 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd30943e-c1ef-4b0f-b447-4a73893f1364-console-config\") pod \"fd30943e-c1ef-4b0f-b447-4a73893f1364\" (UID: \"fd30943e-c1ef-4b0f-b447-4a73893f1364\") " Apr 16 16:52:32.769121 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:32.769108 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd30943e-c1ef-4b0f-b447-4a73893f1364-console-oauth-config\") pod \"fd30943e-c1ef-4b0f-b447-4a73893f1364\" (UID: \"fd30943e-c1ef-4b0f-b447-4a73893f1364\") " Apr 16 16:52:32.769389 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:32.769132 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdl64\" (UniqueName: \"kubernetes.io/projected/fd30943e-c1ef-4b0f-b447-4a73893f1364-kube-api-access-tdl64\") pod \"fd30943e-c1ef-4b0f-b447-4a73893f1364\" (UID: \"fd30943e-c1ef-4b0f-b447-4a73893f1364\") " Apr 16 16:52:32.769389 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:32.769163 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd30943e-c1ef-4b0f-b447-4a73893f1364-trusted-ca-bundle\") pod \"fd30943e-c1ef-4b0f-b447-4a73893f1364\" (UID: \"fd30943e-c1ef-4b0f-b447-4a73893f1364\") " Apr 16 16:52:32.769389 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:32.769191 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd30943e-c1ef-4b0f-b447-4a73893f1364-oauth-serving-cert\") pod \"fd30943e-c1ef-4b0f-b447-4a73893f1364\" (UID: \"fd30943e-c1ef-4b0f-b447-4a73893f1364\") " Apr 16 16:52:32.769389 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:32.769230 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd30943e-c1ef-4b0f-b447-4a73893f1364-console-serving-cert\") pod \"fd30943e-c1ef-4b0f-b447-4a73893f1364\" (UID: \"fd30943e-c1ef-4b0f-b447-4a73893f1364\") " Apr 16 16:52:32.769389 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:32.769268 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd30943e-c1ef-4b0f-b447-4a73893f1364-service-ca\") pod \"fd30943e-c1ef-4b0f-b447-4a73893f1364\" (UID: \"fd30943e-c1ef-4b0f-b447-4a73893f1364\") " Apr 16 16:52:32.769613 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:32.769554 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd30943e-c1ef-4b0f-b447-4a73893f1364-console-config" (OuterVolumeSpecName: "console-config") pod "fd30943e-c1ef-4b0f-b447-4a73893f1364" (UID: "fd30943e-c1ef-4b0f-b447-4a73893f1364"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 16:52:32.769710 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:32.769668 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd30943e-c1ef-4b0f-b447-4a73893f1364-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fd30943e-c1ef-4b0f-b447-4a73893f1364" (UID: "fd30943e-c1ef-4b0f-b447-4a73893f1364"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 16:52:32.769710 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:32.769699 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd30943e-c1ef-4b0f-b447-4a73893f1364-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fd30943e-c1ef-4b0f-b447-4a73893f1364" (UID: "fd30943e-c1ef-4b0f-b447-4a73893f1364"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 16:52:32.769710 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:32.769712 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd30943e-c1ef-4b0f-b447-4a73893f1364-service-ca" (OuterVolumeSpecName: "service-ca") pod "fd30943e-c1ef-4b0f-b447-4a73893f1364" (UID: "fd30943e-c1ef-4b0f-b447-4a73893f1364"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 16:52:32.771105 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:32.771082 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd30943e-c1ef-4b0f-b447-4a73893f1364-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fd30943e-c1ef-4b0f-b447-4a73893f1364" (UID: "fd30943e-c1ef-4b0f-b447-4a73893f1364"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 16:52:32.771613 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:32.771592 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd30943e-c1ef-4b0f-b447-4a73893f1364-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fd30943e-c1ef-4b0f-b447-4a73893f1364" (UID: "fd30943e-c1ef-4b0f-b447-4a73893f1364"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 16:52:32.771707 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:32.771684 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd30943e-c1ef-4b0f-b447-4a73893f1364-kube-api-access-tdl64" (OuterVolumeSpecName: "kube-api-access-tdl64") pod "fd30943e-c1ef-4b0f-b447-4a73893f1364" (UID: "fd30943e-c1ef-4b0f-b447-4a73893f1364"). InnerVolumeSpecName "kube-api-access-tdl64". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 16:52:32.870407 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:32.870377 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd30943e-c1ef-4b0f-b447-4a73893f1364-console-config\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:52:32.870407 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:32.870400 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd30943e-c1ef-4b0f-b447-4a73893f1364-console-oauth-config\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:52:32.870407 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:32.870409 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdl64\" (UniqueName: \"kubernetes.io/projected/fd30943e-c1ef-4b0f-b447-4a73893f1364-kube-api-access-tdl64\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:52:32.870554 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:32.870419 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd30943e-c1ef-4b0f-b447-4a73893f1364-trusted-ca-bundle\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:52:32.870554 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:32.870427 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd30943e-c1ef-4b0f-b447-4a73893f1364-oauth-serving-cert\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:52:32.870554 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:32.870436 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd30943e-c1ef-4b0f-b447-4a73893f1364-console-serving-cert\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:52:32.870554 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:32.870445 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd30943e-c1ef-4b0f-b447-4a73893f1364-service-ca\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:52:33.361502 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:33.361475 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-576956df6b-6gsqb_fd30943e-c1ef-4b0f-b447-4a73893f1364/console/0.log" Apr 16 16:52:33.361912 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:33.361513 2574 generic.go:358] "Generic (PLEG): container finished" podID="fd30943e-c1ef-4b0f-b447-4a73893f1364" containerID="91ad670148e2b2ffeabe93da38c45f6478f1afae90be59cb169e00d1bf2c30ec" exitCode=2 Apr 16 16:52:33.361912 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:33.361544 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-576956df6b-6gsqb" event={"ID":"fd30943e-c1ef-4b0f-b447-4a73893f1364","Type":"ContainerDied","Data":"91ad670148e2b2ffeabe93da38c45f6478f1afae90be59cb169e00d1bf2c30ec"} Apr 16 16:52:33.361912 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:33.361586 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-576956df6b-6gsqb" Apr 16 16:52:33.361912 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:33.361598 2574 scope.go:117] "RemoveContainer" containerID="91ad670148e2b2ffeabe93da38c45f6478f1afae90be59cb169e00d1bf2c30ec" Apr 16 16:52:33.361912 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:33.361586 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-576956df6b-6gsqb" event={"ID":"fd30943e-c1ef-4b0f-b447-4a73893f1364","Type":"ContainerDied","Data":"0548e575060a29ff47c71ae6b83f9b534d8d255a941690943e7e6c59b3b89d90"} Apr 16 16:52:33.371408 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:33.370446 2574 scope.go:117] "RemoveContainer" containerID="91ad670148e2b2ffeabe93da38c45f6478f1afae90be59cb169e00d1bf2c30ec" Apr 16 16:52:33.371408 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:52:33.371363 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91ad670148e2b2ffeabe93da38c45f6478f1afae90be59cb169e00d1bf2c30ec\": container with ID starting with 91ad670148e2b2ffeabe93da38c45f6478f1afae90be59cb169e00d1bf2c30ec not found: ID does not exist" containerID="91ad670148e2b2ffeabe93da38c45f6478f1afae90be59cb169e00d1bf2c30ec" Apr 16 16:52:33.371408 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:33.371397 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91ad670148e2b2ffeabe93da38c45f6478f1afae90be59cb169e00d1bf2c30ec"} err="failed to get container status \"91ad670148e2b2ffeabe93da38c45f6478f1afae90be59cb169e00d1bf2c30ec\": rpc error: code = NotFound desc = could not find container \"91ad670148e2b2ffeabe93da38c45f6478f1afae90be59cb169e00d1bf2c30ec\": container with ID starting with 91ad670148e2b2ffeabe93da38c45f6478f1afae90be59cb169e00d1bf2c30ec not found: ID does not exist" Apr 16 16:52:33.385603 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:33.385583 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-576956df6b-6gsqb"] Apr 16 16:52:33.388875 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:33.388858 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-576956df6b-6gsqb"] Apr 16 16:52:34.510817 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:34.510774 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd30943e-c1ef-4b0f-b447-4a73893f1364" path="/var/lib/kubelet/pods/fd30943e-c1ef-4b0f-b447-4a73893f1364/volumes" Apr 16 16:52:43.397268 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:43.397227 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-6b8bf4f7bc-sc9pn" podUID="acdb3fea-cf64-42b0-9a16-f2ab5c2d674a" containerName="console" containerID="cri-o://e0cccbfe063a928025f435525bb20e7e48945ca1824143773231c2d360216a64" gracePeriod=15 Apr 16 16:52:43.623466 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:43.623446 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6b8bf4f7bc-sc9pn_acdb3fea-cf64-42b0-9a16-f2ab5c2d674a/console/0.log" Apr 16 16:52:43.623565 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:43.623507 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b8bf4f7bc-sc9pn" Apr 16 16:52:43.750323 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:43.750249 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppqsm\" (UniqueName: \"kubernetes.io/projected/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-kube-api-access-ppqsm\") pod \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\" (UID: \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\") " Apr 16 16:52:43.750323 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:43.750281 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-oauth-serving-cert\") pod \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\" (UID: \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\") " Apr 16 16:52:43.750323 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:43.750308 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-console-config\") pod \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\" (UID: \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\") " Apr 16 16:52:43.750577 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:43.750325 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-console-oauth-config\") pod \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\" (UID: \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\") " Apr 16 16:52:43.750577 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:43.750347 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-trusted-ca-bundle\") pod \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\" (UID: \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\") " Apr 16 16:52:43.750577 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:43.750458 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-service-ca\") pod \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\" (UID: \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\") " Apr 16 16:52:43.750577 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:43.750526 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-console-serving-cert\") pod \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\" (UID: \"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a\") " Apr 16 16:52:43.750768 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:43.750745 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-console-config" (OuterVolumeSpecName: "console-config") pod "acdb3fea-cf64-42b0-9a16-f2ab5c2d674a" (UID: "acdb3fea-cf64-42b0-9a16-f2ab5c2d674a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 16:52:43.750813 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:43.750773 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "acdb3fea-cf64-42b0-9a16-f2ab5c2d674a" (UID: "acdb3fea-cf64-42b0-9a16-f2ab5c2d674a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 16:52:43.750813 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:43.750779 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "acdb3fea-cf64-42b0-9a16-f2ab5c2d674a" (UID: "acdb3fea-cf64-42b0-9a16-f2ab5c2d674a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 16:52:43.750986 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:43.750964 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-service-ca" (OuterVolumeSpecName: "service-ca") pod "acdb3fea-cf64-42b0-9a16-f2ab5c2d674a" (UID: "acdb3fea-cf64-42b0-9a16-f2ab5c2d674a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 16:52:43.752593 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:43.752569 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "acdb3fea-cf64-42b0-9a16-f2ab5c2d674a" (UID: "acdb3fea-cf64-42b0-9a16-f2ab5c2d674a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 16:52:43.752877 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:43.752849 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "acdb3fea-cf64-42b0-9a16-f2ab5c2d674a" (UID: "acdb3fea-cf64-42b0-9a16-f2ab5c2d674a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 16:52:43.752877 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:43.752851 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-kube-api-access-ppqsm" (OuterVolumeSpecName: "kube-api-access-ppqsm") pod "acdb3fea-cf64-42b0-9a16-f2ab5c2d674a" (UID: "acdb3fea-cf64-42b0-9a16-f2ab5c2d674a"). InnerVolumeSpecName "kube-api-access-ppqsm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 16:52:43.851682 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:43.851647 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppqsm\" (UniqueName: \"kubernetes.io/projected/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-kube-api-access-ppqsm\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:52:43.851682 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:43.851680 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-oauth-serving-cert\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:52:43.851682 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:43.851690 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-console-config\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:52:43.851834 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:43.851700 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-console-oauth-config\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:52:43.851834 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:43.851709 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-trusted-ca-bundle\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:52:43.851834 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:43.851717 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-service-ca\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:52:43.851834 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:43.851725 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a-console-serving-cert\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:52:44.394307 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:44.394281 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6b8bf4f7bc-sc9pn_acdb3fea-cf64-42b0-9a16-f2ab5c2d674a/console/0.log" Apr 16 16:52:44.394505 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:44.394323 2574 generic.go:358] "Generic (PLEG): container finished" podID="acdb3fea-cf64-42b0-9a16-f2ab5c2d674a" containerID="e0cccbfe063a928025f435525bb20e7e48945ca1824143773231c2d360216a64" exitCode=2 Apr 16 16:52:44.394505 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:44.394376 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b8bf4f7bc-sc9pn" event={"ID":"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a","Type":"ContainerDied","Data":"e0cccbfe063a928025f435525bb20e7e48945ca1824143773231c2d360216a64"} Apr 16 16:52:44.394505 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:44.394407 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b8bf4f7bc-sc9pn" event={"ID":"acdb3fea-cf64-42b0-9a16-f2ab5c2d674a","Type":"ContainerDied","Data":"f03bcd0c068e6f8887e15e4962af8bf610da0dfdeef82dd6f3679dbc184be611"} Apr 16 16:52:44.394505 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:44.394425 2574 scope.go:117] "RemoveContainer" containerID="e0cccbfe063a928025f435525bb20e7e48945ca1824143773231c2d360216a64" Apr 16 16:52:44.394505 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:44.394424 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b8bf4f7bc-sc9pn" Apr 16 16:52:44.402527 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:44.402135 2574 scope.go:117] "RemoveContainer" containerID="e0cccbfe063a928025f435525bb20e7e48945ca1824143773231c2d360216a64" Apr 16 16:52:44.402803 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:52:44.402727 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0cccbfe063a928025f435525bb20e7e48945ca1824143773231c2d360216a64\": container with ID starting with e0cccbfe063a928025f435525bb20e7e48945ca1824143773231c2d360216a64 not found: ID does not exist" containerID="e0cccbfe063a928025f435525bb20e7e48945ca1824143773231c2d360216a64" Apr 16 16:52:44.402803 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:44.402759 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0cccbfe063a928025f435525bb20e7e48945ca1824143773231c2d360216a64"} err="failed to get container status \"e0cccbfe063a928025f435525bb20e7e48945ca1824143773231c2d360216a64\": rpc error: code = NotFound desc = could not find container \"e0cccbfe063a928025f435525bb20e7e48945ca1824143773231c2d360216a64\": container with ID starting with e0cccbfe063a928025f435525bb20e7e48945ca1824143773231c2d360216a64 not found: ID does not exist" Apr 16 16:52:44.422096 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:44.422076 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b8bf4f7bc-sc9pn"] Apr 16 16:52:44.424833 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:44.424816 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b8bf4f7bc-sc9pn"] Apr 16 16:52:44.516380 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:44.516349 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acdb3fea-cf64-42b0-9a16-f2ab5c2d674a" path="/var/lib/kubelet/pods/acdb3fea-cf64-42b0-9a16-f2ab5c2d674a/volumes" Apr 16 16:52:46.385692 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:46.385661 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/1.log" Apr 16 16:52:46.386178 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:46.386161 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/1.log" Apr 16 16:52:46.395328 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:46.395307 2574 kubelet.go:1628] "Image garbage collection succeeded" Apr 16 16:52:50.810220 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:50.810173 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-mtpht"] Apr 16 16:52:50.812608 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:50.810468 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd30943e-c1ef-4b0f-b447-4a73893f1364" containerName="console" Apr 16 16:52:50.812608 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:50.810481 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd30943e-c1ef-4b0f-b447-4a73893f1364" containerName="console" Apr 16 16:52:50.812608 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:50.810495 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acdb3fea-cf64-42b0-9a16-f2ab5c2d674a" containerName="console" Apr 16 16:52:50.812608 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:50.810500 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="acdb3fea-cf64-42b0-9a16-f2ab5c2d674a" containerName="console" Apr 16 16:52:50.812608 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:50.810548 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="acdb3fea-cf64-42b0-9a16-f2ab5c2d674a" containerName="console" Apr 16 16:52:50.812608 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:50.810559 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="fd30943e-c1ef-4b0f-b447-4a73893f1364" containerName="console" Apr 16 16:52:50.813397 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:50.813383 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mtpht" Apr 16 16:52:50.816092 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:50.816056 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Apr 16 16:52:50.832169 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:50.832150 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mtpht"] Apr 16 16:52:50.998961 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:50.998920 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/43838dea-81df-4027-b3a7-1f13ebc11e45-original-pull-secret\") pod \"global-pull-secret-syncer-mtpht\" (UID: \"43838dea-81df-4027-b3a7-1f13ebc11e45\") " pod="kube-system/global-pull-secret-syncer-mtpht" Apr 16 16:52:50.999095 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:50.998973 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/43838dea-81df-4027-b3a7-1f13ebc11e45-dbus\") pod \"global-pull-secret-syncer-mtpht\" (UID: \"43838dea-81df-4027-b3a7-1f13ebc11e45\") " pod="kube-system/global-pull-secret-syncer-mtpht" Apr 16 16:52:50.999095 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:50.999023 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/43838dea-81df-4027-b3a7-1f13ebc11e45-kubelet-config\") pod \"global-pull-secret-syncer-mtpht\" (UID: \"43838dea-81df-4027-b3a7-1f13ebc11e45\") " pod="kube-system/global-pull-secret-syncer-mtpht" Apr 16 16:52:51.100241 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:51.100161 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/43838dea-81df-4027-b3a7-1f13ebc11e45-original-pull-secret\") pod \"global-pull-secret-syncer-mtpht\" (UID: \"43838dea-81df-4027-b3a7-1f13ebc11e45\") " pod="kube-system/global-pull-secret-syncer-mtpht" Apr 16 16:52:51.100241 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:51.100226 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/43838dea-81df-4027-b3a7-1f13ebc11e45-dbus\") pod \"global-pull-secret-syncer-mtpht\" (UID: \"43838dea-81df-4027-b3a7-1f13ebc11e45\") " pod="kube-system/global-pull-secret-syncer-mtpht" Apr 16 16:52:51.100378 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:51.100256 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/43838dea-81df-4027-b3a7-1f13ebc11e45-kubelet-config\") pod \"global-pull-secret-syncer-mtpht\" (UID: \"43838dea-81df-4027-b3a7-1f13ebc11e45\") " pod="kube-system/global-pull-secret-syncer-mtpht" Apr 16 16:52:51.100378 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:51.100355 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/43838dea-81df-4027-b3a7-1f13ebc11e45-kubelet-config\") pod \"global-pull-secret-syncer-mtpht\" (UID: \"43838dea-81df-4027-b3a7-1f13ebc11e45\") " pod="kube-system/global-pull-secret-syncer-mtpht" Apr 16 16:52:51.100453 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:51.100400 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/43838dea-81df-4027-b3a7-1f13ebc11e45-dbus\") pod \"global-pull-secret-syncer-mtpht\" (UID: \"43838dea-81df-4027-b3a7-1f13ebc11e45\") " pod="kube-system/global-pull-secret-syncer-mtpht" Apr 16 16:52:51.102441 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:51.102413 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/43838dea-81df-4027-b3a7-1f13ebc11e45-original-pull-secret\") pod \"global-pull-secret-syncer-mtpht\" (UID: \"43838dea-81df-4027-b3a7-1f13ebc11e45\") " pod="kube-system/global-pull-secret-syncer-mtpht" Apr 16 16:52:51.122027 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:51.122010 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mtpht" Apr 16 16:52:51.242547 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:51.242495 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mtpht"] Apr 16 16:52:51.245156 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:52:51.245118 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43838dea_81df_4027_b3a7_1f13ebc11e45.slice/crio-7753ba000ca0c6aa2f713746247f6a92e881de271b4b952d6d945d8186a55caf WatchSource:0}: Error finding container 7753ba000ca0c6aa2f713746247f6a92e881de271b4b952d6d945d8186a55caf: Status 404 returned error can't find the container with id 7753ba000ca0c6aa2f713746247f6a92e881de271b4b952d6d945d8186a55caf Apr 16 16:52:51.247051 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:51.247033 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 16 16:52:51.414108 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:51.414078 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mtpht" event={"ID":"43838dea-81df-4027-b3a7-1f13ebc11e45","Type":"ContainerStarted","Data":"7753ba000ca0c6aa2f713746247f6a92e881de271b4b952d6d945d8186a55caf"} Apr 16 16:52:55.427035 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:55.426997 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mtpht" event={"ID":"43838dea-81df-4027-b3a7-1f13ebc11e45","Type":"ContainerStarted","Data":"1e72f5077e8d8b6b4f4042471dac144c4ffa4ebd4ca640f818354e2194d32216"} Apr 16 16:52:55.441649 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:52:55.441599 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mtpht" podStartSLOduration=1.888335417 podStartE2EDuration="5.441585887s" podCreationTimestamp="2026-04-16 16:52:50 +0000 UTC" firstStartedPulling="2026-04-16 16:52:51.247250466 +0000 UTC m=+305.328984659" lastFinishedPulling="2026-04-16 16:52:54.800500917 +0000 UTC m=+308.882235129" observedRunningTime="2026-04-16 16:52:55.440863439 +0000 UTC m=+309.522597653" watchObservedRunningTime="2026-04-16 16:52:55.441585887 +0000 UTC m=+309.523320100" Apr 16 16:53:12.070680 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:12.070647 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87"] Apr 16 16:53:12.073925 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:12.073910 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87" Apr 16 16:53:12.076579 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:12.076556 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Apr 16 16:53:12.077563 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:12.077546 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Apr 16 16:53:12.077652 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:12.077577 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-78hs9\"" Apr 16 16:53:12.081394 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:12.080948 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87"] Apr 16 16:53:12.147615 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:12.147592 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdf1e521-5733-4f23-8161-3693145210e0-bundle\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87\" (UID: \"fdf1e521-5733-4f23-8161-3693145210e0\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87" Apr 16 16:53:12.147734 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:12.147662 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bjl2\" (UniqueName: \"kubernetes.io/projected/fdf1e521-5733-4f23-8161-3693145210e0-kube-api-access-4bjl2\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87\" (UID: \"fdf1e521-5733-4f23-8161-3693145210e0\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87" Apr 16 16:53:12.147734 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:12.147695 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdf1e521-5733-4f23-8161-3693145210e0-util\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87\" (UID: \"fdf1e521-5733-4f23-8161-3693145210e0\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87" Apr 16 16:53:12.248325 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:12.248290 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4bjl2\" (UniqueName: \"kubernetes.io/projected/fdf1e521-5733-4f23-8161-3693145210e0-kube-api-access-4bjl2\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87\" (UID: \"fdf1e521-5733-4f23-8161-3693145210e0\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87" Apr 16 16:53:12.248449 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:12.248336 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdf1e521-5733-4f23-8161-3693145210e0-util\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87\" (UID: \"fdf1e521-5733-4f23-8161-3693145210e0\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87" Apr 16 16:53:12.248449 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:12.248356 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdf1e521-5733-4f23-8161-3693145210e0-bundle\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87\" (UID: \"fdf1e521-5733-4f23-8161-3693145210e0\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87" Apr 16 16:53:12.248639 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:12.248625 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdf1e521-5733-4f23-8161-3693145210e0-bundle\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87\" (UID: \"fdf1e521-5733-4f23-8161-3693145210e0\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87" Apr 16 16:53:12.248692 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:12.248652 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdf1e521-5733-4f23-8161-3693145210e0-util\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87\" (UID: \"fdf1e521-5733-4f23-8161-3693145210e0\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87" Apr 16 16:53:12.256619 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:12.256600 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bjl2\" (UniqueName: \"kubernetes.io/projected/fdf1e521-5733-4f23-8161-3693145210e0-kube-api-access-4bjl2\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87\" (UID: \"fdf1e521-5733-4f23-8161-3693145210e0\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87" Apr 16 16:53:12.384341 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:12.384321 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87" Apr 16 16:53:12.496133 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:12.496094 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87"] Apr 16 16:53:12.500053 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:53:12.500028 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdf1e521_5733_4f23_8161_3693145210e0.slice/crio-31abc42c5e3fb3965436237f32c0c4973fbb6b9408b9ef19183971f5028b1f53 WatchSource:0}: Error finding container 31abc42c5e3fb3965436237f32c0c4973fbb6b9408b9ef19183971f5028b1f53: Status 404 returned error can't find the container with id 31abc42c5e3fb3965436237f32c0c4973fbb6b9408b9ef19183971f5028b1f53 Apr 16 16:53:13.476890 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:13.476847 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87" event={"ID":"fdf1e521-5733-4f23-8161-3693145210e0","Type":"ContainerStarted","Data":"31abc42c5e3fb3965436237f32c0c4973fbb6b9408b9ef19183971f5028b1f53"} Apr 16 16:53:17.490897 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:17.490863 2574 generic.go:358] "Generic (PLEG): container finished" podID="fdf1e521-5733-4f23-8161-3693145210e0" containerID="b3a404094f0efb711b6d1c82e2f8a4fb50ed87ec214eb5c1f17299c56ddc9f15" exitCode=0 Apr 16 16:53:17.491266 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:17.490910 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87" event={"ID":"fdf1e521-5733-4f23-8161-3693145210e0","Type":"ContainerDied","Data":"b3a404094f0efb711b6d1c82e2f8a4fb50ed87ec214eb5c1f17299c56ddc9f15"} Apr 16 16:53:20.501909 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:20.501876 2574 generic.go:358] "Generic (PLEG): container finished" podID="fdf1e521-5733-4f23-8161-3693145210e0" containerID="e4b15aeb68baec21b5618e62c01d48ca256a6d7d0b0b05af0fb509c7309b72c7" exitCode=0 Apr 16 16:53:20.502275 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:20.501950 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87" event={"ID":"fdf1e521-5733-4f23-8161-3693145210e0","Type":"ContainerDied","Data":"e4b15aeb68baec21b5618e62c01d48ca256a6d7d0b0b05af0fb509c7309b72c7"} Apr 16 16:53:26.520137 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:26.520105 2574 generic.go:358] "Generic (PLEG): container finished" podID="fdf1e521-5733-4f23-8161-3693145210e0" containerID="5e56e3a7696954074ac4f53457e7b1db35c080007aad1cc2fe15f7e60f8c4813" exitCode=0 Apr 16 16:53:26.520559 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:26.520185 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87" event={"ID":"fdf1e521-5733-4f23-8161-3693145210e0","Type":"ContainerDied","Data":"5e56e3a7696954074ac4f53457e7b1db35c080007aad1cc2fe15f7e60f8c4813"} Apr 16 16:53:27.642459 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:27.642437 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87" Apr 16 16:53:27.672529 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:27.672496 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdf1e521-5733-4f23-8161-3693145210e0-util\") pod \"fdf1e521-5733-4f23-8161-3693145210e0\" (UID: \"fdf1e521-5733-4f23-8161-3693145210e0\") " Apr 16 16:53:27.672671 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:27.672551 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bjl2\" (UniqueName: \"kubernetes.io/projected/fdf1e521-5733-4f23-8161-3693145210e0-kube-api-access-4bjl2\") pod \"fdf1e521-5733-4f23-8161-3693145210e0\" (UID: \"fdf1e521-5733-4f23-8161-3693145210e0\") " Apr 16 16:53:27.672740 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:27.672676 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdf1e521-5733-4f23-8161-3693145210e0-bundle\") pod \"fdf1e521-5733-4f23-8161-3693145210e0\" (UID: \"fdf1e521-5733-4f23-8161-3693145210e0\") " Apr 16 16:53:27.673243 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:27.673195 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdf1e521-5733-4f23-8161-3693145210e0-bundle" (OuterVolumeSpecName: "bundle") pod "fdf1e521-5733-4f23-8161-3693145210e0" (UID: "fdf1e521-5733-4f23-8161-3693145210e0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 16 16:53:27.674582 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:27.674561 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdf1e521-5733-4f23-8161-3693145210e0-kube-api-access-4bjl2" (OuterVolumeSpecName: "kube-api-access-4bjl2") pod "fdf1e521-5733-4f23-8161-3693145210e0" (UID: "fdf1e521-5733-4f23-8161-3693145210e0"). InnerVolumeSpecName "kube-api-access-4bjl2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 16:53:27.676490 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:27.676463 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdf1e521-5733-4f23-8161-3693145210e0-util" (OuterVolumeSpecName: "util") pod "fdf1e521-5733-4f23-8161-3693145210e0" (UID: "fdf1e521-5733-4f23-8161-3693145210e0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 16 16:53:27.774187 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:27.774119 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdf1e521-5733-4f23-8161-3693145210e0-util\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:53:27.774187 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:27.774147 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bjl2\" (UniqueName: \"kubernetes.io/projected/fdf1e521-5733-4f23-8161-3693145210e0-kube-api-access-4bjl2\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:53:27.774187 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:27.774157 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdf1e521-5733-4f23-8161-3693145210e0-bundle\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:53:28.526288 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:28.526261 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87" event={"ID":"fdf1e521-5733-4f23-8161-3693145210e0","Type":"ContainerDied","Data":"31abc42c5e3fb3965436237f32c0c4973fbb6b9408b9ef19183971f5028b1f53"} Apr 16 16:53:28.526288 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:28.526290 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="31abc42c5e3fb3965436237f32c0c4973fbb6b9408b9ef19183971f5028b1f53" Apr 16 16:53:28.526423 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:28.526330 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cj4b87" Apr 16 16:53:33.834368 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:33.834335 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-5kcmw"] Apr 16 16:53:33.834715 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:33.834608 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdf1e521-5733-4f23-8161-3693145210e0" containerName="pull" Apr 16 16:53:33.834715 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:33.834620 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdf1e521-5733-4f23-8161-3693145210e0" containerName="pull" Apr 16 16:53:33.834715 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:33.834636 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdf1e521-5733-4f23-8161-3693145210e0" containerName="extract" Apr 16 16:53:33.834715 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:33.834643 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdf1e521-5733-4f23-8161-3693145210e0" containerName="extract" Apr 16 16:53:33.834715 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:33.834660 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdf1e521-5733-4f23-8161-3693145210e0" containerName="util" Apr 16 16:53:33.834715 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:33.834665 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdf1e521-5733-4f23-8161-3693145210e0" containerName="util" Apr 16 16:53:33.834715 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:33.834710 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="fdf1e521-5733-4f23-8161-3693145210e0" containerName="extract" Apr 16 16:53:33.837751 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:33.837736 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-5kcmw" Apr 16 16:53:33.840527 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:33.840506 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\"" Apr 16 16:53:33.840662 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:33.840567 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-j44m8\"" Apr 16 16:53:33.840662 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:33.840583 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\"" Apr 16 16:53:33.840820 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:33.840805 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\"" Apr 16 16:53:33.846575 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:33.846556 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-5kcmw"] Apr 16 16:53:33.916927 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:33.916900 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/529a90da-b355-475f-b817-9c29cb7cea67-certificates\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-5kcmw\" (UID: \"529a90da-b355-475f-b817-9c29cb7cea67\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-5kcmw" Apr 16 16:53:33.917056 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:33.916941 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk7qn\" (UniqueName: \"kubernetes.io/projected/529a90da-b355-475f-b817-9c29cb7cea67-kube-api-access-dk7qn\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-5kcmw\" (UID: \"529a90da-b355-475f-b817-9c29cb7cea67\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-5kcmw" Apr 16 16:53:34.017601 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:34.017574 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dk7qn\" (UniqueName: \"kubernetes.io/projected/529a90da-b355-475f-b817-9c29cb7cea67-kube-api-access-dk7qn\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-5kcmw\" (UID: \"529a90da-b355-475f-b817-9c29cb7cea67\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-5kcmw" Apr 16 16:53:34.017743 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:34.017633 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/529a90da-b355-475f-b817-9c29cb7cea67-certificates\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-5kcmw\" (UID: \"529a90da-b355-475f-b817-9c29cb7cea67\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-5kcmw" Apr 16 16:53:34.019828 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:34.019797 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/529a90da-b355-475f-b817-9c29cb7cea67-certificates\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-5kcmw\" (UID: \"529a90da-b355-475f-b817-9c29cb7cea67\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-5kcmw" Apr 16 16:53:34.036839 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:34.036814 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk7qn\" (UniqueName: \"kubernetes.io/projected/529a90da-b355-475f-b817-9c29cb7cea67-kube-api-access-dk7qn\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-5kcmw\" (UID: \"529a90da-b355-475f-b817-9c29cb7cea67\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-5kcmw" Apr 16 16:53:34.147493 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:34.147457 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-5kcmw" Apr 16 16:53:34.264805 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:34.264785 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-5kcmw"] Apr 16 16:53:34.267137 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:53:34.267108 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod529a90da_b355_475f_b817_9c29cb7cea67.slice/crio-b783593907b79e5ae0f7e5a6ba23a7627df9c9d5e024392be8e247672684b6ab WatchSource:0}: Error finding container b783593907b79e5ae0f7e5a6ba23a7627df9c9d5e024392be8e247672684b6ab: Status 404 returned error can't find the container with id b783593907b79e5ae0f7e5a6ba23a7627df9c9d5e024392be8e247672684b6ab Apr 16 16:53:34.543367 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:34.543291 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-5kcmw" event={"ID":"529a90da-b355-475f-b817-9c29cb7cea67","Type":"ContainerStarted","Data":"b783593907b79e5ae0f7e5a6ba23a7627df9c9d5e024392be8e247672684b6ab"} Apr 16 16:53:37.553882 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:37.553848 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-5kcmw" event={"ID":"529a90da-b355-475f-b817-9c29cb7cea67","Type":"ContainerStarted","Data":"6c45e660553b98a34a22d68f00e5f4a0f5e2c24823977125062c57614b5a8e51"} Apr 16 16:53:37.554253 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:37.553969 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-5kcmw" Apr 16 16:53:37.589602 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:37.589548 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-5kcmw" podStartSLOduration=1.76179482 podStartE2EDuration="4.589533037s" podCreationTimestamp="2026-04-16 16:53:33 +0000 UTC" firstStartedPulling="2026-04-16 16:53:34.268970588 +0000 UTC m=+348.350704795" lastFinishedPulling="2026-04-16 16:53:37.096708817 +0000 UTC m=+351.178443012" observedRunningTime="2026-04-16 16:53:37.586474339 +0000 UTC m=+351.668208555" watchObservedRunningTime="2026-04-16 16:53:37.589533037 +0000 UTC m=+351.671267252" Apr 16 16:53:37.762279 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:37.762241 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-ffbb595cb-srvs2"] Apr 16 16:53:37.765542 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:37.765501 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-ffbb595cb-srvs2" Apr 16 16:53:37.768760 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:37.768739 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\"" Apr 16 16:53:37.769287 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:37.769269 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\"" Apr 16 16:53:37.771227 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:37.771181 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-fj5x5\"" Apr 16 16:53:37.784465 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:37.784438 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-ffbb595cb-srvs2"] Apr 16 16:53:37.848950 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:37.848873 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4clh\" (UniqueName: \"kubernetes.io/projected/d69ca095-0b91-49b2-9620-eb2ef3b4a44a-kube-api-access-g4clh\") pod \"keda-operator-ffbb595cb-srvs2\" (UID: \"d69ca095-0b91-49b2-9620-eb2ef3b4a44a\") " pod="openshift-keda/keda-operator-ffbb595cb-srvs2" Apr 16 16:53:37.849100 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:37.849030 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/d69ca095-0b91-49b2-9620-eb2ef3b4a44a-cabundle0\") pod \"keda-operator-ffbb595cb-srvs2\" (UID: \"d69ca095-0b91-49b2-9620-eb2ef3b4a44a\") " pod="openshift-keda/keda-operator-ffbb595cb-srvs2" Apr 16 16:53:37.849100 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:37.849072 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d69ca095-0b91-49b2-9620-eb2ef3b4a44a-certificates\") pod \"keda-operator-ffbb595cb-srvs2\" (UID: \"d69ca095-0b91-49b2-9620-eb2ef3b4a44a\") " pod="openshift-keda/keda-operator-ffbb595cb-srvs2" Apr 16 16:53:37.949763 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:37.949731 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/d69ca095-0b91-49b2-9620-eb2ef3b4a44a-cabundle0\") pod \"keda-operator-ffbb595cb-srvs2\" (UID: \"d69ca095-0b91-49b2-9620-eb2ef3b4a44a\") " pod="openshift-keda/keda-operator-ffbb595cb-srvs2" Apr 16 16:53:37.949763 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:37.949770 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d69ca095-0b91-49b2-9620-eb2ef3b4a44a-certificates\") pod \"keda-operator-ffbb595cb-srvs2\" (UID: \"d69ca095-0b91-49b2-9620-eb2ef3b4a44a\") " pod="openshift-keda/keda-operator-ffbb595cb-srvs2" Apr 16 16:53:37.949995 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:37.949792 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g4clh\" (UniqueName: \"kubernetes.io/projected/d69ca095-0b91-49b2-9620-eb2ef3b4a44a-kube-api-access-g4clh\") pod \"keda-operator-ffbb595cb-srvs2\" (UID: \"d69ca095-0b91-49b2-9620-eb2ef3b4a44a\") " pod="openshift-keda/keda-operator-ffbb595cb-srvs2" Apr 16 16:53:37.949995 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:37.949908 2574 projected.go:264] Couldn't get secret openshift-keda/keda-operator-certs: secret "keda-operator-certs" not found Apr 16 16:53:37.949995 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:37.949926 2574 secret.go:281] references non-existent secret key: ca.crt Apr 16 16:53:37.949995 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:37.949934 2574 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt Apr 16 16:53:37.949995 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:37.949948 2574 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-srvs2: [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt] Apr 16 16:53:37.950227 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:37.950004 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d69ca095-0b91-49b2-9620-eb2ef3b4a44a-certificates podName:d69ca095-0b91-49b2-9620-eb2ef3b4a44a nodeName:}" failed. No retries permitted until 2026-04-16 16:53:38.449986264 +0000 UTC m=+352.531720458 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/d69ca095-0b91-49b2-9620-eb2ef3b4a44a-certificates") pod "keda-operator-ffbb595cb-srvs2" (UID: "d69ca095-0b91-49b2-9620-eb2ef3b4a44a") : [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt] Apr 16 16:53:37.950542 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:37.950519 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/d69ca095-0b91-49b2-9620-eb2ef3b4a44a-cabundle0\") pod \"keda-operator-ffbb595cb-srvs2\" (UID: \"d69ca095-0b91-49b2-9620-eb2ef3b4a44a\") " pod="openshift-keda/keda-operator-ffbb595cb-srvs2" Apr 16 16:53:37.971465 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:37.971440 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4clh\" (UniqueName: \"kubernetes.io/projected/d69ca095-0b91-49b2-9620-eb2ef3b4a44a-kube-api-access-g4clh\") pod \"keda-operator-ffbb595cb-srvs2\" (UID: \"d69ca095-0b91-49b2-9620-eb2ef3b4a44a\") " pod="openshift-keda/keda-operator-ffbb595cb-srvs2" Apr 16 16:53:38.083925 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:38.083887 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-7c9f485588-7mjrr"] Apr 16 16:53:38.087337 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:38.087321 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7mjrr" Apr 16 16:53:38.090629 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:38.090606 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\"" Apr 16 16:53:38.106146 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:38.106119 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-7c9f485588-7mjrr"] Apr 16 16:53:38.152415 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:38.152389 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/eaa475f1-8e01-48d3-a3ce-010e15939f85-certificates\") pod \"keda-metrics-apiserver-7c9f485588-7mjrr\" (UID: \"eaa475f1-8e01-48d3-a3ce-010e15939f85\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7mjrr" Apr 16 16:53:38.152561 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:38.152456 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lt5cr\" (UniqueName: \"kubernetes.io/projected/eaa475f1-8e01-48d3-a3ce-010e15939f85-kube-api-access-lt5cr\") pod \"keda-metrics-apiserver-7c9f485588-7mjrr\" (UID: \"eaa475f1-8e01-48d3-a3ce-010e15939f85\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7mjrr" Apr 16 16:53:38.152561 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:38.152493 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/eaa475f1-8e01-48d3-a3ce-010e15939f85-temp-vol\") pod \"keda-metrics-apiserver-7c9f485588-7mjrr\" (UID: \"eaa475f1-8e01-48d3-a3ce-010e15939f85\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7mjrr" Apr 16 16:53:38.252996 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:38.252952 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/eaa475f1-8e01-48d3-a3ce-010e15939f85-certificates\") pod \"keda-metrics-apiserver-7c9f485588-7mjrr\" (UID: \"eaa475f1-8e01-48d3-a3ce-010e15939f85\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7mjrr" Apr 16 16:53:38.253161 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:38.253102 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lt5cr\" (UniqueName: \"kubernetes.io/projected/eaa475f1-8e01-48d3-a3ce-010e15939f85-kube-api-access-lt5cr\") pod \"keda-metrics-apiserver-7c9f485588-7mjrr\" (UID: \"eaa475f1-8e01-48d3-a3ce-010e15939f85\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7mjrr" Apr 16 16:53:38.253161 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:38.253123 2574 secret.go:281] references non-existent secret key: tls.crt Apr 16 16:53:38.253161 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:38.253141 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/eaa475f1-8e01-48d3-a3ce-010e15939f85-temp-vol\") pod \"keda-metrics-apiserver-7c9f485588-7mjrr\" (UID: \"eaa475f1-8e01-48d3-a3ce-010e15939f85\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7mjrr" Apr 16 16:53:38.253161 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:38.253148 2574 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt Apr 16 16:53:38.253375 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:38.253171 2574 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-7c9f485588-7mjrr: references non-existent secret key: tls.crt Apr 16 16:53:38.253375 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:38.253249 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eaa475f1-8e01-48d3-a3ce-010e15939f85-certificates podName:eaa475f1-8e01-48d3-a3ce-010e15939f85 nodeName:}" failed. No retries permitted until 2026-04-16 16:53:38.75322684 +0000 UTC m=+352.834961036 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/eaa475f1-8e01-48d3-a3ce-010e15939f85-certificates") pod "keda-metrics-apiserver-7c9f485588-7mjrr" (UID: "eaa475f1-8e01-48d3-a3ce-010e15939f85") : references non-existent secret key: tls.crt Apr 16 16:53:38.253607 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:38.253588 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/eaa475f1-8e01-48d3-a3ce-010e15939f85-temp-vol\") pod \"keda-metrics-apiserver-7c9f485588-7mjrr\" (UID: \"eaa475f1-8e01-48d3-a3ce-010e15939f85\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7mjrr" Apr 16 16:53:38.277773 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:38.277745 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lt5cr\" (UniqueName: \"kubernetes.io/projected/eaa475f1-8e01-48d3-a3ce-010e15939f85-kube-api-access-lt5cr\") pod \"keda-metrics-apiserver-7c9f485588-7mjrr\" (UID: \"eaa475f1-8e01-48d3-a3ce-010e15939f85\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7mjrr" Apr 16 16:53:38.455597 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:38.455509 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d69ca095-0b91-49b2-9620-eb2ef3b4a44a-certificates\") pod \"keda-operator-ffbb595cb-srvs2\" (UID: \"d69ca095-0b91-49b2-9620-eb2ef3b4a44a\") " pod="openshift-keda/keda-operator-ffbb595cb-srvs2" Apr 16 16:53:38.455724 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:38.455668 2574 secret.go:281] references non-existent secret key: ca.crt Apr 16 16:53:38.455724 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:38.455691 2574 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt Apr 16 16:53:38.455724 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:38.455705 2574 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-srvs2: references non-existent secret key: ca.crt Apr 16 16:53:38.455814 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:38.455767 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d69ca095-0b91-49b2-9620-eb2ef3b4a44a-certificates podName:d69ca095-0b91-49b2-9620-eb2ef3b4a44a nodeName:}" failed. No retries permitted until 2026-04-16 16:53:39.455748446 +0000 UTC m=+353.537482644 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/d69ca095-0b91-49b2-9620-eb2ef3b4a44a-certificates") pod "keda-operator-ffbb595cb-srvs2" (UID: "d69ca095-0b91-49b2-9620-eb2ef3b4a44a") : references non-existent secret key: ca.crt Apr 16 16:53:38.757060 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:38.756976 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/eaa475f1-8e01-48d3-a3ce-010e15939f85-certificates\") pod \"keda-metrics-apiserver-7c9f485588-7mjrr\" (UID: \"eaa475f1-8e01-48d3-a3ce-010e15939f85\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7mjrr" Apr 16 16:53:38.757428 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:38.757137 2574 secret.go:281] references non-existent secret key: tls.crt Apr 16 16:53:38.757428 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:38.757161 2574 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt Apr 16 16:53:38.757428 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:38.757184 2574 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-7c9f485588-7mjrr: references non-existent secret key: tls.crt Apr 16 16:53:38.757428 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:38.757265 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eaa475f1-8e01-48d3-a3ce-010e15939f85-certificates podName:eaa475f1-8e01-48d3-a3ce-010e15939f85 nodeName:}" failed. No retries permitted until 2026-04-16 16:53:39.757245766 +0000 UTC m=+353.838979973 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/eaa475f1-8e01-48d3-a3ce-010e15939f85-certificates") pod "keda-metrics-apiserver-7c9f485588-7mjrr" (UID: "eaa475f1-8e01-48d3-a3ce-010e15939f85") : references non-existent secret key: tls.crt Apr 16 16:53:39.462517 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:39.462483 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d69ca095-0b91-49b2-9620-eb2ef3b4a44a-certificates\") pod \"keda-operator-ffbb595cb-srvs2\" (UID: \"d69ca095-0b91-49b2-9620-eb2ef3b4a44a\") " pod="openshift-keda/keda-operator-ffbb595cb-srvs2" Apr 16 16:53:39.462686 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:39.462606 2574 secret.go:281] references non-existent secret key: ca.crt Apr 16 16:53:39.462686 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:39.462618 2574 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt Apr 16 16:53:39.462686 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:39.462626 2574 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-srvs2: references non-existent secret key: ca.crt Apr 16 16:53:39.462686 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:39.462670 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d69ca095-0b91-49b2-9620-eb2ef3b4a44a-certificates podName:d69ca095-0b91-49b2-9620-eb2ef3b4a44a nodeName:}" failed. No retries permitted until 2026-04-16 16:53:41.462659072 +0000 UTC m=+355.544393265 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/d69ca095-0b91-49b2-9620-eb2ef3b4a44a-certificates") pod "keda-operator-ffbb595cb-srvs2" (UID: "d69ca095-0b91-49b2-9620-eb2ef3b4a44a") : references non-existent secret key: ca.crt Apr 16 16:53:39.765363 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:39.765264 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/eaa475f1-8e01-48d3-a3ce-010e15939f85-certificates\") pod \"keda-metrics-apiserver-7c9f485588-7mjrr\" (UID: \"eaa475f1-8e01-48d3-a3ce-010e15939f85\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7mjrr" Apr 16 16:53:39.765714 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:39.765417 2574 secret.go:281] references non-existent secret key: tls.crt Apr 16 16:53:39.765714 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:39.765437 2574 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt Apr 16 16:53:39.765714 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:39.765457 2574 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-7c9f485588-7mjrr: references non-existent secret key: tls.crt Apr 16 16:53:39.765714 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:39.765510 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eaa475f1-8e01-48d3-a3ce-010e15939f85-certificates podName:eaa475f1-8e01-48d3-a3ce-010e15939f85 nodeName:}" failed. No retries permitted until 2026-04-16 16:53:41.765495472 +0000 UTC m=+355.847229664 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/eaa475f1-8e01-48d3-a3ce-010e15939f85-certificates") pod "keda-metrics-apiserver-7c9f485588-7mjrr" (UID: "eaa475f1-8e01-48d3-a3ce-010e15939f85") : references non-existent secret key: tls.crt Apr 16 16:53:41.479764 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:41.479730 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d69ca095-0b91-49b2-9620-eb2ef3b4a44a-certificates\") pod \"keda-operator-ffbb595cb-srvs2\" (UID: \"d69ca095-0b91-49b2-9620-eb2ef3b4a44a\") " pod="openshift-keda/keda-operator-ffbb595cb-srvs2" Apr 16 16:53:41.480225 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:41.479895 2574 secret.go:281] references non-existent secret key: ca.crt Apr 16 16:53:41.480225 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:41.479921 2574 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt Apr 16 16:53:41.480225 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:41.479932 2574 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-srvs2: references non-existent secret key: ca.crt Apr 16 16:53:41.480225 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:41.479997 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d69ca095-0b91-49b2-9620-eb2ef3b4a44a-certificates podName:d69ca095-0b91-49b2-9620-eb2ef3b4a44a nodeName:}" failed. No retries permitted until 2026-04-16 16:53:45.479977643 +0000 UTC m=+359.561711854 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/d69ca095-0b91-49b2-9620-eb2ef3b4a44a-certificates") pod "keda-operator-ffbb595cb-srvs2" (UID: "d69ca095-0b91-49b2-9620-eb2ef3b4a44a") : references non-existent secret key: ca.crt Apr 16 16:53:41.782241 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:41.782151 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/eaa475f1-8e01-48d3-a3ce-010e15939f85-certificates\") pod \"keda-metrics-apiserver-7c9f485588-7mjrr\" (UID: \"eaa475f1-8e01-48d3-a3ce-010e15939f85\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7mjrr" Apr 16 16:53:41.782365 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:41.782277 2574 secret.go:281] references non-existent secret key: tls.crt Apr 16 16:53:41.782365 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:41.782289 2574 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt Apr 16 16:53:41.782365 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:41.782305 2574 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-7c9f485588-7mjrr: references non-existent secret key: tls.crt Apr 16 16:53:41.782365 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:53:41.782353 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eaa475f1-8e01-48d3-a3ce-010e15939f85-certificates podName:eaa475f1-8e01-48d3-a3ce-010e15939f85 nodeName:}" failed. No retries permitted until 2026-04-16 16:53:45.782337793 +0000 UTC m=+359.864071999 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/eaa475f1-8e01-48d3-a3ce-010e15939f85-certificates") pod "keda-metrics-apiserver-7c9f485588-7mjrr" (UID: "eaa475f1-8e01-48d3-a3ce-010e15939f85") : references non-existent secret key: tls.crt Apr 16 16:53:45.510961 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:45.510926 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d69ca095-0b91-49b2-9620-eb2ef3b4a44a-certificates\") pod \"keda-operator-ffbb595cb-srvs2\" (UID: \"d69ca095-0b91-49b2-9620-eb2ef3b4a44a\") " pod="openshift-keda/keda-operator-ffbb595cb-srvs2" Apr 16 16:53:45.513230 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:45.513189 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d69ca095-0b91-49b2-9620-eb2ef3b4a44a-certificates\") pod \"keda-operator-ffbb595cb-srvs2\" (UID: \"d69ca095-0b91-49b2-9620-eb2ef3b4a44a\") " pod="openshift-keda/keda-operator-ffbb595cb-srvs2" Apr 16 16:53:45.575395 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:45.575368 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-ffbb595cb-srvs2" Apr 16 16:53:45.687160 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:45.687101 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-ffbb595cb-srvs2"] Apr 16 16:53:45.689601 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:53:45.689574 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd69ca095_0b91_49b2_9620_eb2ef3b4a44a.slice/crio-cd1ae09b45a4f9cd4f9c33c2dae2d58e4d62f25d76dd83d0be893595260a0f5a WatchSource:0}: Error finding container cd1ae09b45a4f9cd4f9c33c2dae2d58e4d62f25d76dd83d0be893595260a0f5a: Status 404 returned error can't find the container with id cd1ae09b45a4f9cd4f9c33c2dae2d58e4d62f25d76dd83d0be893595260a0f5a Apr 16 16:53:45.813852 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:45.813785 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/eaa475f1-8e01-48d3-a3ce-010e15939f85-certificates\") pod \"keda-metrics-apiserver-7c9f485588-7mjrr\" (UID: \"eaa475f1-8e01-48d3-a3ce-010e15939f85\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7mjrr" Apr 16 16:53:45.816077 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:45.816056 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/eaa475f1-8e01-48d3-a3ce-010e15939f85-certificates\") pod \"keda-metrics-apiserver-7c9f485588-7mjrr\" (UID: \"eaa475f1-8e01-48d3-a3ce-010e15939f85\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7mjrr" Apr 16 16:53:45.898404 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:45.898372 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7mjrr" Apr 16 16:53:46.017681 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:46.017660 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-7c9f485588-7mjrr"] Apr 16 16:53:46.019223 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:53:46.019179 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaa475f1_8e01_48d3_a3ce_010e15939f85.slice/crio-933cec74b96ff8703c5da56d8d17854e14805b1f294c99bb59565cf177ced062 WatchSource:0}: Error finding container 933cec74b96ff8703c5da56d8d17854e14805b1f294c99bb59565cf177ced062: Status 404 returned error can't find the container with id 933cec74b96ff8703c5da56d8d17854e14805b1f294c99bb59565cf177ced062 Apr 16 16:53:46.582722 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:46.582657 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7mjrr" event={"ID":"eaa475f1-8e01-48d3-a3ce-010e15939f85","Type":"ContainerStarted","Data":"933cec74b96ff8703c5da56d8d17854e14805b1f294c99bb59565cf177ced062"} Apr 16 16:53:46.583993 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:46.583960 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-ffbb595cb-srvs2" event={"ID":"d69ca095-0b91-49b2-9620-eb2ef3b4a44a","Type":"ContainerStarted","Data":"cd1ae09b45a4f9cd4f9c33c2dae2d58e4d62f25d76dd83d0be893595260a0f5a"} Apr 16 16:53:50.598039 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:50.598003 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7mjrr" event={"ID":"eaa475f1-8e01-48d3-a3ce-010e15939f85","Type":"ContainerStarted","Data":"8cc83b855732d43bae775d04247e203c46cb4a58337e1f021270af5cbbded17f"} Apr 16 16:53:50.598595 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:50.598229 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7mjrr" Apr 16 16:53:50.599268 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:50.599244 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-ffbb595cb-srvs2" event={"ID":"d69ca095-0b91-49b2-9620-eb2ef3b4a44a","Type":"ContainerStarted","Data":"25c3220bfe84c68134278d55f798e24399440597479e46e2ba1ba3d7090cb603"} Apr 16 16:53:50.599353 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:50.599299 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-ffbb595cb-srvs2" Apr 16 16:53:50.614620 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:50.614577 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7mjrr" podStartSLOduration=8.888625249 podStartE2EDuration="12.614564828s" podCreationTimestamp="2026-04-16 16:53:38 +0000 UTC" firstStartedPulling="2026-04-16 16:53:46.02060709 +0000 UTC m=+360.102341282" lastFinishedPulling="2026-04-16 16:53:49.746546669 +0000 UTC m=+363.828280861" observedRunningTime="2026-04-16 16:53:50.613620145 +0000 UTC m=+364.695354360" watchObservedRunningTime="2026-04-16 16:53:50.614564828 +0000 UTC m=+364.696299041" Apr 16 16:53:50.629333 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:50.629290 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-keda/keda-operator-ffbb595cb-srvs2" podStartSLOduration=9.561677276 podStartE2EDuration="13.629278754s" podCreationTimestamp="2026-04-16 16:53:37 +0000 UTC" firstStartedPulling="2026-04-16 16:53:45.691126476 +0000 UTC m=+359.772860678" lastFinishedPulling="2026-04-16 16:53:49.758727948 +0000 UTC m=+363.840462156" observedRunningTime="2026-04-16 16:53:50.628917981 +0000 UTC m=+364.710652218" watchObservedRunningTime="2026-04-16 16:53:50.629278754 +0000 UTC m=+364.711012968" Apr 16 16:53:58.558905 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:53:58.558874 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-5kcmw" Apr 16 16:54:01.606617 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:01.606588 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-7mjrr" Apr 16 16:54:11.604286 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:11.604249 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-ffbb595cb-srvs2" Apr 16 16:54:46.178072 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.178037 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-6986579898-qnzn6"] Apr 16 16:54:46.181267 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.181252 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-6986579898-qnzn6" Apr 16 16:54:46.184052 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.184019 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\"" Apr 16 16:54:46.184885 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.184864 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-xpbhb\"" Apr 16 16:54:46.185010 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.184899 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Apr 16 16:54:46.185010 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.184899 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Apr 16 16:54:46.185457 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.185435 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-68cc5db7c4-vht6t"] Apr 16 16:54:46.189398 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.189378 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-6986579898-qnzn6"] Apr 16 16:54:46.189508 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.189493 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-68cc5db7c4-vht6t" Apr 16 16:54:46.191682 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.191665 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\"" Apr 16 16:54:46.191921 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.191907 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-7q8vs\"" Apr 16 16:54:46.197523 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.197502 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-68cc5db7c4-vht6t"] Apr 16 16:54:46.259950 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.259882 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6af7bcdc-a6ae-4c15-bac1-f9fe5b614473-cert\") pod \"kserve-controller-manager-6986579898-qnzn6\" (UID: \"6af7bcdc-a6ae-4c15-bac1-f9fe5b614473\") " pod="kserve/kserve-controller-manager-6986579898-qnzn6" Apr 16 16:54:46.259950 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.259926 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn7gm\" (UniqueName: \"kubernetes.io/projected/6af7bcdc-a6ae-4c15-bac1-f9fe5b614473-kube-api-access-bn7gm\") pod \"kserve-controller-manager-6986579898-qnzn6\" (UID: \"6af7bcdc-a6ae-4c15-bac1-f9fe5b614473\") " pod="kserve/kserve-controller-manager-6986579898-qnzn6" Apr 16 16:54:46.260120 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.259996 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b290ecac-fda9-416d-92b4-926c2fe09ee7-cert\") pod \"llmisvc-controller-manager-68cc5db7c4-vht6t\" (UID: \"b290ecac-fda9-416d-92b4-926c2fe09ee7\") " pod="kserve/llmisvc-controller-manager-68cc5db7c4-vht6t" Apr 16 16:54:46.260120 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.260021 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwmzt\" (UniqueName: \"kubernetes.io/projected/b290ecac-fda9-416d-92b4-926c2fe09ee7-kube-api-access-rwmzt\") pod \"llmisvc-controller-manager-68cc5db7c4-vht6t\" (UID: \"b290ecac-fda9-416d-92b4-926c2fe09ee7\") " pod="kserve/llmisvc-controller-manager-68cc5db7c4-vht6t" Apr 16 16:54:46.361358 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.361332 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b290ecac-fda9-416d-92b4-926c2fe09ee7-cert\") pod \"llmisvc-controller-manager-68cc5db7c4-vht6t\" (UID: \"b290ecac-fda9-416d-92b4-926c2fe09ee7\") " pod="kserve/llmisvc-controller-manager-68cc5db7c4-vht6t" Apr 16 16:54:46.361480 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.361373 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rwmzt\" (UniqueName: \"kubernetes.io/projected/b290ecac-fda9-416d-92b4-926c2fe09ee7-kube-api-access-rwmzt\") pod \"llmisvc-controller-manager-68cc5db7c4-vht6t\" (UID: \"b290ecac-fda9-416d-92b4-926c2fe09ee7\") " pod="kserve/llmisvc-controller-manager-68cc5db7c4-vht6t" Apr 16 16:54:46.361480 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.361413 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6af7bcdc-a6ae-4c15-bac1-f9fe5b614473-cert\") pod \"kserve-controller-manager-6986579898-qnzn6\" (UID: \"6af7bcdc-a6ae-4c15-bac1-f9fe5b614473\") " pod="kserve/kserve-controller-manager-6986579898-qnzn6" Apr 16 16:54:46.361480 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.361432 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bn7gm\" (UniqueName: \"kubernetes.io/projected/6af7bcdc-a6ae-4c15-bac1-f9fe5b614473-kube-api-access-bn7gm\") pod \"kserve-controller-manager-6986579898-qnzn6\" (UID: \"6af7bcdc-a6ae-4c15-bac1-f9fe5b614473\") " pod="kserve/kserve-controller-manager-6986579898-qnzn6" Apr 16 16:54:46.364286 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.364270 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\"" Apr 16 16:54:46.364348 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.364270 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\"" Apr 16 16:54:46.369515 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.369497 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Apr 16 16:54:46.373672 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.373649 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b290ecac-fda9-416d-92b4-926c2fe09ee7-cert\") pod \"llmisvc-controller-manager-68cc5db7c4-vht6t\" (UID: \"b290ecac-fda9-416d-92b4-926c2fe09ee7\") " pod="kserve/llmisvc-controller-manager-68cc5db7c4-vht6t" Apr 16 16:54:46.373749 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.373689 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6af7bcdc-a6ae-4c15-bac1-f9fe5b614473-cert\") pod \"kserve-controller-manager-6986579898-qnzn6\" (UID: \"6af7bcdc-a6ae-4c15-bac1-f9fe5b614473\") " pod="kserve/kserve-controller-manager-6986579898-qnzn6" Apr 16 16:54:46.380663 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.380637 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Apr 16 16:54:46.390623 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.390598 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn7gm\" (UniqueName: \"kubernetes.io/projected/6af7bcdc-a6ae-4c15-bac1-f9fe5b614473-kube-api-access-bn7gm\") pod \"kserve-controller-manager-6986579898-qnzn6\" (UID: \"6af7bcdc-a6ae-4c15-bac1-f9fe5b614473\") " pod="kserve/kserve-controller-manager-6986579898-qnzn6" Apr 16 16:54:46.390809 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.390791 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwmzt\" (UniqueName: \"kubernetes.io/projected/b290ecac-fda9-416d-92b4-926c2fe09ee7-kube-api-access-rwmzt\") pod \"llmisvc-controller-manager-68cc5db7c4-vht6t\" (UID: \"b290ecac-fda9-416d-92b4-926c2fe09ee7\") " pod="kserve/llmisvc-controller-manager-68cc5db7c4-vht6t" Apr 16 16:54:46.496994 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.496942 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-xpbhb\"" Apr 16 16:54:46.504299 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.504285 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-6986579898-qnzn6" Apr 16 16:54:46.506815 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.506797 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-7q8vs\"" Apr 16 16:54:46.514741 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.514717 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-68cc5db7c4-vht6t" Apr 16 16:54:46.629566 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.629460 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-6986579898-qnzn6"] Apr 16 16:54:46.632431 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:54:46.632407 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6af7bcdc_a6ae_4c15_bac1_f9fe5b614473.slice/crio-e58aa50406266be166bd6d7d4d7a206546131c631e3399eb55529fd35e275eac WatchSource:0}: Error finding container e58aa50406266be166bd6d7d4d7a206546131c631e3399eb55529fd35e275eac: Status 404 returned error can't find the container with id e58aa50406266be166bd6d7d4d7a206546131c631e3399eb55529fd35e275eac Apr 16 16:54:46.650552 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.650530 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-68cc5db7c4-vht6t"] Apr 16 16:54:46.652439 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:54:46.652415 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb290ecac_fda9_416d_92b4_926c2fe09ee7.slice/crio-fff97a7b0158dae4d13f3e28d92cf86417f808002df2527216811daec5d7f312 WatchSource:0}: Error finding container fff97a7b0158dae4d13f3e28d92cf86417f808002df2527216811daec5d7f312: Status 404 returned error can't find the container with id fff97a7b0158dae4d13f3e28d92cf86417f808002df2527216811daec5d7f312 Apr 16 16:54:46.772834 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.772772 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-68cc5db7c4-vht6t" event={"ID":"b290ecac-fda9-416d-92b4-926c2fe09ee7","Type":"ContainerStarted","Data":"fff97a7b0158dae4d13f3e28d92cf86417f808002df2527216811daec5d7f312"} Apr 16 16:54:46.773694 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:46.773669 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-6986579898-qnzn6" event={"ID":"6af7bcdc-a6ae-4c15-bac1-f9fe5b614473","Type":"ContainerStarted","Data":"e58aa50406266be166bd6d7d4d7a206546131c631e3399eb55529fd35e275eac"} Apr 16 16:54:50.789192 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:50.789157 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-6986579898-qnzn6" event={"ID":"6af7bcdc-a6ae-4c15-bac1-f9fe5b614473","Type":"ContainerStarted","Data":"4aa80eefacbb4726289980661c6f5179286d39fe229790274b4b4dc1428bde5f"} Apr 16 16:54:50.789611 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:50.789241 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-6986579898-qnzn6" Apr 16 16:54:50.790561 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:50.790535 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-68cc5db7c4-vht6t" event={"ID":"b290ecac-fda9-416d-92b4-926c2fe09ee7","Type":"ContainerStarted","Data":"afd30a0b992a2446fa0288b47a5333ef7a503605265e88e04d4fbe6ae505baa8"} Apr 16 16:54:50.790679 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:50.790655 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-68cc5db7c4-vht6t" Apr 16 16:54:50.822484 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:50.822447 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/kserve-controller-manager-6986579898-qnzn6" podStartSLOduration=1.5703411840000001 podStartE2EDuration="4.822434504s" podCreationTimestamp="2026-04-16 16:54:46 +0000 UTC" firstStartedPulling="2026-04-16 16:54:46.63409039 +0000 UTC m=+420.715824583" lastFinishedPulling="2026-04-16 16:54:49.886183711 +0000 UTC m=+423.967917903" observedRunningTime="2026-04-16 16:54:50.821464295 +0000 UTC m=+424.903198508" watchObservedRunningTime="2026-04-16 16:54:50.822434504 +0000 UTC m=+424.904168716" Apr 16 16:54:50.842972 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:54:50.842924 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-68cc5db7c4-vht6t" podStartSLOduration=1.6564640819999998 podStartE2EDuration="4.842910159s" podCreationTimestamp="2026-04-16 16:54:46 +0000 UTC" firstStartedPulling="2026-04-16 16:54:46.65368613 +0000 UTC m=+420.735420322" lastFinishedPulling="2026-04-16 16:54:49.840132206 +0000 UTC m=+423.921866399" observedRunningTime="2026-04-16 16:54:50.842354247 +0000 UTC m=+424.924088462" watchObservedRunningTime="2026-04-16 16:54:50.842910159 +0000 UTC m=+424.924644372" Apr 16 16:55:21.795373 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:21.795344 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-68cc5db7c4-vht6t" Apr 16 16:55:21.798460 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:21.798439 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-6986579898-qnzn6" Apr 16 16:55:23.026622 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:23.026590 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-6986579898-qnzn6"] Apr 16 16:55:23.026983 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:23.026797 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve/kserve-controller-manager-6986579898-qnzn6" podUID="6af7bcdc-a6ae-4c15-bac1-f9fe5b614473" containerName="manager" containerID="cri-o://4aa80eefacbb4726289980661c6f5179286d39fe229790274b4b4dc1428bde5f" gracePeriod=10 Apr 16 16:55:23.283579 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:23.283528 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-6986579898-qnzn6" Apr 16 16:55:23.450951 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:23.450906 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6af7bcdc-a6ae-4c15-bac1-f9fe5b614473-cert\") pod \"6af7bcdc-a6ae-4c15-bac1-f9fe5b614473\" (UID: \"6af7bcdc-a6ae-4c15-bac1-f9fe5b614473\") " Apr 16 16:55:23.450951 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:23.450958 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bn7gm\" (UniqueName: \"kubernetes.io/projected/6af7bcdc-a6ae-4c15-bac1-f9fe5b614473-kube-api-access-bn7gm\") pod \"6af7bcdc-a6ae-4c15-bac1-f9fe5b614473\" (UID: \"6af7bcdc-a6ae-4c15-bac1-f9fe5b614473\") " Apr 16 16:55:23.453066 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:23.453040 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6af7bcdc-a6ae-4c15-bac1-f9fe5b614473-cert" (OuterVolumeSpecName: "cert") pod "6af7bcdc-a6ae-4c15-bac1-f9fe5b614473" (UID: "6af7bcdc-a6ae-4c15-bac1-f9fe5b614473"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 16:55:23.453162 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:23.453045 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6af7bcdc-a6ae-4c15-bac1-f9fe5b614473-kube-api-access-bn7gm" (OuterVolumeSpecName: "kube-api-access-bn7gm") pod "6af7bcdc-a6ae-4c15-bac1-f9fe5b614473" (UID: "6af7bcdc-a6ae-4c15-bac1-f9fe5b614473"). InnerVolumeSpecName "kube-api-access-bn7gm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 16:55:23.552256 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:23.552187 2574 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6af7bcdc-a6ae-4c15-bac1-f9fe5b614473-cert\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:55:23.552256 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:23.552222 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bn7gm\" (UniqueName: \"kubernetes.io/projected/6af7bcdc-a6ae-4c15-bac1-f9fe5b614473-kube-api-access-bn7gm\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:55:23.890276 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:23.890242 2574 generic.go:358] "Generic (PLEG): container finished" podID="6af7bcdc-a6ae-4c15-bac1-f9fe5b614473" containerID="4aa80eefacbb4726289980661c6f5179286d39fe229790274b4b4dc1428bde5f" exitCode=0 Apr 16 16:55:23.890450 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:23.890300 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-6986579898-qnzn6" Apr 16 16:55:23.890450 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:23.890306 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-6986579898-qnzn6" event={"ID":"6af7bcdc-a6ae-4c15-bac1-f9fe5b614473","Type":"ContainerDied","Data":"4aa80eefacbb4726289980661c6f5179286d39fe229790274b4b4dc1428bde5f"} Apr 16 16:55:23.890450 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:23.890337 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-6986579898-qnzn6" event={"ID":"6af7bcdc-a6ae-4c15-bac1-f9fe5b614473","Type":"ContainerDied","Data":"e58aa50406266be166bd6d7d4d7a206546131c631e3399eb55529fd35e275eac"} Apr 16 16:55:23.890450 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:23.890357 2574 scope.go:117] "RemoveContainer" containerID="4aa80eefacbb4726289980661c6f5179286d39fe229790274b4b4dc1428bde5f" Apr 16 16:55:23.898373 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:23.898350 2574 scope.go:117] "RemoveContainer" containerID="4aa80eefacbb4726289980661c6f5179286d39fe229790274b4b4dc1428bde5f" Apr 16 16:55:23.898636 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:55:23.898613 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4aa80eefacbb4726289980661c6f5179286d39fe229790274b4b4dc1428bde5f\": container with ID starting with 4aa80eefacbb4726289980661c6f5179286d39fe229790274b4b4dc1428bde5f not found: ID does not exist" containerID="4aa80eefacbb4726289980661c6f5179286d39fe229790274b4b4dc1428bde5f" Apr 16 16:55:23.898700 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:23.898645 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4aa80eefacbb4726289980661c6f5179286d39fe229790274b4b4dc1428bde5f"} err="failed to get container status \"4aa80eefacbb4726289980661c6f5179286d39fe229790274b4b4dc1428bde5f\": rpc error: code = NotFound desc = could not find container \"4aa80eefacbb4726289980661c6f5179286d39fe229790274b4b4dc1428bde5f\": container with ID starting with 4aa80eefacbb4726289980661c6f5179286d39fe229790274b4b4dc1428bde5f not found: ID does not exist" Apr 16 16:55:23.912214 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:23.912175 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-6986579898-qnzn6"] Apr 16 16:55:23.915109 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:23.915090 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-6986579898-qnzn6"] Apr 16 16:55:24.511645 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:24.511617 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6af7bcdc-a6ae-4c15-bac1-f9fe5b614473" path="/var/lib/kubelet/pods/6af7bcdc-a6ae-4c15-bac1-f9fe5b614473/volumes" Apr 16 16:55:58.291331 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:58.291295 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-696fc77849-rc4qq"] Apr 16 16:55:58.291781 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:58.291595 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6af7bcdc-a6ae-4c15-bac1-f9fe5b614473" containerName="manager" Apr 16 16:55:58.291781 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:58.291606 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6af7bcdc-a6ae-4c15-bac1-f9fe5b614473" containerName="manager" Apr 16 16:55:58.291781 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:58.291667 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="6af7bcdc-a6ae-4c15-bac1-f9fe5b614473" containerName="manager" Apr 16 16:55:58.294362 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:58.294344 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-696fc77849-rc4qq" Apr 16 16:55:58.299717 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:58.299699 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-tklxj\"" Apr 16 16:55:58.300634 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:58.300331 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\"" Apr 16 16:55:58.315888 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:58.315863 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-696fc77849-rc4qq"] Apr 16 16:55:58.318903 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:58.318878 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d9df816-3acd-4fd8-a61c-27820fe76f58-cert\") pod \"odh-model-controller-696fc77849-rc4qq\" (UID: \"9d9df816-3acd-4fd8-a61c-27820fe76f58\") " pod="kserve/odh-model-controller-696fc77849-rc4qq" Apr 16 16:55:58.318986 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:58.318940 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxddz\" (UniqueName: \"kubernetes.io/projected/9d9df816-3acd-4fd8-a61c-27820fe76f58-kube-api-access-qxddz\") pod \"odh-model-controller-696fc77849-rc4qq\" (UID: \"9d9df816-3acd-4fd8-a61c-27820fe76f58\") " pod="kserve/odh-model-controller-696fc77849-rc4qq" Apr 16 16:55:58.419955 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:58.419923 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d9df816-3acd-4fd8-a61c-27820fe76f58-cert\") pod \"odh-model-controller-696fc77849-rc4qq\" (UID: \"9d9df816-3acd-4fd8-a61c-27820fe76f58\") " pod="kserve/odh-model-controller-696fc77849-rc4qq" Apr 16 16:55:58.420098 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:58.419964 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qxddz\" (UniqueName: \"kubernetes.io/projected/9d9df816-3acd-4fd8-a61c-27820fe76f58-kube-api-access-qxddz\") pod \"odh-model-controller-696fc77849-rc4qq\" (UID: \"9d9df816-3acd-4fd8-a61c-27820fe76f58\") " pod="kserve/odh-model-controller-696fc77849-rc4qq" Apr 16 16:55:58.420098 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:55:58.420062 2574 secret.go:189] Couldn't get secret kserve/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Apr 16 16:55:58.420171 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:55:58.420120 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d9df816-3acd-4fd8-a61c-27820fe76f58-cert podName:9d9df816-3acd-4fd8-a61c-27820fe76f58 nodeName:}" failed. No retries permitted until 2026-04-16 16:55:58.920104758 +0000 UTC m=+493.001838951 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9d9df816-3acd-4fd8-a61c-27820fe76f58-cert") pod "odh-model-controller-696fc77849-rc4qq" (UID: "9d9df816-3acd-4fd8-a61c-27820fe76f58") : secret "odh-model-controller-webhook-cert" not found Apr 16 16:55:58.432969 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:58.432940 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxddz\" (UniqueName: \"kubernetes.io/projected/9d9df816-3acd-4fd8-a61c-27820fe76f58-kube-api-access-qxddz\") pod \"odh-model-controller-696fc77849-rc4qq\" (UID: \"9d9df816-3acd-4fd8-a61c-27820fe76f58\") " pod="kserve/odh-model-controller-696fc77849-rc4qq" Apr 16 16:55:58.923243 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:58.923182 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d9df816-3acd-4fd8-a61c-27820fe76f58-cert\") pod \"odh-model-controller-696fc77849-rc4qq\" (UID: \"9d9df816-3acd-4fd8-a61c-27820fe76f58\") " pod="kserve/odh-model-controller-696fc77849-rc4qq" Apr 16 16:55:58.923416 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:55:58.923332 2574 secret.go:189] Couldn't get secret kserve/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Apr 16 16:55:58.923416 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:55:58.923395 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d9df816-3acd-4fd8-a61c-27820fe76f58-cert podName:9d9df816-3acd-4fd8-a61c-27820fe76f58 nodeName:}" failed. No retries permitted until 2026-04-16 16:55:59.923380119 +0000 UTC m=+494.005114312 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9d9df816-3acd-4fd8-a61c-27820fe76f58-cert") pod "odh-model-controller-696fc77849-rc4qq" (UID: "9d9df816-3acd-4fd8-a61c-27820fe76f58") : secret "odh-model-controller-webhook-cert" not found Apr 16 16:55:59.930622 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:59.930587 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d9df816-3acd-4fd8-a61c-27820fe76f58-cert\") pod \"odh-model-controller-696fc77849-rc4qq\" (UID: \"9d9df816-3acd-4fd8-a61c-27820fe76f58\") " pod="kserve/odh-model-controller-696fc77849-rc4qq" Apr 16 16:55:59.932834 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:55:59.932815 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d9df816-3acd-4fd8-a61c-27820fe76f58-cert\") pod \"odh-model-controller-696fc77849-rc4qq\" (UID: \"9d9df816-3acd-4fd8-a61c-27820fe76f58\") " pod="kserve/odh-model-controller-696fc77849-rc4qq" Apr 16 16:56:00.103743 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:00.103717 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-696fc77849-rc4qq" Apr 16 16:56:00.232580 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:00.232558 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-696fc77849-rc4qq"] Apr 16 16:56:00.234842 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:56:00.234810 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d9df816_3acd_4fd8_a61c_27820fe76f58.slice/crio-5324c49617e2db309b2d9f10da9766d00ba5182d8c9b1dce238627371ffeacf4 WatchSource:0}: Error finding container 5324c49617e2db309b2d9f10da9766d00ba5182d8c9b1dce238627371ffeacf4: Status 404 returned error can't find the container with id 5324c49617e2db309b2d9f10da9766d00ba5182d8c9b1dce238627371ffeacf4 Apr 16 16:56:01.007837 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:01.007790 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-696fc77849-rc4qq" event={"ID":"9d9df816-3acd-4fd8-a61c-27820fe76f58","Type":"ContainerStarted","Data":"5324c49617e2db309b2d9f10da9766d00ba5182d8c9b1dce238627371ffeacf4"} Apr 16 16:56:03.016043 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:03.015970 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-696fc77849-rc4qq" event={"ID":"9d9df816-3acd-4fd8-a61c-27820fe76f58","Type":"ContainerStarted","Data":"fef6baaf66a21a7725eaa2c3308d620b503f256caa46236cfaa3ab960ed7dc5b"} Apr 16 16:56:03.016418 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:03.016092 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-696fc77849-rc4qq" Apr 16 16:56:03.056456 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:03.056403 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/odh-model-controller-696fc77849-rc4qq" podStartSLOduration=2.549196634 podStartE2EDuration="5.056388695s" podCreationTimestamp="2026-04-16 16:55:58 +0000 UTC" firstStartedPulling="2026-04-16 16:56:00.236005034 +0000 UTC m=+494.317739227" lastFinishedPulling="2026-04-16 16:56:02.743197095 +0000 UTC m=+496.824931288" observedRunningTime="2026-04-16 16:56:03.052873341 +0000 UTC m=+497.134607555" watchObservedRunningTime="2026-04-16 16:56:03.056388695 +0000 UTC m=+497.138122909" Apr 16 16:56:14.021479 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:14.021450 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-696fc77849-rc4qq" Apr 16 16:56:26.156554 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:26.156478 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-tls-custom-ddd4dbfd-zczvt"] Apr 16 16:56:26.163718 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:26.163698 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-ddd4dbfd-zczvt" Apr 16 16:56:26.166641 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:26.166607 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-ddd4dbfd-zczvt"] Apr 16 16:56:26.166641 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:26.166636 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-9c52w\"" Apr 16 16:56:26.166810 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:26.166677 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom-artifact\"" Apr 16 16:56:26.245899 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:26.245871 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6cs8\" (UniqueName: \"kubernetes.io/projected/f7daa72d-84c3-4330-882f-e55649f2efd2-kube-api-access-x6cs8\") pod \"seaweedfs-tls-custom-ddd4dbfd-zczvt\" (UID: \"f7daa72d-84c3-4330-882f-e55649f2efd2\") " pod="kserve/seaweedfs-tls-custom-ddd4dbfd-zczvt" Apr 16 16:56:26.246021 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:26.245930 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f7daa72d-84c3-4330-882f-e55649f2efd2-data\") pod \"seaweedfs-tls-custom-ddd4dbfd-zczvt\" (UID: \"f7daa72d-84c3-4330-882f-e55649f2efd2\") " pod="kserve/seaweedfs-tls-custom-ddd4dbfd-zczvt" Apr 16 16:56:26.346457 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:26.346426 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x6cs8\" (UniqueName: \"kubernetes.io/projected/f7daa72d-84c3-4330-882f-e55649f2efd2-kube-api-access-x6cs8\") pod \"seaweedfs-tls-custom-ddd4dbfd-zczvt\" (UID: \"f7daa72d-84c3-4330-882f-e55649f2efd2\") " pod="kserve/seaweedfs-tls-custom-ddd4dbfd-zczvt" Apr 16 16:56:26.346577 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:26.346518 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f7daa72d-84c3-4330-882f-e55649f2efd2-data\") pod \"seaweedfs-tls-custom-ddd4dbfd-zczvt\" (UID: \"f7daa72d-84c3-4330-882f-e55649f2efd2\") " pod="kserve/seaweedfs-tls-custom-ddd4dbfd-zczvt" Apr 16 16:56:26.346877 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:26.346856 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f7daa72d-84c3-4330-882f-e55649f2efd2-data\") pod \"seaweedfs-tls-custom-ddd4dbfd-zczvt\" (UID: \"f7daa72d-84c3-4330-882f-e55649f2efd2\") " pod="kserve/seaweedfs-tls-custom-ddd4dbfd-zczvt" Apr 16 16:56:26.355185 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:26.355161 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6cs8\" (UniqueName: \"kubernetes.io/projected/f7daa72d-84c3-4330-882f-e55649f2efd2-kube-api-access-x6cs8\") pod \"seaweedfs-tls-custom-ddd4dbfd-zczvt\" (UID: \"f7daa72d-84c3-4330-882f-e55649f2efd2\") " pod="kserve/seaweedfs-tls-custom-ddd4dbfd-zczvt" Apr 16 16:56:26.474549 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:26.474452 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-ddd4dbfd-zczvt" Apr 16 16:56:26.593610 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:26.590142 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-ddd4dbfd-zczvt"] Apr 16 16:56:26.595162 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:56:26.595136 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7daa72d_84c3_4330_882f_e55649f2efd2.slice/crio-c7875266a212cf629c8ebf68094b57c20cca06e4234e981bc47f6141139940bd WatchSource:0}: Error finding container c7875266a212cf629c8ebf68094b57c20cca06e4234e981bc47f6141139940bd: Status 404 returned error can't find the container with id c7875266a212cf629c8ebf68094b57c20cca06e4234e981bc47f6141139940bd Apr 16 16:56:27.095163 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:27.095130 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-ddd4dbfd-zczvt" event={"ID":"f7daa72d-84c3-4330-882f-e55649f2efd2","Type":"ContainerStarted","Data":"c7875266a212cf629c8ebf68094b57c20cca06e4234e981bc47f6141139940bd"} Apr 16 16:56:30.107235 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:30.107185 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-ddd4dbfd-zczvt" event={"ID":"f7daa72d-84c3-4330-882f-e55649f2efd2","Type":"ContainerStarted","Data":"9fe59579586414ef50500470365fb0542ef251e279c9d91ab07635e0ac82a30f"} Apr 16 16:56:30.127088 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:30.127044 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/seaweedfs-tls-custom-ddd4dbfd-zczvt" podStartSLOduration=1.626009964 podStartE2EDuration="4.127032373s" podCreationTimestamp="2026-04-16 16:56:26 +0000 UTC" firstStartedPulling="2026-04-16 16:56:26.596431503 +0000 UTC m=+520.678165695" lastFinishedPulling="2026-04-16 16:56:29.097453911 +0000 UTC m=+523.179188104" observedRunningTime="2026-04-16 16:56:30.126021627 +0000 UTC m=+524.207755841" watchObservedRunningTime="2026-04-16 16:56:30.127032373 +0000 UTC m=+524.208766586" Apr 16 16:56:32.306380 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:32.306347 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve/seaweedfs-tls-custom-ddd4dbfd-zczvt"] Apr 16 16:56:32.306764 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:32.306532 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve/seaweedfs-tls-custom-ddd4dbfd-zczvt" podUID="f7daa72d-84c3-4330-882f-e55649f2efd2" containerName="seaweedfs-tls-custom" containerID="cri-o://9fe59579586414ef50500470365fb0542ef251e279c9d91ab07635e0ac82a30f" gracePeriod=30 Apr 16 16:56:33.554044 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:33.554016 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-ddd4dbfd-zczvt" Apr 16 16:56:33.706842 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:33.706799 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6cs8\" (UniqueName: \"kubernetes.io/projected/f7daa72d-84c3-4330-882f-e55649f2efd2-kube-api-access-x6cs8\") pod \"f7daa72d-84c3-4330-882f-e55649f2efd2\" (UID: \"f7daa72d-84c3-4330-882f-e55649f2efd2\") " Apr 16 16:56:33.707014 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:33.706876 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f7daa72d-84c3-4330-882f-e55649f2efd2-data\") pod \"f7daa72d-84c3-4330-882f-e55649f2efd2\" (UID: \"f7daa72d-84c3-4330-882f-e55649f2efd2\") " Apr 16 16:56:33.708079 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:33.708048 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7daa72d-84c3-4330-882f-e55649f2efd2-data" (OuterVolumeSpecName: "data") pod "f7daa72d-84c3-4330-882f-e55649f2efd2" (UID: "f7daa72d-84c3-4330-882f-e55649f2efd2"). InnerVolumeSpecName "data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 16 16:56:33.708786 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:33.708763 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7daa72d-84c3-4330-882f-e55649f2efd2-kube-api-access-x6cs8" (OuterVolumeSpecName: "kube-api-access-x6cs8") pod "f7daa72d-84c3-4330-882f-e55649f2efd2" (UID: "f7daa72d-84c3-4330-882f-e55649f2efd2"). InnerVolumeSpecName "kube-api-access-x6cs8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 16:56:33.807627 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:33.807606 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6cs8\" (UniqueName: \"kubernetes.io/projected/f7daa72d-84c3-4330-882f-e55649f2efd2-kube-api-access-x6cs8\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:56:33.807627 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:33.807626 2574 reconciler_common.go:299] "Volume detached for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f7daa72d-84c3-4330-882f-e55649f2efd2-data\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:56:34.120953 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.120920 2574 generic.go:358] "Generic (PLEG): container finished" podID="f7daa72d-84c3-4330-882f-e55649f2efd2" containerID="9fe59579586414ef50500470365fb0542ef251e279c9d91ab07635e0ac82a30f" exitCode=0 Apr 16 16:56:34.121137 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.120973 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-ddd4dbfd-zczvt" event={"ID":"f7daa72d-84c3-4330-882f-e55649f2efd2","Type":"ContainerDied","Data":"9fe59579586414ef50500470365fb0542ef251e279c9d91ab07635e0ac82a30f"} Apr 16 16:56:34.121137 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.120997 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-ddd4dbfd-zczvt" event={"ID":"f7daa72d-84c3-4330-882f-e55649f2efd2","Type":"ContainerDied","Data":"c7875266a212cf629c8ebf68094b57c20cca06e4234e981bc47f6141139940bd"} Apr 16 16:56:34.121137 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.121011 2574 scope.go:117] "RemoveContainer" containerID="9fe59579586414ef50500470365fb0542ef251e279c9d91ab07635e0ac82a30f" Apr 16 16:56:34.121137 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.121019 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-ddd4dbfd-zczvt" Apr 16 16:56:34.129852 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.129834 2574 scope.go:117] "RemoveContainer" containerID="9fe59579586414ef50500470365fb0542ef251e279c9d91ab07635e0ac82a30f" Apr 16 16:56:34.130090 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:56:34.130071 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fe59579586414ef50500470365fb0542ef251e279c9d91ab07635e0ac82a30f\": container with ID starting with 9fe59579586414ef50500470365fb0542ef251e279c9d91ab07635e0ac82a30f not found: ID does not exist" containerID="9fe59579586414ef50500470365fb0542ef251e279c9d91ab07635e0ac82a30f" Apr 16 16:56:34.130136 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.130099 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fe59579586414ef50500470365fb0542ef251e279c9d91ab07635e0ac82a30f"} err="failed to get container status \"9fe59579586414ef50500470365fb0542ef251e279c9d91ab07635e0ac82a30f\": rpc error: code = NotFound desc = could not find container \"9fe59579586414ef50500470365fb0542ef251e279c9d91ab07635e0ac82a30f\": container with ID starting with 9fe59579586414ef50500470365fb0542ef251e279c9d91ab07635e0ac82a30f not found: ID does not exist" Apr 16 16:56:34.142988 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.142967 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve/seaweedfs-tls-custom-ddd4dbfd-zczvt"] Apr 16 16:56:34.144382 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.144365 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve/seaweedfs-tls-custom-ddd4dbfd-zczvt"] Apr 16 16:56:34.178812 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.178791 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-tls-custom-5c88b85bb7-wmrmt"] Apr 16 16:56:34.179105 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.179093 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7daa72d-84c3-4330-882f-e55649f2efd2" containerName="seaweedfs-tls-custom" Apr 16 16:56:34.179154 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.179105 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7daa72d-84c3-4330-882f-e55649f2efd2" containerName="seaweedfs-tls-custom" Apr 16 16:56:34.179189 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.179163 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="f7daa72d-84c3-4330-882f-e55649f2efd2" containerName="seaweedfs-tls-custom" Apr 16 16:56:34.183116 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.183102 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-5c88b85bb7-wmrmt" Apr 16 16:56:34.185740 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.185718 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom\"" Apr 16 16:56:34.185842 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.185748 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom-artifact\"" Apr 16 16:56:34.185842 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.185778 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-9c52w\"" Apr 16 16:56:34.190764 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.190740 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-5c88b85bb7-wmrmt"] Apr 16 16:56:34.315597 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.315576 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"seaweedfs-tls-custom\" (UniqueName: \"kubernetes.io/projected/99c4631c-42c7-4173-b33e-055c2de79f60-seaweedfs-tls-custom\") pod \"seaweedfs-tls-custom-5c88b85bb7-wmrmt\" (UID: \"99c4631c-42c7-4173-b33e-055c2de79f60\") " pod="kserve/seaweedfs-tls-custom-5c88b85bb7-wmrmt" Apr 16 16:56:34.315701 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.315616 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq8wk\" (UniqueName: \"kubernetes.io/projected/99c4631c-42c7-4173-b33e-055c2de79f60-kube-api-access-pq8wk\") pod \"seaweedfs-tls-custom-5c88b85bb7-wmrmt\" (UID: \"99c4631c-42c7-4173-b33e-055c2de79f60\") " pod="kserve/seaweedfs-tls-custom-5c88b85bb7-wmrmt" Apr 16 16:56:34.315701 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.315643 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/99c4631c-42c7-4173-b33e-055c2de79f60-data\") pod \"seaweedfs-tls-custom-5c88b85bb7-wmrmt\" (UID: \"99c4631c-42c7-4173-b33e-055c2de79f60\") " pod="kserve/seaweedfs-tls-custom-5c88b85bb7-wmrmt" Apr 16 16:56:34.416915 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.416845 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/99c4631c-42c7-4173-b33e-055c2de79f60-data\") pod \"seaweedfs-tls-custom-5c88b85bb7-wmrmt\" (UID: \"99c4631c-42c7-4173-b33e-055c2de79f60\") " pod="kserve/seaweedfs-tls-custom-5c88b85bb7-wmrmt" Apr 16 16:56:34.417039 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.416934 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"seaweedfs-tls-custom\" (UniqueName: \"kubernetes.io/projected/99c4631c-42c7-4173-b33e-055c2de79f60-seaweedfs-tls-custom\") pod \"seaweedfs-tls-custom-5c88b85bb7-wmrmt\" (UID: \"99c4631c-42c7-4173-b33e-055c2de79f60\") " pod="kserve/seaweedfs-tls-custom-5c88b85bb7-wmrmt" Apr 16 16:56:34.417039 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.416972 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pq8wk\" (UniqueName: \"kubernetes.io/projected/99c4631c-42c7-4173-b33e-055c2de79f60-kube-api-access-pq8wk\") pod \"seaweedfs-tls-custom-5c88b85bb7-wmrmt\" (UID: \"99c4631c-42c7-4173-b33e-055c2de79f60\") " pod="kserve/seaweedfs-tls-custom-5c88b85bb7-wmrmt" Apr 16 16:56:34.417262 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.417242 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/99c4631c-42c7-4173-b33e-055c2de79f60-data\") pod \"seaweedfs-tls-custom-5c88b85bb7-wmrmt\" (UID: \"99c4631c-42c7-4173-b33e-055c2de79f60\") " pod="kserve/seaweedfs-tls-custom-5c88b85bb7-wmrmt" Apr 16 16:56:34.419354 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.419336 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"seaweedfs-tls-custom\" (UniqueName: \"kubernetes.io/projected/99c4631c-42c7-4173-b33e-055c2de79f60-seaweedfs-tls-custom\") pod \"seaweedfs-tls-custom-5c88b85bb7-wmrmt\" (UID: \"99c4631c-42c7-4173-b33e-055c2de79f60\") " pod="kserve/seaweedfs-tls-custom-5c88b85bb7-wmrmt" Apr 16 16:56:34.425400 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.425377 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq8wk\" (UniqueName: \"kubernetes.io/projected/99c4631c-42c7-4173-b33e-055c2de79f60-kube-api-access-pq8wk\") pod \"seaweedfs-tls-custom-5c88b85bb7-wmrmt\" (UID: \"99c4631c-42c7-4173-b33e-055c2de79f60\") " pod="kserve/seaweedfs-tls-custom-5c88b85bb7-wmrmt" Apr 16 16:56:34.491699 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.491678 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-5c88b85bb7-wmrmt" Apr 16 16:56:34.512416 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.512390 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7daa72d-84c3-4330-882f-e55649f2efd2" path="/var/lib/kubelet/pods/f7daa72d-84c3-4330-882f-e55649f2efd2/volumes" Apr 16 16:56:34.603927 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:34.603906 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-5c88b85bb7-wmrmt"] Apr 16 16:56:34.606264 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:56:34.606236 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99c4631c_42c7_4173_b33e_055c2de79f60.slice/crio-2a3a6f29505211d14d4b101c86eb4cc9aef08af86ed28fa1747a7833091512ee WatchSource:0}: Error finding container 2a3a6f29505211d14d4b101c86eb4cc9aef08af86ed28fa1747a7833091512ee: Status 404 returned error can't find the container with id 2a3a6f29505211d14d4b101c86eb4cc9aef08af86ed28fa1747a7833091512ee Apr 16 16:56:35.125981 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:35.125904 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-5c88b85bb7-wmrmt" event={"ID":"99c4631c-42c7-4173-b33e-055c2de79f60","Type":"ContainerStarted","Data":"8fe7811f90f71805b08437aed43617f8c549edaee0f1ee49219a628b498b85d8"} Apr 16 16:56:35.125981 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:35.125943 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-5c88b85bb7-wmrmt" event={"ID":"99c4631c-42c7-4173-b33e-055c2de79f60","Type":"ContainerStarted","Data":"2a3a6f29505211d14d4b101c86eb4cc9aef08af86ed28fa1747a7833091512ee"} Apr 16 16:56:35.144000 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:35.143958 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/seaweedfs-tls-custom-5c88b85bb7-wmrmt" podStartSLOduration=0.881694728 podStartE2EDuration="1.143943188s" podCreationTimestamp="2026-04-16 16:56:34 +0000 UTC" firstStartedPulling="2026-04-16 16:56:34.607523784 +0000 UTC m=+528.689257980" lastFinishedPulling="2026-04-16 16:56:34.86977223 +0000 UTC m=+528.951506440" observedRunningTime="2026-04-16 16:56:35.142149688 +0000 UTC m=+529.223883898" watchObservedRunningTime="2026-04-16 16:56:35.143943188 +0000 UTC m=+529.225677401" Apr 16 16:56:58.094006 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:58.093974 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-66488bc844-rqprq"] Apr 16 16:56:58.097156 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:58.097124 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66488bc844-rqprq" Apr 16 16:56:58.108828 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:58.108800 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66488bc844-rqprq"] Apr 16 16:56:58.186095 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:58.186057 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7881d457-3a14-41d2-879b-0e812fe08ef2-console-config\") pod \"console-66488bc844-rqprq\" (UID: \"7881d457-3a14-41d2-879b-0e812fe08ef2\") " pod="openshift-console/console-66488bc844-rqprq" Apr 16 16:56:58.186095 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:58.186097 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7jlf\" (UniqueName: \"kubernetes.io/projected/7881d457-3a14-41d2-879b-0e812fe08ef2-kube-api-access-k7jlf\") pod \"console-66488bc844-rqprq\" (UID: \"7881d457-3a14-41d2-879b-0e812fe08ef2\") " pod="openshift-console/console-66488bc844-rqprq" Apr 16 16:56:58.186336 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:58.186162 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7881d457-3a14-41d2-879b-0e812fe08ef2-console-serving-cert\") pod \"console-66488bc844-rqprq\" (UID: \"7881d457-3a14-41d2-879b-0e812fe08ef2\") " pod="openshift-console/console-66488bc844-rqprq" Apr 16 16:56:58.186336 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:58.186230 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7881d457-3a14-41d2-879b-0e812fe08ef2-service-ca\") pod \"console-66488bc844-rqprq\" (UID: \"7881d457-3a14-41d2-879b-0e812fe08ef2\") " pod="openshift-console/console-66488bc844-rqprq" Apr 16 16:56:58.186336 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:58.186276 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7881d457-3a14-41d2-879b-0e812fe08ef2-console-oauth-config\") pod \"console-66488bc844-rqprq\" (UID: \"7881d457-3a14-41d2-879b-0e812fe08ef2\") " pod="openshift-console/console-66488bc844-rqprq" Apr 16 16:56:58.186336 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:58.186303 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7881d457-3a14-41d2-879b-0e812fe08ef2-trusted-ca-bundle\") pod \"console-66488bc844-rqprq\" (UID: \"7881d457-3a14-41d2-879b-0e812fe08ef2\") " pod="openshift-console/console-66488bc844-rqprq" Apr 16 16:56:58.186474 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:58.186371 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7881d457-3a14-41d2-879b-0e812fe08ef2-oauth-serving-cert\") pod \"console-66488bc844-rqprq\" (UID: \"7881d457-3a14-41d2-879b-0e812fe08ef2\") " pod="openshift-console/console-66488bc844-rqprq" Apr 16 16:56:58.287453 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:58.287421 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7881d457-3a14-41d2-879b-0e812fe08ef2-console-config\") pod \"console-66488bc844-rqprq\" (UID: \"7881d457-3a14-41d2-879b-0e812fe08ef2\") " pod="openshift-console/console-66488bc844-rqprq" Apr 16 16:56:58.287593 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:58.287461 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k7jlf\" (UniqueName: \"kubernetes.io/projected/7881d457-3a14-41d2-879b-0e812fe08ef2-kube-api-access-k7jlf\") pod \"console-66488bc844-rqprq\" (UID: \"7881d457-3a14-41d2-879b-0e812fe08ef2\") " pod="openshift-console/console-66488bc844-rqprq" Apr 16 16:56:58.287593 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:58.287577 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7881d457-3a14-41d2-879b-0e812fe08ef2-console-serving-cert\") pod \"console-66488bc844-rqprq\" (UID: \"7881d457-3a14-41d2-879b-0e812fe08ef2\") " pod="openshift-console/console-66488bc844-rqprq" Apr 16 16:56:58.287708 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:58.287625 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7881d457-3a14-41d2-879b-0e812fe08ef2-service-ca\") pod \"console-66488bc844-rqprq\" (UID: \"7881d457-3a14-41d2-879b-0e812fe08ef2\") " pod="openshift-console/console-66488bc844-rqprq" Apr 16 16:56:58.287708 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:58.287659 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7881d457-3a14-41d2-879b-0e812fe08ef2-console-oauth-config\") pod \"console-66488bc844-rqprq\" (UID: \"7881d457-3a14-41d2-879b-0e812fe08ef2\") " pod="openshift-console/console-66488bc844-rqprq" Apr 16 16:56:58.287708 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:58.287678 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7881d457-3a14-41d2-879b-0e812fe08ef2-trusted-ca-bundle\") pod \"console-66488bc844-rqprq\" (UID: \"7881d457-3a14-41d2-879b-0e812fe08ef2\") " pod="openshift-console/console-66488bc844-rqprq" Apr 16 16:56:58.287844 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:58.287706 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7881d457-3a14-41d2-879b-0e812fe08ef2-oauth-serving-cert\") pod \"console-66488bc844-rqprq\" (UID: \"7881d457-3a14-41d2-879b-0e812fe08ef2\") " pod="openshift-console/console-66488bc844-rqprq" Apr 16 16:56:58.288379 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:58.288350 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7881d457-3a14-41d2-879b-0e812fe08ef2-console-config\") pod \"console-66488bc844-rqprq\" (UID: \"7881d457-3a14-41d2-879b-0e812fe08ef2\") " pod="openshift-console/console-66488bc844-rqprq" Apr 16 16:56:58.288379 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:58.288362 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7881d457-3a14-41d2-879b-0e812fe08ef2-service-ca\") pod \"console-66488bc844-rqprq\" (UID: \"7881d457-3a14-41d2-879b-0e812fe08ef2\") " pod="openshift-console/console-66488bc844-rqprq" Apr 16 16:56:58.288516 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:58.288423 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7881d457-3a14-41d2-879b-0e812fe08ef2-oauth-serving-cert\") pod \"console-66488bc844-rqprq\" (UID: \"7881d457-3a14-41d2-879b-0e812fe08ef2\") " pod="openshift-console/console-66488bc844-rqprq" Apr 16 16:56:58.288631 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:58.288607 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7881d457-3a14-41d2-879b-0e812fe08ef2-trusted-ca-bundle\") pod \"console-66488bc844-rqprq\" (UID: \"7881d457-3a14-41d2-879b-0e812fe08ef2\") " pod="openshift-console/console-66488bc844-rqprq" Apr 16 16:56:58.290005 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:58.289983 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7881d457-3a14-41d2-879b-0e812fe08ef2-console-oauth-config\") pod \"console-66488bc844-rqprq\" (UID: \"7881d457-3a14-41d2-879b-0e812fe08ef2\") " pod="openshift-console/console-66488bc844-rqprq" Apr 16 16:56:58.290084 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:58.290068 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7881d457-3a14-41d2-879b-0e812fe08ef2-console-serving-cert\") pod \"console-66488bc844-rqprq\" (UID: \"7881d457-3a14-41d2-879b-0e812fe08ef2\") " pod="openshift-console/console-66488bc844-rqprq" Apr 16 16:56:58.295819 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:58.295793 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7jlf\" (UniqueName: \"kubernetes.io/projected/7881d457-3a14-41d2-879b-0e812fe08ef2-kube-api-access-k7jlf\") pod \"console-66488bc844-rqprq\" (UID: \"7881d457-3a14-41d2-879b-0e812fe08ef2\") " pod="openshift-console/console-66488bc844-rqprq" Apr 16 16:56:58.407097 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:58.407058 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66488bc844-rqprq" Apr 16 16:56:58.534893 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:58.534867 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66488bc844-rqprq"] Apr 16 16:56:58.536631 ip-10-0-140-49 kubenswrapper[2574]: W0416 16:56:58.536609 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7881d457_3a14_41d2_879b_0e812fe08ef2.slice/crio-74f828fbf195e9598ad881e2be48fd520496ef822b2e32381e226eaaf6cbab84 WatchSource:0}: Error finding container 74f828fbf195e9598ad881e2be48fd520496ef822b2e32381e226eaaf6cbab84: Status 404 returned error can't find the container with id 74f828fbf195e9598ad881e2be48fd520496ef822b2e32381e226eaaf6cbab84 Apr 16 16:56:59.202320 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:59.202281 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66488bc844-rqprq" event={"ID":"7881d457-3a14-41d2-879b-0e812fe08ef2","Type":"ContainerStarted","Data":"b135518b20331c8e4f902e9a2bbe4c9bbddafe7224dcc5e72c4f405acf7ec5f3"} Apr 16 16:56:59.202320 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:59.202322 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66488bc844-rqprq" event={"ID":"7881d457-3a14-41d2-879b-0e812fe08ef2","Type":"ContainerStarted","Data":"74f828fbf195e9598ad881e2be48fd520496ef822b2e32381e226eaaf6cbab84"} Apr 16 16:56:59.220786 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:56:59.220746 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-66488bc844-rqprq" podStartSLOduration=1.220732364 podStartE2EDuration="1.220732364s" podCreationTimestamp="2026-04-16 16:56:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 16:56:59.218472698 +0000 UTC m=+553.300206911" watchObservedRunningTime="2026-04-16 16:56:59.220732364 +0000 UTC m=+553.302466578" Apr 16 16:57:08.407283 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:08.407255 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66488bc844-rqprq" Apr 16 16:57:08.407283 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:08.407293 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66488bc844-rqprq" Apr 16 16:57:08.411834 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:08.411812 2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66488bc844-rqprq" Apr 16 16:57:09.238126 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:09.238102 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66488bc844-rqprq" Apr 16 16:57:09.284301 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:09.284274 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d9c66dbb8-vpcjt"] Apr 16 16:57:34.304102 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:34.304037 2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-6d9c66dbb8-vpcjt" podUID="11eb1f23-be6d-4f81-b5ac-752e3c7fc46c" containerName="console" containerID="cri-o://13b3a06d861b14e6efd1e6c9a9167b9e1b097349a0c7bd65e2c19d16e16e6008" gracePeriod=15 Apr 16 16:57:34.543985 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:34.543965 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6d9c66dbb8-vpcjt_11eb1f23-be6d-4f81-b5ac-752e3c7fc46c/console/0.log" Apr 16 16:57:34.544095 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:34.544023 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d9c66dbb8-vpcjt" Apr 16 16:57:34.662700 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:34.662676 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-console-config\") pod \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\" (UID: \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\") " Apr 16 16:57:34.662862 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:34.662721 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrg4b\" (UniqueName: \"kubernetes.io/projected/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-kube-api-access-nrg4b\") pod \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\" (UID: \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\") " Apr 16 16:57:34.662862 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:34.662751 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-trusted-ca-bundle\") pod \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\" (UID: \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\") " Apr 16 16:57:34.662862 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:34.662776 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-oauth-serving-cert\") pod \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\" (UID: \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\") " Apr 16 16:57:34.662862 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:34.662808 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-console-serving-cert\") pod \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\" (UID: \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\") " Apr 16 16:57:34.662862 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:34.662840 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-service-ca\") pod \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\" (UID: \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\") " Apr 16 16:57:34.663110 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:34.662956 2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-console-oauth-config\") pod \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\" (UID: \"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c\") " Apr 16 16:57:34.663190 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:34.663162 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "11eb1f23-be6d-4f81-b5ac-752e3c7fc46c" (UID: "11eb1f23-be6d-4f81-b5ac-752e3c7fc46c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 16:57:34.663308 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:34.663278 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-service-ca" (OuterVolumeSpecName: "service-ca") pod "11eb1f23-be6d-4f81-b5ac-752e3c7fc46c" (UID: "11eb1f23-be6d-4f81-b5ac-752e3c7fc46c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 16:57:34.663308 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:34.663281 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-console-config" (OuterVolumeSpecName: "console-config") pod "11eb1f23-be6d-4f81-b5ac-752e3c7fc46c" (UID: "11eb1f23-be6d-4f81-b5ac-752e3c7fc46c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 16:57:34.663502 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:34.663475 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "11eb1f23-be6d-4f81-b5ac-752e3c7fc46c" (UID: "11eb1f23-be6d-4f81-b5ac-752e3c7fc46c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 16:57:34.664863 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:34.664842 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-kube-api-access-nrg4b" (OuterVolumeSpecName: "kube-api-access-nrg4b") pod "11eb1f23-be6d-4f81-b5ac-752e3c7fc46c" (UID: "11eb1f23-be6d-4f81-b5ac-752e3c7fc46c"). InnerVolumeSpecName "kube-api-access-nrg4b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 16:57:34.665018 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:34.664999 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "11eb1f23-be6d-4f81-b5ac-752e3c7fc46c" (UID: "11eb1f23-be6d-4f81-b5ac-752e3c7fc46c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 16:57:34.665104 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:34.665081 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "11eb1f23-be6d-4f81-b5ac-752e3c7fc46c" (UID: "11eb1f23-be6d-4f81-b5ac-752e3c7fc46c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 16:57:34.764407 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:34.764382 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-console-config\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:57:34.764407 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:34.764404 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrg4b\" (UniqueName: \"kubernetes.io/projected/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-kube-api-access-nrg4b\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:57:34.764522 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:34.764415 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-trusted-ca-bundle\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:57:34.764522 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:34.764424 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-oauth-serving-cert\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:57:34.764522 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:34.764433 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-console-serving-cert\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:57:34.764522 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:34.764442 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-service-ca\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:57:34.764522 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:34.764450 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c-console-oauth-config\") on node \"ip-10-0-140-49.ec2.internal\" DevicePath \"\"" Apr 16 16:57:35.323665 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:35.323638 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6d9c66dbb8-vpcjt_11eb1f23-be6d-4f81-b5ac-752e3c7fc46c/console/0.log" Apr 16 16:57:35.324051 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:35.323679 2574 generic.go:358] "Generic (PLEG): container finished" podID="11eb1f23-be6d-4f81-b5ac-752e3c7fc46c" containerID="13b3a06d861b14e6efd1e6c9a9167b9e1b097349a0c7bd65e2c19d16e16e6008" exitCode=2 Apr 16 16:57:35.324051 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:35.323757 2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d9c66dbb8-vpcjt" Apr 16 16:57:35.324051 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:35.323773 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d9c66dbb8-vpcjt" event={"ID":"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c","Type":"ContainerDied","Data":"13b3a06d861b14e6efd1e6c9a9167b9e1b097349a0c7bd65e2c19d16e16e6008"} Apr 16 16:57:35.324051 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:35.323819 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d9c66dbb8-vpcjt" event={"ID":"11eb1f23-be6d-4f81-b5ac-752e3c7fc46c","Type":"ContainerDied","Data":"87c70900666d3118c3dc0307ed07d0cccf9bd308c0c856bb7f8ca321796b05cb"} Apr 16 16:57:35.324051 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:35.323842 2574 scope.go:117] "RemoveContainer" containerID="13b3a06d861b14e6efd1e6c9a9167b9e1b097349a0c7bd65e2c19d16e16e6008" Apr 16 16:57:35.332426 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:35.332407 2574 scope.go:117] "RemoveContainer" containerID="13b3a06d861b14e6efd1e6c9a9167b9e1b097349a0c7bd65e2c19d16e16e6008" Apr 16 16:57:35.332661 ip-10-0-140-49 kubenswrapper[2574]: E0416 16:57:35.332644 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13b3a06d861b14e6efd1e6c9a9167b9e1b097349a0c7bd65e2c19d16e16e6008\": container with ID starting with 13b3a06d861b14e6efd1e6c9a9167b9e1b097349a0c7bd65e2c19d16e16e6008 not found: ID does not exist" containerID="13b3a06d861b14e6efd1e6c9a9167b9e1b097349a0c7bd65e2c19d16e16e6008" Apr 16 16:57:35.332727 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:35.332667 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13b3a06d861b14e6efd1e6c9a9167b9e1b097349a0c7bd65e2c19d16e16e6008"} err="failed to get container status \"13b3a06d861b14e6efd1e6c9a9167b9e1b097349a0c7bd65e2c19d16e16e6008\": rpc error: code = NotFound desc = could not find container \"13b3a06d861b14e6efd1e6c9a9167b9e1b097349a0c7bd65e2c19d16e16e6008\": container with ID starting with 13b3a06d861b14e6efd1e6c9a9167b9e1b097349a0c7bd65e2c19d16e16e6008 not found: ID does not exist" Apr 16 16:57:35.346092 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:35.346069 2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d9c66dbb8-vpcjt"] Apr 16 16:57:35.349703 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:35.349680 2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d9c66dbb8-vpcjt"] Apr 16 16:57:36.511916 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:36.511882 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11eb1f23-be6d-4f81-b5ac-752e3c7fc46c" path="/var/lib/kubelet/pods/11eb1f23-be6d-4f81-b5ac-752e3c7fc46c/volumes" Apr 16 16:57:46.408856 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:46.408830 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/1.log" Apr 16 16:57:46.409849 ip-10-0-140-49 kubenswrapper[2574]: I0416 16:57:46.409832 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/1.log" Apr 16 17:01:25.463710 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:01:25.463673 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-lightgbm-predictor-78c8d484d6-hrnjg"] Apr 16 17:01:25.464146 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:01:25.464116 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11eb1f23-be6d-4f81-b5ac-752e3c7fc46c" containerName="console" Apr 16 17:01:25.464146 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:01:25.464129 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="11eb1f23-be6d-4f81-b5ac-752e3c7fc46c" containerName="console" Apr 16 17:01:25.464255 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:01:25.464243 2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="11eb1f23-be6d-4f81-b5ac-752e3c7fc46c" containerName="console" Apr 16 17:01:25.467179 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:01:25.467162 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-lightgbm-predictor-78c8d484d6-hrnjg" Apr 16 17:01:25.469760 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:01:25.469742 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-5ftrw\"" Apr 16 17:01:25.473469 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:01:25.473403 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-lightgbm-predictor-78c8d484d6-hrnjg"] Apr 16 17:01:25.596676 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:01:25.596646 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2f62a498-f7fb-46e3-b63a-9e7cee1d3375-kserve-provision-location\") pod \"isvc-lightgbm-predictor-78c8d484d6-hrnjg\" (UID: \"2f62a498-f7fb-46e3-b63a-9e7cee1d3375\") " pod="kserve-ci-e2e-test/isvc-lightgbm-predictor-78c8d484d6-hrnjg" Apr 16 17:01:25.697115 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:01:25.697089 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2f62a498-f7fb-46e3-b63a-9e7cee1d3375-kserve-provision-location\") pod \"isvc-lightgbm-predictor-78c8d484d6-hrnjg\" (UID: \"2f62a498-f7fb-46e3-b63a-9e7cee1d3375\") " pod="kserve-ci-e2e-test/isvc-lightgbm-predictor-78c8d484d6-hrnjg" Apr 16 17:01:25.697410 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:01:25.697394 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2f62a498-f7fb-46e3-b63a-9e7cee1d3375-kserve-provision-location\") pod \"isvc-lightgbm-predictor-78c8d484d6-hrnjg\" (UID: \"2f62a498-f7fb-46e3-b63a-9e7cee1d3375\") " pod="kserve-ci-e2e-test/isvc-lightgbm-predictor-78c8d484d6-hrnjg" Apr 16 17:01:25.778035 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:01:25.777961 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-lightgbm-predictor-78c8d484d6-hrnjg" Apr 16 17:01:26.105662 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:01:26.105636 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-lightgbm-predictor-78c8d484d6-hrnjg"] Apr 16 17:01:26.107737 ip-10-0-140-49 kubenswrapper[2574]: W0416 17:01:26.107712 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f62a498_f7fb_46e3_b63a_9e7cee1d3375.slice/crio-2f1a9aaa5afa30c046d23d1ab1fa14800c05d536df187c227cfcf45550ee4a1b WatchSource:0}: Error finding container 2f1a9aaa5afa30c046d23d1ab1fa14800c05d536df187c227cfcf45550ee4a1b: Status 404 returned error can't find the container with id 2f1a9aaa5afa30c046d23d1ab1fa14800c05d536df187c227cfcf45550ee4a1b Apr 16 17:01:26.109841 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:01:26.109824 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 16 17:01:27.057292 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:01:27.057253 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-lightgbm-predictor-78c8d484d6-hrnjg" event={"ID":"2f62a498-f7fb-46e3-b63a-9e7cee1d3375","Type":"ContainerStarted","Data":"2f1a9aaa5afa30c046d23d1ab1fa14800c05d536df187c227cfcf45550ee4a1b"} Apr 16 17:01:30.069494 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:01:30.069452 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-lightgbm-predictor-78c8d484d6-hrnjg" event={"ID":"2f62a498-f7fb-46e3-b63a-9e7cee1d3375","Type":"ContainerStarted","Data":"ea609ce3c72be3aa9000893a1c488e2ed052d361db72ec7c3dfedf606c0c39fb"} Apr 16 17:01:33.081148 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:01:33.081109 2574 generic.go:358] "Generic (PLEG): container finished" podID="2f62a498-f7fb-46e3-b63a-9e7cee1d3375" containerID="ea609ce3c72be3aa9000893a1c488e2ed052d361db72ec7c3dfedf606c0c39fb" exitCode=0 Apr 16 17:01:33.081502 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:01:33.081183 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-lightgbm-predictor-78c8d484d6-hrnjg" event={"ID":"2f62a498-f7fb-46e3-b63a-9e7cee1d3375","Type":"ContainerDied","Data":"ea609ce3c72be3aa9000893a1c488e2ed052d361db72ec7c3dfedf606c0c39fb"} Apr 16 17:01:40.107982 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:01:40.107947 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-lightgbm-predictor-78c8d484d6-hrnjg" event={"ID":"2f62a498-f7fb-46e3-b63a-9e7cee1d3375","Type":"ContainerStarted","Data":"f3ce7b16481aba1859b571e3daae8954544f5b1a173604e4e7158a82094c0fc7"} Apr 16 17:01:40.108388 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:01:40.108230 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-lightgbm-predictor-78c8d484d6-hrnjg" Apr 16 17:01:40.109710 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:01:40.109676 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-lightgbm-predictor-78c8d484d6-hrnjg" podUID="2f62a498-f7fb-46e3-b63a-9e7cee1d3375" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused" Apr 16 17:01:40.127035 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:01:40.126991 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-lightgbm-predictor-78c8d484d6-hrnjg" podStartSLOduration=2.031292183 podStartE2EDuration="15.126977866s" podCreationTimestamp="2026-04-16 17:01:25 +0000 UTC" firstStartedPulling="2026-04-16 17:01:26.109969917 +0000 UTC m=+820.191704110" lastFinishedPulling="2026-04-16 17:01:39.205655584 +0000 UTC m=+833.287389793" observedRunningTime="2026-04-16 17:01:40.124959249 +0000 UTC m=+834.206693464" watchObservedRunningTime="2026-04-16 17:01:40.126977866 +0000 UTC m=+834.208712079" Apr 16 17:01:41.111271 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:01:41.111232 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-lightgbm-predictor-78c8d484d6-hrnjg" podUID="2f62a498-f7fb-46e3-b63a-9e7cee1d3375" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused" Apr 16 17:01:51.111633 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:01:51.111594 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-lightgbm-predictor-78c8d484d6-hrnjg" podUID="2f62a498-f7fb-46e3-b63a-9e7cee1d3375" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused" Apr 16 17:02:01.111947 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:02:01.111902 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-lightgbm-predictor-78c8d484d6-hrnjg" podUID="2f62a498-f7fb-46e3-b63a-9e7cee1d3375" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused" Apr 16 17:02:11.111602 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:02:11.111561 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-lightgbm-predictor-78c8d484d6-hrnjg" podUID="2f62a498-f7fb-46e3-b63a-9e7cee1d3375" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused" Apr 16 17:02:21.111462 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:02:21.111423 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-lightgbm-predictor-78c8d484d6-hrnjg" podUID="2f62a498-f7fb-46e3-b63a-9e7cee1d3375" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused" Apr 16 17:02:31.111588 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:02:31.111505 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-lightgbm-predictor-78c8d484d6-hrnjg" podUID="2f62a498-f7fb-46e3-b63a-9e7cee1d3375" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused" Apr 16 17:02:41.112047 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:02:41.112005 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-lightgbm-predictor-78c8d484d6-hrnjg" podUID="2f62a498-f7fb-46e3-b63a-9e7cee1d3375" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused" Apr 16 17:02:46.429953 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:02:46.429927 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/1.log" Apr 16 17:02:46.431955 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:02:46.431931 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/1.log" Apr 16 17:02:46.509580 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:02:46.509545 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-lightgbm-predictor-78c8d484d6-hrnjg" podUID="2f62a498-f7fb-46e3-b63a-9e7cee1d3375" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused" Apr 16 17:02:56.511763 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:02:56.511733 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-lightgbm-predictor-78c8d484d6-hrnjg" Apr 16 17:03:05.717129 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:03:05.717083 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4"] Apr 16 17:03:05.720287 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:03:05.720270 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4" Apr 16 17:03:05.729846 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:03:05.729819 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4"] Apr 16 17:03:05.756802 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:03:05.756774 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/902aabef-a060-4f86-9c71-5913e069001e-kserve-provision-location\") pod \"isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4\" (UID: \"902aabef-a060-4f86-9c71-5913e069001e\") " pod="kserve-ci-e2e-test/isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4" Apr 16 17:03:05.858031 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:03:05.858008 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/902aabef-a060-4f86-9c71-5913e069001e-kserve-provision-location\") pod \"isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4\" (UID: \"902aabef-a060-4f86-9c71-5913e069001e\") " pod="kserve-ci-e2e-test/isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4" Apr 16 17:03:05.858345 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:03:05.858330 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/902aabef-a060-4f86-9c71-5913e069001e-kserve-provision-location\") pod \"isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4\" (UID: \"902aabef-a060-4f86-9c71-5913e069001e\") " pod="kserve-ci-e2e-test/isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4" Apr 16 17:03:06.031340 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:03:06.031286 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4" Apr 16 17:03:06.152238 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:03:06.152215 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4"] Apr 16 17:03:06.154773 ip-10-0-140-49 kubenswrapper[2574]: W0416 17:03:06.154737 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod902aabef_a060_4f86_9c71_5913e069001e.slice/crio-98e20e5db447ad49bccfd0d5e6b69196437bcf45009d3aba92292d8f696fe0ce WatchSource:0}: Error finding container 98e20e5db447ad49bccfd0d5e6b69196437bcf45009d3aba92292d8f696fe0ce: Status 404 returned error can't find the container with id 98e20e5db447ad49bccfd0d5e6b69196437bcf45009d3aba92292d8f696fe0ce Apr 16 17:03:06.384960 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:03:06.384928 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4" event={"ID":"902aabef-a060-4f86-9c71-5913e069001e","Type":"ContainerStarted","Data":"b223900db7b460f45afa9fc931ac8aac40e73a5ba8d1afe32d2a0547c5a12ae2"} Apr 16 17:03:06.385124 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:03:06.384968 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4" event={"ID":"902aabef-a060-4f86-9c71-5913e069001e","Type":"ContainerStarted","Data":"98e20e5db447ad49bccfd0d5e6b69196437bcf45009d3aba92292d8f696fe0ce"} Apr 16 17:03:10.399704 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:03:10.399674 2574 generic.go:358] "Generic (PLEG): container finished" podID="902aabef-a060-4f86-9c71-5913e069001e" containerID="b223900db7b460f45afa9fc931ac8aac40e73a5ba8d1afe32d2a0547c5a12ae2" exitCode=0 Apr 16 17:03:10.400071 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:03:10.399754 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4" event={"ID":"902aabef-a060-4f86-9c71-5913e069001e","Type":"ContainerDied","Data":"b223900db7b460f45afa9fc931ac8aac40e73a5ba8d1afe32d2a0547c5a12ae2"} Apr 16 17:03:11.405301 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:03:11.405268 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4" event={"ID":"902aabef-a060-4f86-9c71-5913e069001e","Type":"ContainerStarted","Data":"3c7d1cc81e448e065d0d0f58b8539cdd9e6dd5a6452a8519956a5f34b5e0a1e2"} Apr 16 17:03:11.405719 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:03:11.405538 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4" Apr 16 17:03:11.406889 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:03:11.406863 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4" podUID="902aabef-a060-4f86-9c71-5913e069001e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused" Apr 16 17:03:11.422458 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:03:11.422421 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4" podStartSLOduration=6.422409252 podStartE2EDuration="6.422409252s" podCreationTimestamp="2026-04-16 17:03:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 17:03:11.419871382 +0000 UTC m=+925.501605607" watchObservedRunningTime="2026-04-16 17:03:11.422409252 +0000 UTC m=+925.504143466" Apr 16 17:03:12.409250 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:03:12.409196 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4" podUID="902aabef-a060-4f86-9c71-5913e069001e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused" Apr 16 17:03:22.409561 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:03:22.409522 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4" podUID="902aabef-a060-4f86-9c71-5913e069001e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused" Apr 16 17:03:32.409334 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:03:32.409286 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4" podUID="902aabef-a060-4f86-9c71-5913e069001e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused" Apr 16 17:03:42.409448 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:03:42.409407 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4" podUID="902aabef-a060-4f86-9c71-5913e069001e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused" Apr 16 17:03:52.409886 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:03:52.409843 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4" podUID="902aabef-a060-4f86-9c71-5913e069001e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused" Apr 16 17:04:02.409710 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:04:02.409629 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4" podUID="902aabef-a060-4f86-9c71-5913e069001e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused" Apr 16 17:04:12.409527 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:04:12.409487 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4" podUID="902aabef-a060-4f86-9c71-5913e069001e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused" Apr 16 17:04:21.508941 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:04:21.508911 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4" Apr 16 17:04:25.890454 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:04:25.890426 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v"] Apr 16 17:04:25.893792 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:04:25.893763 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v" Apr 16 17:04:25.901507 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:04:25.901461 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v"] Apr 16 17:04:25.985014 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:04:25.984984 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/34163a51-b7e0-40f5-aa47-7a39addab1a4-kserve-provision-location\") pod \"isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v\" (UID: \"34163a51-b7e0-40f5-aa47-7a39addab1a4\") " pod="kserve-ci-e2e-test/isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v" Apr 16 17:04:26.085830 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:04:26.085797 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/34163a51-b7e0-40f5-aa47-7a39addab1a4-kserve-provision-location\") pod \"isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v\" (UID: \"34163a51-b7e0-40f5-aa47-7a39addab1a4\") " pod="kserve-ci-e2e-test/isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v" Apr 16 17:04:26.086163 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:04:26.086145 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/34163a51-b7e0-40f5-aa47-7a39addab1a4-kserve-provision-location\") pod \"isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v\" (UID: \"34163a51-b7e0-40f5-aa47-7a39addab1a4\") " pod="kserve-ci-e2e-test/isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v" Apr 16 17:04:26.205368 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:04:26.205293 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v" Apr 16 17:04:26.336185 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:04:26.336160 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v"] Apr 16 17:04:26.338153 ip-10-0-140-49 kubenswrapper[2574]: W0416 17:04:26.338127 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34163a51_b7e0_40f5_aa47_7a39addab1a4.slice/crio-a9ca358180dc2615a2e6c0b3ee1eb69a82eebaa71a378ea7fc3e1f6b11a69213 WatchSource:0}: Error finding container a9ca358180dc2615a2e6c0b3ee1eb69a82eebaa71a378ea7fc3e1f6b11a69213: Status 404 returned error can't find the container with id a9ca358180dc2615a2e6c0b3ee1eb69a82eebaa71a378ea7fc3e1f6b11a69213 Apr 16 17:04:26.649651 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:04:26.649614 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v" event={"ID":"34163a51-b7e0-40f5-aa47-7a39addab1a4","Type":"ContainerStarted","Data":"71102bfb5afb96754ec8708b8bc13ceed3767fe98ecb09566ca2d13c9afdc270"} Apr 16 17:04:26.649651 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:04:26.649656 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v" event={"ID":"34163a51-b7e0-40f5-aa47-7a39addab1a4","Type":"ContainerStarted","Data":"a9ca358180dc2615a2e6c0b3ee1eb69a82eebaa71a378ea7fc3e1f6b11a69213"} Apr 16 17:04:30.662474 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:04:30.662442 2574 generic.go:358] "Generic (PLEG): container finished" podID="34163a51-b7e0-40f5-aa47-7a39addab1a4" containerID="71102bfb5afb96754ec8708b8bc13ceed3767fe98ecb09566ca2d13c9afdc270" exitCode=0 Apr 16 17:04:30.662885 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:04:30.662489 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v" event={"ID":"34163a51-b7e0-40f5-aa47-7a39addab1a4","Type":"ContainerDied","Data":"71102bfb5afb96754ec8708b8bc13ceed3767fe98ecb09566ca2d13c9afdc270"} Apr 16 17:06:47.156007 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:06:47.155974 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v" event={"ID":"34163a51-b7e0-40f5-aa47-7a39addab1a4","Type":"ContainerStarted","Data":"11859726fa5b978abc497728dd5bde71918df303b6fda12d746249326c30f7be"} Apr 16 17:06:47.156402 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:06:47.156133 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v" Apr 16 17:06:47.180833 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:06:47.180786 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v" podStartSLOduration=6.493696021 podStartE2EDuration="2m22.180773822s" podCreationTimestamp="2026-04-16 17:04:25 +0000 UTC" firstStartedPulling="2026-04-16 17:04:30.663516548 +0000 UTC m=+1004.745250740" lastFinishedPulling="2026-04-16 17:06:46.350594345 +0000 UTC m=+1140.432328541" observedRunningTime="2026-04-16 17:06:47.179090182 +0000 UTC m=+1141.260824418" watchObservedRunningTime="2026-04-16 17:06:47.180773822 +0000 UTC m=+1141.262508036" Apr 16 17:07:18.164790 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:18.164762 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v" Apr 16 17:07:26.183377 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:26.183339 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp"] Apr 16 17:07:26.208490 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:26.208457 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp"] Apr 16 17:07:26.208643 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:26.208554 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp" Apr 16 17:07:26.312571 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:26.312543 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/312dd04f-11ce-4a90-afea-d471128c9fd3-kserve-provision-location\") pod \"isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp\" (UID: \"312dd04f-11ce-4a90-afea-d471128c9fd3\") " pod="kserve-ci-e2e-test/isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp" Apr 16 17:07:26.413689 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:26.413657 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/312dd04f-11ce-4a90-afea-d471128c9fd3-kserve-provision-location\") pod \"isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp\" (UID: \"312dd04f-11ce-4a90-afea-d471128c9fd3\") " pod="kserve-ci-e2e-test/isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp" Apr 16 17:07:26.413989 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:26.413971 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/312dd04f-11ce-4a90-afea-d471128c9fd3-kserve-provision-location\") pod \"isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp\" (UID: \"312dd04f-11ce-4a90-afea-d471128c9fd3\") " pod="kserve-ci-e2e-test/isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp" Apr 16 17:07:26.519244 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:26.519175 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp" Apr 16 17:07:26.635587 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:26.635560 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp"] Apr 16 17:07:26.637606 ip-10-0-140-49 kubenswrapper[2574]: W0416 17:07:26.637583 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod312dd04f_11ce_4a90_afea_d471128c9fd3.slice/crio-56f762d84abce626fa4a8e35dc85247000ae27b9f32b4cd45dddeaa51d23b510 WatchSource:0}: Error finding container 56f762d84abce626fa4a8e35dc85247000ae27b9f32b4cd45dddeaa51d23b510: Status 404 returned error can't find the container with id 56f762d84abce626fa4a8e35dc85247000ae27b9f32b4cd45dddeaa51d23b510 Apr 16 17:07:26.639658 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:26.639642 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 16 17:07:27.290064 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:27.290033 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp" event={"ID":"312dd04f-11ce-4a90-afea-d471128c9fd3","Type":"ContainerStarted","Data":"f4200aef9fd29c99c3a089abca4b3dce072ed17a10e8367fe699377fc5140372"} Apr 16 17:07:27.290064 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:27.290067 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp" event={"ID":"312dd04f-11ce-4a90-afea-d471128c9fd3","Type":"ContainerStarted","Data":"56f762d84abce626fa4a8e35dc85247000ae27b9f32b4cd45dddeaa51d23b510"} Apr 16 17:07:30.300331 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:30.300299 2574 generic.go:358] "Generic (PLEG): container finished" podID="312dd04f-11ce-4a90-afea-d471128c9fd3" containerID="f4200aef9fd29c99c3a089abca4b3dce072ed17a10e8367fe699377fc5140372" exitCode=0 Apr 16 17:07:30.300759 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:30.300375 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp" event={"ID":"312dd04f-11ce-4a90-afea-d471128c9fd3","Type":"ContainerDied","Data":"f4200aef9fd29c99c3a089abca4b3dce072ed17a10e8367fe699377fc5140372"} Apr 16 17:07:31.305950 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:31.305919 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp" event={"ID":"312dd04f-11ce-4a90-afea-d471128c9fd3","Type":"ContainerStarted","Data":"1f62b62fab845bc6c8d8cd3ea03cede53185f2a2f8a58df2d06814228a057c19"} Apr 16 17:07:31.306379 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:31.306223 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp" Apr 16 17:07:31.307521 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:31.307493 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp" podUID="312dd04f-11ce-4a90-afea-d471128c9fd3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused" Apr 16 17:07:31.322661 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:31.322619 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp" podStartSLOduration=5.32260621 podStartE2EDuration="5.32260621s" podCreationTimestamp="2026-04-16 17:07:26 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 17:07:31.321611031 +0000 UTC m=+1185.403345246" watchObservedRunningTime="2026-04-16 17:07:31.32260621 +0000 UTC m=+1185.404340424" Apr 16 17:07:32.310080 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:32.310043 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp" podUID="312dd04f-11ce-4a90-afea-d471128c9fd3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused" Apr 16 17:07:42.311247 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:42.311217 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp" Apr 16 17:07:46.395653 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:46.395623 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55"] Apr 16 17:07:46.399228 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:46.399196 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55" Apr 16 17:07:46.406482 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:46.406459 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55"] Apr 16 17:07:46.458612 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:46.458574 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/1.log" Apr 16 17:07:46.461709 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:46.461689 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/1.log" Apr 16 17:07:46.478450 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:46.478426 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/21c509c7-82b6-4e6e-af25-b691fc24a898-kserve-provision-location\") pod \"isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55\" (UID: \"21c509c7-82b6-4e6e-af25-b691fc24a898\") " pod="kserve-ci-e2e-test/isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55" Apr 16 17:07:46.579785 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:46.579752 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/21c509c7-82b6-4e6e-af25-b691fc24a898-kserve-provision-location\") pod \"isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55\" (UID: \"21c509c7-82b6-4e6e-af25-b691fc24a898\") " pod="kserve-ci-e2e-test/isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55" Apr 16 17:07:46.580074 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:46.580058 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/21c509c7-82b6-4e6e-af25-b691fc24a898-kserve-provision-location\") pod \"isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55\" (UID: \"21c509c7-82b6-4e6e-af25-b691fc24a898\") " pod="kserve-ci-e2e-test/isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55" Apr 16 17:07:46.710554 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:46.710478 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55" Apr 16 17:07:46.829432 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:46.829408 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55"] Apr 16 17:07:46.831884 ip-10-0-140-49 kubenswrapper[2574]: W0416 17:07:46.831855 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21c509c7_82b6_4e6e_af25_b691fc24a898.slice/crio-fed230e86bd8a4b8f4c10ad8c14de7b549488e31fe1c7fd72d2153f0658f2190 WatchSource:0}: Error finding container fed230e86bd8a4b8f4c10ad8c14de7b549488e31fe1c7fd72d2153f0658f2190: Status 404 returned error can't find the container with id fed230e86bd8a4b8f4c10ad8c14de7b549488e31fe1c7fd72d2153f0658f2190 Apr 16 17:07:47.361993 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:47.361959 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55" event={"ID":"21c509c7-82b6-4e6e-af25-b691fc24a898","Type":"ContainerStarted","Data":"0341941be7f3f10a379ac5ae76b034e9b0b4284585b7af15516a645307e4e87e"} Apr 16 17:07:47.361993 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:47.361991 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55" event={"ID":"21c509c7-82b6-4e6e-af25-b691fc24a898","Type":"ContainerStarted","Data":"fed230e86bd8a4b8f4c10ad8c14de7b549488e31fe1c7fd72d2153f0658f2190"} Apr 16 17:07:51.380718 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:51.380684 2574 generic.go:358] "Generic (PLEG): container finished" podID="21c509c7-82b6-4e6e-af25-b691fc24a898" containerID="0341941be7f3f10a379ac5ae76b034e9b0b4284585b7af15516a645307e4e87e" exitCode=0 Apr 16 17:07:51.381120 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:51.380745 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55" event={"ID":"21c509c7-82b6-4e6e-af25-b691fc24a898","Type":"ContainerDied","Data":"0341941be7f3f10a379ac5ae76b034e9b0b4284585b7af15516a645307e4e87e"} Apr 16 17:07:52.386631 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:52.386600 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55" event={"ID":"21c509c7-82b6-4e6e-af25-b691fc24a898","Type":"ContainerStarted","Data":"f0748ac5fb1770c7560d5f49b7e119ddb5f99267eab367cf0e3f3dbf66f4f3e6"} Apr 16 17:07:52.386985 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:52.386813 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55" Apr 16 17:07:52.404655 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:07:52.404607 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55" podStartSLOduration=6.404592041 podStartE2EDuration="6.404592041s" podCreationTimestamp="2026-04-16 17:07:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 17:07:52.403318078 +0000 UTC m=+1206.485052314" watchObservedRunningTime="2026-04-16 17:07:52.404592041 +0000 UTC m=+1206.486326254" Apr 16 17:08:23.394811 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:08:23.394771 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55" Apr 16 17:12:38.608236 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:12:38.608187 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-pmml-predictor-5584ffd8c9-zvfhk"] Apr 16 17:12:38.611595 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:12:38.611577 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-pmml-predictor-5584ffd8c9-zvfhk" Apr 16 17:12:38.618745 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:12:38.618726 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-pmml-predictor-5584ffd8c9-zvfhk"] Apr 16 17:12:38.712299 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:12:38.712272 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b09b7f8f-a1fa-4062-8de9-a929f3af5713-kserve-provision-location\") pod \"isvc-pmml-predictor-5584ffd8c9-zvfhk\" (UID: \"b09b7f8f-a1fa-4062-8de9-a929f3af5713\") " pod="kserve-ci-e2e-test/isvc-pmml-predictor-5584ffd8c9-zvfhk" Apr 16 17:12:38.812863 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:12:38.812838 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b09b7f8f-a1fa-4062-8de9-a929f3af5713-kserve-provision-location\") pod \"isvc-pmml-predictor-5584ffd8c9-zvfhk\" (UID: \"b09b7f8f-a1fa-4062-8de9-a929f3af5713\") " pod="kserve-ci-e2e-test/isvc-pmml-predictor-5584ffd8c9-zvfhk" Apr 16 17:12:38.813161 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:12:38.813132 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b09b7f8f-a1fa-4062-8de9-a929f3af5713-kserve-provision-location\") pod \"isvc-pmml-predictor-5584ffd8c9-zvfhk\" (UID: \"b09b7f8f-a1fa-4062-8de9-a929f3af5713\") " pod="kserve-ci-e2e-test/isvc-pmml-predictor-5584ffd8c9-zvfhk" Apr 16 17:12:38.924409 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:12:38.924333 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-pmml-predictor-5584ffd8c9-zvfhk" Apr 16 17:12:39.036958 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:12:39.036933 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-pmml-predictor-5584ffd8c9-zvfhk"] Apr 16 17:12:39.038665 ip-10-0-140-49 kubenswrapper[2574]: W0416 17:12:39.038640 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb09b7f8f_a1fa_4062_8de9_a929f3af5713.slice/crio-b447d3874786eab124600a3fdaf2365d4fbaf839e713e8a87832f53daa67002e WatchSource:0}: Error finding container b447d3874786eab124600a3fdaf2365d4fbaf839e713e8a87832f53daa67002e: Status 404 returned error can't find the container with id b447d3874786eab124600a3fdaf2365d4fbaf839e713e8a87832f53daa67002e Apr 16 17:12:39.040530 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:12:39.040512 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 16 17:12:39.334527 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:12:39.334441 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-pmml-predictor-5584ffd8c9-zvfhk" event={"ID":"b09b7f8f-a1fa-4062-8de9-a929f3af5713","Type":"ContainerStarted","Data":"d28cb2a89666306d18b8ca9bc368b41f3468e25fadae7371c58d1f3c6e0fcb3c"} Apr 16 17:12:39.334527 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:12:39.334485 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-pmml-predictor-5584ffd8c9-zvfhk" event={"ID":"b09b7f8f-a1fa-4062-8de9-a929f3af5713","Type":"ContainerStarted","Data":"b447d3874786eab124600a3fdaf2365d4fbaf839e713e8a87832f53daa67002e"} Apr 16 17:12:43.349248 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:12:43.349198 2574 generic.go:358] "Generic (PLEG): container finished" podID="b09b7f8f-a1fa-4062-8de9-a929f3af5713" containerID="d28cb2a89666306d18b8ca9bc368b41f3468e25fadae7371c58d1f3c6e0fcb3c" exitCode=0 Apr 16 17:12:43.349704 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:12:43.349274 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-pmml-predictor-5584ffd8c9-zvfhk" event={"ID":"b09b7f8f-a1fa-4062-8de9-a929f3af5713","Type":"ContainerDied","Data":"d28cb2a89666306d18b8ca9bc368b41f3468e25fadae7371c58d1f3c6e0fcb3c"} Apr 16 17:12:50.248436 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:12:50.248410 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/1.log" Apr 16 17:12:50.248837 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:12:50.248617 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/1.log" Apr 16 17:12:50.380928 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:12:50.380904 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-pmml-predictor-5584ffd8c9-zvfhk" event={"ID":"b09b7f8f-a1fa-4062-8de9-a929f3af5713","Type":"ContainerStarted","Data":"6f7edeab45e83447472b19dcb7652c1f88240b059d1ff3ad54ec717df4a85f45"} Apr 16 17:12:50.381193 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:12:50.381171 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-pmml-predictor-5584ffd8c9-zvfhk" Apr 16 17:12:50.382264 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:12:50.382240 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-pmml-predictor-5584ffd8c9-zvfhk" podUID="b09b7f8f-a1fa-4062-8de9-a929f3af5713" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused" Apr 16 17:12:51.385319 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:12:51.385283 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-pmml-predictor-5584ffd8c9-zvfhk" podUID="b09b7f8f-a1fa-4062-8de9-a929f3af5713" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused" Apr 16 17:13:01.385777 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:13:01.385734 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-pmml-predictor-5584ffd8c9-zvfhk" podUID="b09b7f8f-a1fa-4062-8de9-a929f3af5713" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused" Apr 16 17:13:11.385770 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:13:11.385729 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-pmml-predictor-5584ffd8c9-zvfhk" podUID="b09b7f8f-a1fa-4062-8de9-a929f3af5713" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused" Apr 16 17:13:21.385217 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:13:21.385174 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-pmml-predictor-5584ffd8c9-zvfhk" podUID="b09b7f8f-a1fa-4062-8de9-a929f3af5713" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused" Apr 16 17:13:31.386035 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:13:31.385992 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-pmml-predictor-5584ffd8c9-zvfhk" podUID="b09b7f8f-a1fa-4062-8de9-a929f3af5713" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused" Apr 16 17:13:41.386157 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:13:41.386117 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-pmml-predictor-5584ffd8c9-zvfhk" podUID="b09b7f8f-a1fa-4062-8de9-a929f3af5713" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused" Apr 16 17:13:51.385535 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:13:51.385489 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-pmml-predictor-5584ffd8c9-zvfhk" podUID="b09b7f8f-a1fa-4062-8de9-a929f3af5713" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused" Apr 16 17:14:01.386426 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:14:01.386388 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-pmml-predictor-5584ffd8c9-zvfhk" Apr 16 17:14:01.402141 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:14:01.402093 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-pmml-predictor-5584ffd8c9-zvfhk" podStartSLOduration=76.442495932 podStartE2EDuration="1m23.402076656s" podCreationTimestamp="2026-04-16 17:12:38 +0000 UTC" firstStartedPulling="2026-04-16 17:12:43.350454384 +0000 UTC m=+1497.432188577" lastFinishedPulling="2026-04-16 17:12:50.310035101 +0000 UTC m=+1504.391769301" observedRunningTime="2026-04-16 17:12:50.395537376 +0000 UTC m=+1504.477271591" watchObservedRunningTime="2026-04-16 17:14:01.402076656 +0000 UTC m=+1575.483810869" Apr 16 17:14:08.846255 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:14:08.846225 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-pmml-runtime-predictor-7576f6b69f-7dqpk"] Apr 16 17:14:08.849674 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:14:08.849654 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-pmml-runtime-predictor-7576f6b69f-7dqpk" Apr 16 17:14:08.857284 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:14:08.856911 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-pmml-runtime-predictor-7576f6b69f-7dqpk"] Apr 16 17:14:08.920110 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:14:08.920080 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/15fdc27b-a100-4955-96f3-a033027e5c44-kserve-provision-location\") pod \"isvc-pmml-runtime-predictor-7576f6b69f-7dqpk\" (UID: \"15fdc27b-a100-4955-96f3-a033027e5c44\") " pod="kserve-ci-e2e-test/isvc-pmml-runtime-predictor-7576f6b69f-7dqpk" Apr 16 17:14:09.020669 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:14:09.020641 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/15fdc27b-a100-4955-96f3-a033027e5c44-kserve-provision-location\") pod \"isvc-pmml-runtime-predictor-7576f6b69f-7dqpk\" (UID: \"15fdc27b-a100-4955-96f3-a033027e5c44\") " pod="kserve-ci-e2e-test/isvc-pmml-runtime-predictor-7576f6b69f-7dqpk" Apr 16 17:14:09.021033 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:14:09.021015 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/15fdc27b-a100-4955-96f3-a033027e5c44-kserve-provision-location\") pod \"isvc-pmml-runtime-predictor-7576f6b69f-7dqpk\" (UID: \"15fdc27b-a100-4955-96f3-a033027e5c44\") " pod="kserve-ci-e2e-test/isvc-pmml-runtime-predictor-7576f6b69f-7dqpk" Apr 16 17:14:09.160983 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:14:09.160951 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-pmml-runtime-predictor-7576f6b69f-7dqpk" Apr 16 17:14:09.484161 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:14:09.484135 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-pmml-runtime-predictor-7576f6b69f-7dqpk"] Apr 16 17:14:09.486491 ip-10-0-140-49 kubenswrapper[2574]: W0416 17:14:09.486464 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15fdc27b_a100_4955_96f3_a033027e5c44.slice/crio-4e429a9cd895c7bbbc2c5f1496a3ac2ca3a2b4125b16e950066e78011de8f458 WatchSource:0}: Error finding container 4e429a9cd895c7bbbc2c5f1496a3ac2ca3a2b4125b16e950066e78011de8f458: Status 404 returned error can't find the container with id 4e429a9cd895c7bbbc2c5f1496a3ac2ca3a2b4125b16e950066e78011de8f458 Apr 16 17:14:09.634716 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:14:09.634689 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-pmml-runtime-predictor-7576f6b69f-7dqpk" event={"ID":"15fdc27b-a100-4955-96f3-a033027e5c44","Type":"ContainerStarted","Data":"c4f15553a0cf646e5ecc21557e3486eaf8c22aa5bf9dcea1fa6d473bb2211623"} Apr 16 17:14:09.634716 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:14:09.634721 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-pmml-runtime-predictor-7576f6b69f-7dqpk" event={"ID":"15fdc27b-a100-4955-96f3-a033027e5c44","Type":"ContainerStarted","Data":"4e429a9cd895c7bbbc2c5f1496a3ac2ca3a2b4125b16e950066e78011de8f458"} Apr 16 17:14:13.649418 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:14:13.649384 2574 generic.go:358] "Generic (PLEG): container finished" podID="15fdc27b-a100-4955-96f3-a033027e5c44" containerID="c4f15553a0cf646e5ecc21557e3486eaf8c22aa5bf9dcea1fa6d473bb2211623" exitCode=0 Apr 16 17:14:13.649793 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:14:13.649459 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-pmml-runtime-predictor-7576f6b69f-7dqpk" event={"ID":"15fdc27b-a100-4955-96f3-a033027e5c44","Type":"ContainerDied","Data":"c4f15553a0cf646e5ecc21557e3486eaf8c22aa5bf9dcea1fa6d473bb2211623"} Apr 16 17:14:14.655011 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:14:14.654977 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-pmml-runtime-predictor-7576f6b69f-7dqpk" event={"ID":"15fdc27b-a100-4955-96f3-a033027e5c44","Type":"ContainerStarted","Data":"ed8e6b8aff552af8bef290fd4ede070c272ffaa31f3429ae12c6c2e2e0d6a3cb"} Apr 16 17:14:14.655403 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:14:14.655256 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-pmml-runtime-predictor-7576f6b69f-7dqpk" Apr 16 17:14:14.656571 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:14:14.656543 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-pmml-runtime-predictor-7576f6b69f-7dqpk" podUID="15fdc27b-a100-4955-96f3-a033027e5c44" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused" Apr 16 17:14:14.671252 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:14:14.671193 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-pmml-runtime-predictor-7576f6b69f-7dqpk" podStartSLOduration=6.671182568 podStartE2EDuration="6.671182568s" podCreationTimestamp="2026-04-16 17:14:08 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 17:14:14.669694029 +0000 UTC m=+1588.751428237" watchObservedRunningTime="2026-04-16 17:14:14.671182568 +0000 UTC m=+1588.752916783" Apr 16 17:14:15.659089 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:14:15.659047 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-pmml-runtime-predictor-7576f6b69f-7dqpk" podUID="15fdc27b-a100-4955-96f3-a033027e5c44" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused" Apr 16 17:14:25.659985 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:14:25.659941 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-pmml-runtime-predictor-7576f6b69f-7dqpk" podUID="15fdc27b-a100-4955-96f3-a033027e5c44" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused" Apr 16 17:14:35.659256 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:14:35.659197 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-pmml-runtime-predictor-7576f6b69f-7dqpk" podUID="15fdc27b-a100-4955-96f3-a033027e5c44" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused" Apr 16 17:14:45.659686 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:14:45.659649 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-pmml-runtime-predictor-7576f6b69f-7dqpk" podUID="15fdc27b-a100-4955-96f3-a033027e5c44" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused" Apr 16 17:14:55.659560 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:14:55.659520 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-pmml-runtime-predictor-7576f6b69f-7dqpk" podUID="15fdc27b-a100-4955-96f3-a033027e5c44" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused" Apr 16 17:15:05.659747 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:15:05.659710 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-pmml-runtime-predictor-7576f6b69f-7dqpk" podUID="15fdc27b-a100-4955-96f3-a033027e5c44" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused" Apr 16 17:15:15.659047 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:15:15.659006 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-pmml-runtime-predictor-7576f6b69f-7dqpk" podUID="15fdc27b-a100-4955-96f3-a033027e5c44" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused" Apr 16 17:15:25.660173 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:15:25.660144 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-pmml-runtime-predictor-7576f6b69f-7dqpk" Apr 16 17:17:50.273859 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:17:50.273834 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/1.log" Apr 16 17:17:50.274710 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:17:50.274691 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/1.log" Apr 16 17:18:50.787855 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:18:50.787821 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-predictive-sklearn-predictor-85bccb8945-5ppfc"] Apr 16 17:18:50.791543 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:18:50.791526 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-predictive-sklearn-predictor-85bccb8945-5ppfc" Apr 16 17:18:50.797400 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:18:50.797360 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-predictive-sklearn-predictor-85bccb8945-5ppfc"] Apr 16 17:18:50.905616 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:18:50.905585 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f2948b8d-7d4a-4c39-83e2-09cf271d48d1-kserve-provision-location\") pod \"isvc-predictive-sklearn-predictor-85bccb8945-5ppfc\" (UID: \"f2948b8d-7d4a-4c39-83e2-09cf271d48d1\") " pod="kserve-ci-e2e-test/isvc-predictive-sklearn-predictor-85bccb8945-5ppfc" Apr 16 17:18:51.006413 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:18:51.006384 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f2948b8d-7d4a-4c39-83e2-09cf271d48d1-kserve-provision-location\") pod \"isvc-predictive-sklearn-predictor-85bccb8945-5ppfc\" (UID: \"f2948b8d-7d4a-4c39-83e2-09cf271d48d1\") " pod="kserve-ci-e2e-test/isvc-predictive-sklearn-predictor-85bccb8945-5ppfc" Apr 16 17:18:51.006761 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:18:51.006740 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f2948b8d-7d4a-4c39-83e2-09cf271d48d1-kserve-provision-location\") pod \"isvc-predictive-sklearn-predictor-85bccb8945-5ppfc\" (UID: \"f2948b8d-7d4a-4c39-83e2-09cf271d48d1\") " pod="kserve-ci-e2e-test/isvc-predictive-sklearn-predictor-85bccb8945-5ppfc" Apr 16 17:18:51.103090 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:18:51.103067 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-predictive-sklearn-predictor-85bccb8945-5ppfc" Apr 16 17:18:51.220636 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:18:51.220604 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-predictive-sklearn-predictor-85bccb8945-5ppfc"] Apr 16 17:18:51.224083 ip-10-0-140-49 kubenswrapper[2574]: W0416 17:18:51.224055 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2948b8d_7d4a_4c39_83e2_09cf271d48d1.slice/crio-c2bddf47a1753b3b38af8e56ebc9ae6d24fb53b6cbc10c2dcd121ace25ed3f66 WatchSource:0}: Error finding container c2bddf47a1753b3b38af8e56ebc9ae6d24fb53b6cbc10c2dcd121ace25ed3f66: Status 404 returned error can't find the container with id c2bddf47a1753b3b38af8e56ebc9ae6d24fb53b6cbc10c2dcd121ace25ed3f66 Apr 16 17:18:51.226087 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:18:51.226065 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 16 17:18:51.597164 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:18:51.597080 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-predictive-sklearn-predictor-85bccb8945-5ppfc" event={"ID":"f2948b8d-7d4a-4c39-83e2-09cf271d48d1","Type":"ContainerStarted","Data":"0d95fc0c6107ec5f3af0f7d9c132957247d263f15d92b610f57e9a9d8a766915"} Apr 16 17:18:51.597164 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:18:51.597118 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-predictive-sklearn-predictor-85bccb8945-5ppfc" event={"ID":"f2948b8d-7d4a-4c39-83e2-09cf271d48d1","Type":"ContainerStarted","Data":"c2bddf47a1753b3b38af8e56ebc9ae6d24fb53b6cbc10c2dcd121ace25ed3f66"} Apr 16 17:18:55.616816 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:18:55.616780 2574 generic.go:358] "Generic (PLEG): container finished" podID="f2948b8d-7d4a-4c39-83e2-09cf271d48d1" containerID="0d95fc0c6107ec5f3af0f7d9c132957247d263f15d92b610f57e9a9d8a766915" exitCode=0 Apr 16 17:18:55.617307 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:18:55.616838 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-predictive-sklearn-predictor-85bccb8945-5ppfc" event={"ID":"f2948b8d-7d4a-4c39-83e2-09cf271d48d1","Type":"ContainerDied","Data":"0d95fc0c6107ec5f3af0f7d9c132957247d263f15d92b610f57e9a9d8a766915"} Apr 16 17:19:16.702110 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:19:16.702082 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-predictive-sklearn-predictor-85bccb8945-5ppfc" event={"ID":"f2948b8d-7d4a-4c39-83e2-09cf271d48d1","Type":"ContainerStarted","Data":"c2e21296198a7ed9293b26242361a06a6e1dcd2063a0f7377be2ed622a5420f2"} Apr 16 17:19:16.702491 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:19:16.702384 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-predictive-sklearn-predictor-85bccb8945-5ppfc" Apr 16 17:19:16.703844 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:19:16.703817 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-predictive-sklearn-predictor-85bccb8945-5ppfc" podUID="f2948b8d-7d4a-4c39-83e2-09cf271d48d1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused" Apr 16 17:19:16.720262 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:19:16.720194 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-predictive-sklearn-predictor-85bccb8945-5ppfc" podStartSLOduration=5.864274045 podStartE2EDuration="26.720182205s" podCreationTimestamp="2026-04-16 17:18:50 +0000 UTC" firstStartedPulling="2026-04-16 17:18:55.618082711 +0000 UTC m=+1869.699816903" lastFinishedPulling="2026-04-16 17:19:16.473990867 +0000 UTC m=+1890.555725063" observedRunningTime="2026-04-16 17:19:16.71787211 +0000 UTC m=+1890.799606320" watchObservedRunningTime="2026-04-16 17:19:16.720182205 +0000 UTC m=+1890.801916463" Apr 16 17:19:17.705751 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:19:17.705714 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-predictive-sklearn-predictor-85bccb8945-5ppfc" podUID="f2948b8d-7d4a-4c39-83e2-09cf271d48d1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused" Apr 16 17:19:27.706678 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:19:27.706636 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-predictive-sklearn-predictor-85bccb8945-5ppfc" podUID="f2948b8d-7d4a-4c39-83e2-09cf271d48d1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused" Apr 16 17:19:37.706770 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:19:37.706728 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-predictive-sklearn-predictor-85bccb8945-5ppfc" podUID="f2948b8d-7d4a-4c39-83e2-09cf271d48d1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused" Apr 16 17:19:47.706316 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:19:47.706273 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-predictive-sklearn-predictor-85bccb8945-5ppfc" podUID="f2948b8d-7d4a-4c39-83e2-09cf271d48d1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused" Apr 16 17:19:57.706448 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:19:57.706407 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-predictive-sklearn-predictor-85bccb8945-5ppfc" podUID="f2948b8d-7d4a-4c39-83e2-09cf271d48d1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused" Apr 16 17:20:07.706077 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:20:07.706037 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-predictive-sklearn-predictor-85bccb8945-5ppfc" podUID="f2948b8d-7d4a-4c39-83e2-09cf271d48d1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused" Apr 16 17:20:17.706422 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:20:17.706380 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-predictive-sklearn-predictor-85bccb8945-5ppfc" podUID="f2948b8d-7d4a-4c39-83e2-09cf271d48d1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused" Apr 16 17:20:27.707186 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:20:27.707114 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-predictive-sklearn-predictor-85bccb8945-5ppfc" Apr 16 17:22:21.370085 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:22:21.370047 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-predictive-lightgbm-predictor-669896799c-54xhl"] Apr 16 17:22:21.374006 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:22:21.373984 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-predictive-lightgbm-predictor-669896799c-54xhl" Apr 16 17:22:21.374342 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:22:21.374319 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-predictive-lightgbm-predictor-669896799c-54xhl"] Apr 16 17:22:21.389390 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:22:21.389364 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b-kserve-provision-location\") pod \"isvc-predictive-lightgbm-predictor-669896799c-54xhl\" (UID: \"fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b\") " pod="kserve-ci-e2e-test/isvc-predictive-lightgbm-predictor-669896799c-54xhl" Apr 16 17:22:21.490617 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:22:21.490584 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b-kserve-provision-location\") pod \"isvc-predictive-lightgbm-predictor-669896799c-54xhl\" (UID: \"fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b\") " pod="kserve-ci-e2e-test/isvc-predictive-lightgbm-predictor-669896799c-54xhl" Apr 16 17:22:21.490972 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:22:21.490955 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b-kserve-provision-location\") pod \"isvc-predictive-lightgbm-predictor-669896799c-54xhl\" (UID: \"fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b\") " pod="kserve-ci-e2e-test/isvc-predictive-lightgbm-predictor-669896799c-54xhl" Apr 16 17:22:21.686543 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:22:21.686453 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-predictive-lightgbm-predictor-669896799c-54xhl" Apr 16 17:22:21.803836 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:22:21.803810 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-predictive-lightgbm-predictor-669896799c-54xhl"] Apr 16 17:22:21.805658 ip-10-0-140-49 kubenswrapper[2574]: W0416 17:22:21.805620 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbeb89a1_7819_4f6c_b1e4_f25fbf54b19b.slice/crio-3b284beb91e927996c5b7671c3f70d61dbd1f617559774d1dfdb397d22c36a4a WatchSource:0}: Error finding container 3b284beb91e927996c5b7671c3f70d61dbd1f617559774d1dfdb397d22c36a4a: Status 404 returned error can't find the container with id 3b284beb91e927996c5b7671c3f70d61dbd1f617559774d1dfdb397d22c36a4a Apr 16 17:22:22.353511 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:22:22.353475 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-predictive-lightgbm-predictor-669896799c-54xhl" event={"ID":"fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b","Type":"ContainerStarted","Data":"3469ea83cf7bf8da845911b325b12bae4ace6f355f07efc6948766d413ab5602"} Apr 16 17:22:22.353511 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:22:22.353512 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-predictive-lightgbm-predictor-669896799c-54xhl" event={"ID":"fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b","Type":"ContainerStarted","Data":"3b284beb91e927996c5b7671c3f70d61dbd1f617559774d1dfdb397d22c36a4a"} Apr 16 17:22:26.368572 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:22:26.368537 2574 generic.go:358] "Generic (PLEG): container finished" podID="fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b" containerID="3469ea83cf7bf8da845911b325b12bae4ace6f355f07efc6948766d413ab5602" exitCode=0 Apr 16 17:22:26.369020 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:22:26.368614 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-predictive-lightgbm-predictor-669896799c-54xhl" event={"ID":"fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b","Type":"ContainerDied","Data":"3469ea83cf7bf8da845911b325b12bae4ace6f355f07efc6948766d413ab5602"} Apr 16 17:22:27.374173 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:22:27.374140 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-predictive-lightgbm-predictor-669896799c-54xhl" event={"ID":"fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b","Type":"ContainerStarted","Data":"75ce9c48f89d675eec27804d70d757eabb3307d3db6bc34d3df931b6976300d5"} Apr 16 17:22:27.374640 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:22:27.374515 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-predictive-lightgbm-predictor-669896799c-54xhl" Apr 16 17:22:27.375631 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:22:27.375607 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-predictive-lightgbm-predictor-669896799c-54xhl" podUID="fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused" Apr 16 17:22:27.391148 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:22:27.391108 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-predictive-lightgbm-predictor-669896799c-54xhl" podStartSLOduration=6.39109691 podStartE2EDuration="6.39109691s" podCreationTimestamp="2026-04-16 17:22:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 17:22:27.389775127 +0000 UTC m=+2081.471509343" watchObservedRunningTime="2026-04-16 17:22:27.39109691 +0000 UTC m=+2081.472831123" Apr 16 17:22:28.378473 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:22:28.378435 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-predictive-lightgbm-predictor-669896799c-54xhl" podUID="fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused" Apr 16 17:22:38.378978 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:22:38.378939 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-predictive-lightgbm-predictor-669896799c-54xhl" podUID="fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused" Apr 16 17:22:48.379100 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:22:48.379056 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-predictive-lightgbm-predictor-669896799c-54xhl" podUID="fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused" Apr 16 17:22:50.299163 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:22:50.299132 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/1.log" Apr 16 17:22:50.301190 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:22:50.301166 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/1.log" Apr 16 17:22:58.378517 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:22:58.378474 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-predictive-lightgbm-predictor-669896799c-54xhl" podUID="fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused" Apr 16 17:23:08.378905 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:23:08.378865 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-predictive-lightgbm-predictor-669896799c-54xhl" podUID="fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused" Apr 16 17:23:18.379179 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:23:18.379135 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-predictive-lightgbm-predictor-669896799c-54xhl" podUID="fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused" Apr 16 17:23:28.378492 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:23:28.378452 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-predictive-lightgbm-predictor-669896799c-54xhl" podUID="fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused" Apr 16 17:23:38.379380 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:23:38.379350 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-predictive-lightgbm-predictor-669896799c-54xhl" Apr 16 17:27:50.327075 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:27:50.327048 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/1.log" Apr 16 17:27:50.328692 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:27:50.328671 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/1.log" Apr 16 17:29:44.468362 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:29:44.468268 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sklearn-v2-mlserver-predictor-695d5f5568-fvc2w"] Apr 16 17:29:44.471897 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:29:44.471882 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sklearn-v2-mlserver-predictor-695d5f5568-fvc2w" Apr 16 17:29:44.477657 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:29:44.477632 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sklearn-v2-mlserver-predictor-695d5f5568-fvc2w"] Apr 16 17:29:44.626564 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:29:44.626523 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6e510c2a-a7c8-4c1a-8754-018a372a6279-kserve-provision-location\") pod \"sklearn-v2-mlserver-predictor-695d5f5568-fvc2w\" (UID: \"6e510c2a-a7c8-4c1a-8754-018a372a6279\") " pod="kserve-ci-e2e-test/sklearn-v2-mlserver-predictor-695d5f5568-fvc2w" Apr 16 17:29:44.727829 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:29:44.727726 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6e510c2a-a7c8-4c1a-8754-018a372a6279-kserve-provision-location\") pod \"sklearn-v2-mlserver-predictor-695d5f5568-fvc2w\" (UID: \"6e510c2a-a7c8-4c1a-8754-018a372a6279\") " pod="kserve-ci-e2e-test/sklearn-v2-mlserver-predictor-695d5f5568-fvc2w" Apr 16 17:29:44.728170 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:29:44.728150 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6e510c2a-a7c8-4c1a-8754-018a372a6279-kserve-provision-location\") pod \"sklearn-v2-mlserver-predictor-695d5f5568-fvc2w\" (UID: \"6e510c2a-a7c8-4c1a-8754-018a372a6279\") " pod="kserve-ci-e2e-test/sklearn-v2-mlserver-predictor-695d5f5568-fvc2w" Apr 16 17:29:44.783430 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:29:44.783401 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sklearn-v2-mlserver-predictor-695d5f5568-fvc2w" Apr 16 17:29:44.908601 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:29:44.908567 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sklearn-v2-mlserver-predictor-695d5f5568-fvc2w"] Apr 16 17:29:44.909738 ip-10-0-140-49 kubenswrapper[2574]: W0416 17:29:44.909707 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e510c2a_a7c8_4c1a_8754_018a372a6279.slice/crio-3961cb8b466fdfb20f21ce53f6255d63473ce2927e670c8aa6007a27476f24a7 WatchSource:0}: Error finding container 3961cb8b466fdfb20f21ce53f6255d63473ce2927e670c8aa6007a27476f24a7: Status 404 returned error can't find the container with id 3961cb8b466fdfb20f21ce53f6255d63473ce2927e670c8aa6007a27476f24a7 Apr 16 17:29:44.911930 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:29:44.911915 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 16 17:29:45.891473 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:29:45.891440 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sklearn-v2-mlserver-predictor-695d5f5568-fvc2w" event={"ID":"6e510c2a-a7c8-4c1a-8754-018a372a6279","Type":"ContainerStarted","Data":"f223124361ab4f446bc2e50699366a76f8e53e6b43859978da445bddc32fe30b"} Apr 16 17:29:45.891845 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:29:45.891479 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sklearn-v2-mlserver-predictor-695d5f5568-fvc2w" event={"ID":"6e510c2a-a7c8-4c1a-8754-018a372a6279","Type":"ContainerStarted","Data":"3961cb8b466fdfb20f21ce53f6255d63473ce2927e670c8aa6007a27476f24a7"} Apr 16 17:29:48.902792 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:29:48.902718 2574 generic.go:358] "Generic (PLEG): container finished" podID="6e510c2a-a7c8-4c1a-8754-018a372a6279" containerID="f223124361ab4f446bc2e50699366a76f8e53e6b43859978da445bddc32fe30b" exitCode=0 Apr 16 17:29:48.903116 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:29:48.902791 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sklearn-v2-mlserver-predictor-695d5f5568-fvc2w" event={"ID":"6e510c2a-a7c8-4c1a-8754-018a372a6279","Type":"ContainerDied","Data":"f223124361ab4f446bc2e50699366a76f8e53e6b43859978da445bddc32fe30b"} Apr 16 17:29:49.908252 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:29:49.908187 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sklearn-v2-mlserver-predictor-695d5f5568-fvc2w" event={"ID":"6e510c2a-a7c8-4c1a-8754-018a372a6279","Type":"ContainerStarted","Data":"8d4fad8b9a10ec808ca7c9e7dbe6f75fd9f7223464f7af6b280e7defb71afa18"} Apr 16 17:29:49.908711 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:29:49.908444 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sklearn-v2-mlserver-predictor-695d5f5568-fvc2w" Apr 16 17:29:49.925244 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:29:49.925177 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/sklearn-v2-mlserver-predictor-695d5f5568-fvc2w" podStartSLOduration=5.9251629260000005 podStartE2EDuration="5.925162926s" podCreationTimestamp="2026-04-16 17:29:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 17:29:49.92295717 +0000 UTC m=+2524.004691379" watchObservedRunningTime="2026-04-16 17:29:49.925162926 +0000 UTC m=+2524.006897141" Apr 16 17:30:20.939516 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:30:20.939473 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sklearn-v2-mlserver-predictor-695d5f5568-fvc2w" podUID="6e510c2a-a7c8-4c1a-8754-018a372a6279" containerName="kserve-container" probeResult="failure" output="HTTP probe failed with statuscode: 400" Apr 16 17:30:30.914940 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:30:30.914905 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sklearn-v2-mlserver-predictor-695d5f5568-fvc2w" Apr 16 17:31:15.825181 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:31:15.825148 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2"] Apr 16 17:31:15.828988 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:31:15.828971 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2" Apr 16 17:31:15.835465 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:31:15.835444 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2"] Apr 16 17:31:15.856506 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:31:15.856482 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/059eab9b-8d48-4612-a432-a19551e35a31-kserve-provision-location\") pod \"isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2\" (UID: \"059eab9b-8d48-4612-a432-a19551e35a31\") " pod="kserve-ci-e2e-test/isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2" Apr 16 17:31:15.956917 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:31:15.956887 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/059eab9b-8d48-4612-a432-a19551e35a31-kserve-provision-location\") pod \"isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2\" (UID: \"059eab9b-8d48-4612-a432-a19551e35a31\") " pod="kserve-ci-e2e-test/isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2" Apr 16 17:31:15.957266 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:31:15.957246 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/059eab9b-8d48-4612-a432-a19551e35a31-kserve-provision-location\") pod \"isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2\" (UID: \"059eab9b-8d48-4612-a432-a19551e35a31\") " pod="kserve-ci-e2e-test/isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2" Apr 16 17:31:16.141546 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:31:16.141515 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2" Apr 16 17:31:16.471022 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:31:16.471000 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2"] Apr 16 17:31:16.473058 ip-10-0-140-49 kubenswrapper[2574]: W0416 17:31:16.473028 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod059eab9b_8d48_4612_a432_a19551e35a31.slice/crio-e3063b6261550cb2e68b766572d32a4439ce66b6803ad8830a08cf7d68b9c1e7 WatchSource:0}: Error finding container e3063b6261550cb2e68b766572d32a4439ce66b6803ad8830a08cf7d68b9c1e7: Status 404 returned error can't find the container with id e3063b6261550cb2e68b766572d32a4439ce66b6803ad8830a08cf7d68b9c1e7 Apr 16 17:31:17.218546 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:31:17.218510 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2" event={"ID":"059eab9b-8d48-4612-a432-a19551e35a31","Type":"ContainerStarted","Data":"1dedd77a319b68d91b0c9cd9e9d6e46dfc6457636c4edb4aa6cfff62d6d557a4"} Apr 16 17:31:17.218546 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:31:17.218549 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2" event={"ID":"059eab9b-8d48-4612-a432-a19551e35a31","Type":"ContainerStarted","Data":"e3063b6261550cb2e68b766572d32a4439ce66b6803ad8830a08cf7d68b9c1e7"} Apr 16 17:31:21.233949 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:31:21.233916 2574 generic.go:358] "Generic (PLEG): container finished" podID="059eab9b-8d48-4612-a432-a19551e35a31" containerID="1dedd77a319b68d91b0c9cd9e9d6e46dfc6457636c4edb4aa6cfff62d6d557a4" exitCode=0 Apr 16 17:31:21.234366 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:31:21.233994 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2" event={"ID":"059eab9b-8d48-4612-a432-a19551e35a31","Type":"ContainerDied","Data":"1dedd77a319b68d91b0c9cd9e9d6e46dfc6457636c4edb4aa6cfff62d6d557a4"} Apr 16 17:31:22.241870 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:31:22.241832 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2" event={"ID":"059eab9b-8d48-4612-a432-a19551e35a31","Type":"ContainerStarted","Data":"ab16f5b693911ef2841a2c27dc88e31e82a54af50bdae43f0351a964f4976eee"} Apr 16 17:31:22.242293 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:31:22.242057 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2" Apr 16 17:31:22.259742 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:31:22.259700 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2" podStartSLOduration=7.259687543 podStartE2EDuration="7.259687543s" podCreationTimestamp="2026-04-16 17:31:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 17:31:22.258175063 +0000 UTC m=+2616.339909279" watchObservedRunningTime="2026-04-16 17:31:22.259687543 +0000 UTC m=+2616.341421758" Apr 16 17:31:53.340100 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:31:53.340059 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2" podUID="059eab9b-8d48-4612-a432-a19551e35a31" containerName="kserve-container" probeResult="failure" output="HTTP probe failed with statuscode: 400" Apr 16 17:32:03.248385 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:32:03.248349 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2" Apr 16 17:32:50.355381 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:32:50.355302 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/1.log" Apr 16 17:32:50.357488 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:32:50.357468 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/1.log" Apr 16 17:37:50.382564 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:37:50.382536 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/1.log" Apr 16 17:37:50.388020 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:37:50.388001 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/1.log" Apr 16 17:39:17.422330 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:39:17.422295 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77"] Apr 16 17:39:17.425777 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:39:17.425756 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77" Apr 16 17:39:17.434717 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:39:17.434691 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77"] Apr 16 17:39:17.562840 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:39:17.562793 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/59ee056f-54da-42d8-8925-48fa8e215f7a-kserve-provision-location\") pod \"isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77\" (UID: \"59ee056f-54da-42d8-8925-48fa8e215f7a\") " pod="kserve-ci-e2e-test/isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77" Apr 16 17:39:17.663767 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:39:17.663735 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/59ee056f-54da-42d8-8925-48fa8e215f7a-kserve-provision-location\") pod \"isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77\" (UID: \"59ee056f-54da-42d8-8925-48fa8e215f7a\") " pod="kserve-ci-e2e-test/isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77" Apr 16 17:39:17.664244 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:39:17.664188 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/59ee056f-54da-42d8-8925-48fa8e215f7a-kserve-provision-location\") pod \"isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77\" (UID: \"59ee056f-54da-42d8-8925-48fa8e215f7a\") " pod="kserve-ci-e2e-test/isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77" Apr 16 17:39:17.736883 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:39:17.736815 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77" Apr 16 17:39:17.860008 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:39:17.859986 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77"] Apr 16 17:39:17.861840 ip-10-0-140-49 kubenswrapper[2574]: W0416 17:39:17.861809 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59ee056f_54da_42d8_8925_48fa8e215f7a.slice/crio-6425d1e7a783aaa7f752db988c60caa3a76074959dafc51a42d1cd1bdde5720e WatchSource:0}: Error finding container 6425d1e7a783aaa7f752db988c60caa3a76074959dafc51a42d1cd1bdde5720e: Status 404 returned error can't find the container with id 6425d1e7a783aaa7f752db988c60caa3a76074959dafc51a42d1cd1bdde5720e Apr 16 17:39:17.863699 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:39:17.863683 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 16 17:39:17.916259 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:39:17.916232 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77" event={"ID":"59ee056f-54da-42d8-8925-48fa8e215f7a","Type":"ContainerStarted","Data":"6425d1e7a783aaa7f752db988c60caa3a76074959dafc51a42d1cd1bdde5720e"} Apr 16 17:39:18.921183 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:39:18.921147 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77" event={"ID":"59ee056f-54da-42d8-8925-48fa8e215f7a","Type":"ContainerStarted","Data":"ed4fecc2ca039f6b79635090ef0f2f5bc862584fb7643c1c9283d6d7cd065500"} Apr 16 17:39:21.933399 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:39:21.933370 2574 generic.go:358] "Generic (PLEG): container finished" podID="59ee056f-54da-42d8-8925-48fa8e215f7a" containerID="ed4fecc2ca039f6b79635090ef0f2f5bc862584fb7643c1c9283d6d7cd065500" exitCode=0 Apr 16 17:39:21.933707 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:39:21.933429 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77" event={"ID":"59ee056f-54da-42d8-8925-48fa8e215f7a","Type":"ContainerDied","Data":"ed4fecc2ca039f6b79635090ef0f2f5bc862584fb7643c1c9283d6d7cd065500"} Apr 16 17:39:22.939075 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:39:22.939035 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77" event={"ID":"59ee056f-54da-42d8-8925-48fa8e215f7a","Type":"ContainerStarted","Data":"b7dfca91c429ec16a794985c67c0d86848519f06a09325f3c3013bd5f2264ec3"} Apr 16 17:39:22.939577 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:39:22.939253 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77" Apr 16 17:39:22.956748 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:39:22.956705 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77" podStartSLOduration=5.956693724 podStartE2EDuration="5.956693724s" podCreationTimestamp="2026-04-16 17:39:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 17:39:22.955102532 +0000 UTC m=+3097.036836748" watchObservedRunningTime="2026-04-16 17:39:22.956693724 +0000 UTC m=+3097.038427939" Apr 16 17:39:54.039952 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:39:54.039916 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77" Apr 16 17:39:57.554356 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:39:57.554277 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/xgboost-v2-mlserver-predictor-c58d48f-f64lm"] Apr 16 17:39:57.558032 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:39:57.558017 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/xgboost-v2-mlserver-predictor-c58d48f-f64lm" Apr 16 17:39:57.564352 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:39:57.564323 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/xgboost-v2-mlserver-predictor-c58d48f-f64lm"] Apr 16 17:39:57.666716 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:39:57.666687 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/306d7e24-f7b2-4a97-9e1d-8ded6e6ef290-kserve-provision-location\") pod \"xgboost-v2-mlserver-predictor-c58d48f-f64lm\" (UID: \"306d7e24-f7b2-4a97-9e1d-8ded6e6ef290\") " pod="kserve-ci-e2e-test/xgboost-v2-mlserver-predictor-c58d48f-f64lm" Apr 16 17:39:57.767549 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:39:57.767524 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/306d7e24-f7b2-4a97-9e1d-8ded6e6ef290-kserve-provision-location\") pod \"xgboost-v2-mlserver-predictor-c58d48f-f64lm\" (UID: \"306d7e24-f7b2-4a97-9e1d-8ded6e6ef290\") " pod="kserve-ci-e2e-test/xgboost-v2-mlserver-predictor-c58d48f-f64lm" Apr 16 17:39:57.767880 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:39:57.767862 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/306d7e24-f7b2-4a97-9e1d-8ded6e6ef290-kserve-provision-location\") pod \"xgboost-v2-mlserver-predictor-c58d48f-f64lm\" (UID: \"306d7e24-f7b2-4a97-9e1d-8ded6e6ef290\") " pod="kserve-ci-e2e-test/xgboost-v2-mlserver-predictor-c58d48f-f64lm" Apr 16 17:39:57.870185 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:39:57.870154 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/xgboost-v2-mlserver-predictor-c58d48f-f64lm" Apr 16 17:39:57.992222 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:39:57.992171 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/xgboost-v2-mlserver-predictor-c58d48f-f64lm"] Apr 16 17:39:57.994294 ip-10-0-140-49 kubenswrapper[2574]: W0416 17:39:57.994263 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod306d7e24_f7b2_4a97_9e1d_8ded6e6ef290.slice/crio-94570abd6049d333a8290c74a646fa33e898cf4ae63de58b3ac177fa700b1134 WatchSource:0}: Error finding container 94570abd6049d333a8290c74a646fa33e898cf4ae63de58b3ac177fa700b1134: Status 404 returned error can't find the container with id 94570abd6049d333a8290c74a646fa33e898cf4ae63de58b3ac177fa700b1134 Apr 16 17:39:58.069513 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:39:58.069489 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/xgboost-v2-mlserver-predictor-c58d48f-f64lm" event={"ID":"306d7e24-f7b2-4a97-9e1d-8ded6e6ef290","Type":"ContainerStarted","Data":"8a0377223b0f93a73bfa1122df7c97eb35a51745935280d2dad92a5742a9aba1"} Apr 16 17:39:58.069632 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:39:58.069521 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/xgboost-v2-mlserver-predictor-c58d48f-f64lm" event={"ID":"306d7e24-f7b2-4a97-9e1d-8ded6e6ef290","Type":"ContainerStarted","Data":"94570abd6049d333a8290c74a646fa33e898cf4ae63de58b3ac177fa700b1134"} Apr 16 17:40:02.085734 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:40:02.085655 2574 generic.go:358] "Generic (PLEG): container finished" podID="306d7e24-f7b2-4a97-9e1d-8ded6e6ef290" containerID="8a0377223b0f93a73bfa1122df7c97eb35a51745935280d2dad92a5742a9aba1" exitCode=0 Apr 16 17:40:02.086161 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:40:02.085739 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/xgboost-v2-mlserver-predictor-c58d48f-f64lm" event={"ID":"306d7e24-f7b2-4a97-9e1d-8ded6e6ef290","Type":"ContainerDied","Data":"8a0377223b0f93a73bfa1122df7c97eb35a51745935280d2dad92a5742a9aba1"} Apr 16 17:40:03.091417 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:40:03.091375 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/xgboost-v2-mlserver-predictor-c58d48f-f64lm" event={"ID":"306d7e24-f7b2-4a97-9e1d-8ded6e6ef290","Type":"ContainerStarted","Data":"1aca9d902a60854e400b2a1a08a33586460e303effafbfbcada167e70141ae06"} Apr 16 17:40:03.091838 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:40:03.091718 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/xgboost-v2-mlserver-predictor-c58d48f-f64lm" Apr 16 17:40:03.109057 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:40:03.109000 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/xgboost-v2-mlserver-predictor-c58d48f-f64lm" podStartSLOduration=6.108988109 podStartE2EDuration="6.108988109s" podCreationTimestamp="2026-04-16 17:39:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 17:40:03.106493997 +0000 UTC m=+3137.188228227" watchObservedRunningTime="2026-04-16 17:40:03.108988109 +0000 UTC m=+3137.190722323" Apr 16 17:40:34.140069 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:40:34.140036 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/xgboost-v2-mlserver-predictor-c58d48f-f64lm" Apr 16 17:41:47.905158 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:41:47.905073 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg"] Apr 16 17:41:47.908854 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:41:47.908839 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg" Apr 16 17:41:47.916516 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:41:47.916490 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg"] Apr 16 17:41:48.039727 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:41:48.039696 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/13d15bf0-ed3e-456a-b5f5-ee62c25e3220-kserve-provision-location\") pod \"isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg\" (UID: \"13d15bf0-ed3e-456a-b5f5-ee62c25e3220\") " pod="kserve-ci-e2e-test/isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg" Apr 16 17:41:48.140995 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:41:48.140961 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/13d15bf0-ed3e-456a-b5f5-ee62c25e3220-kserve-provision-location\") pod \"isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg\" (UID: \"13d15bf0-ed3e-456a-b5f5-ee62c25e3220\") " pod="kserve-ci-e2e-test/isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg" Apr 16 17:41:48.141382 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:41:48.141357 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/13d15bf0-ed3e-456a-b5f5-ee62c25e3220-kserve-provision-location\") pod \"isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg\" (UID: \"13d15bf0-ed3e-456a-b5f5-ee62c25e3220\") " pod="kserve-ci-e2e-test/isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg" Apr 16 17:41:48.220929 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:41:48.220866 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg" Apr 16 17:41:48.344701 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:41:48.344674 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg"] Apr 16 17:41:48.346244 ip-10-0-140-49 kubenswrapper[2574]: W0416 17:41:48.346194 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13d15bf0_ed3e_456a_b5f5_ee62c25e3220.slice/crio-95afc4ff1d6abbd6d4dc0c22c6dcc28f238052e01200d7d2eee7dd92cf31e96d WatchSource:0}: Error finding container 95afc4ff1d6abbd6d4dc0c22c6dcc28f238052e01200d7d2eee7dd92cf31e96d: Status 404 returned error can't find the container with id 95afc4ff1d6abbd6d4dc0c22c6dcc28f238052e01200d7d2eee7dd92cf31e96d Apr 16 17:41:48.479886 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:41:48.479808 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg" event={"ID":"13d15bf0-ed3e-456a-b5f5-ee62c25e3220","Type":"ContainerStarted","Data":"0807254b41204e7f3794038da47bb25ff3cb5b10dbf2db82bb982c96ac20c872"} Apr 16 17:41:48.479886 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:41:48.479853 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg" event={"ID":"13d15bf0-ed3e-456a-b5f5-ee62c25e3220","Type":"ContainerStarted","Data":"95afc4ff1d6abbd6d4dc0c22c6dcc28f238052e01200d7d2eee7dd92cf31e96d"} Apr 16 17:41:52.496262 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:41:52.496223 2574 generic.go:358] "Generic (PLEG): container finished" podID="13d15bf0-ed3e-456a-b5f5-ee62c25e3220" containerID="0807254b41204e7f3794038da47bb25ff3cb5b10dbf2db82bb982c96ac20c872" exitCode=0 Apr 16 17:41:52.496660 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:41:52.496281 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg" event={"ID":"13d15bf0-ed3e-456a-b5f5-ee62c25e3220","Type":"ContainerDied","Data":"0807254b41204e7f3794038da47bb25ff3cb5b10dbf2db82bb982c96ac20c872"} Apr 16 17:41:53.501442 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:41:53.501409 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg" event={"ID":"13d15bf0-ed3e-456a-b5f5-ee62c25e3220","Type":"ContainerStarted","Data":"50bd520ee6003c7a0fe46c52f4334ac35039fb77feded7d35ce5704f53e0394d"} Apr 16 17:41:53.501868 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:41:53.501634 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg" Apr 16 17:41:53.519176 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:41:53.519127 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg" podStartSLOduration=6.51911403 podStartE2EDuration="6.51911403s" podCreationTimestamp="2026-04-16 17:41:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 17:41:53.516860928 +0000 UTC m=+3247.598595143" watchObservedRunningTime="2026-04-16 17:41:53.51911403 +0000 UTC m=+3247.600848241" Apr 16 17:42:24.540293 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:42:24.540243 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg" podUID="13d15bf0-ed3e-456a-b5f5-ee62c25e3220" containerName="kserve-container" probeResult="failure" output="HTTP probe failed with statuscode: 400" Apr 16 17:42:34.512175 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:42:34.512148 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg" Apr 16 17:42:50.412006 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:42:50.411971 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/1.log" Apr 16 17:42:50.423880 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:42:50.423859 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/1.log" Apr 16 17:47:50.444807 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:47:50.444778 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/1.log" Apr 16 17:47:50.453300 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:47:50.453278 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/1.log" Apr 16 17:49:40.429133 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:40.429099 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:49:40.448578 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:40.448554 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:49:40.459053 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:40.459031 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:49:40.495392 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:40.495364 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:49:40.507460 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:40.507419 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:49:40.516260 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:40.516244 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:49:40.530516 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:40.530497 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:49:40.536151 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:40.536131 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:49:40.606696 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:40.606675 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:49:40.612393 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:40.612376 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:49:40.681432 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:40.681363 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:49:40.689949 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:40.689927 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:49:40.700396 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:40.700349 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:49:40.708833 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:40.708814 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:49:40.741135 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:40.741109 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:49:40.749890 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:40.749869 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:49:40.791052 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:40.791034 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:49:40.823419 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:40.823397 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:49:41.065219 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:41.065122 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:49:41.073013 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:41.072994 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:49:41.212508 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:41.212482 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:49:41.220739 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:41.220716 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:49:41.251385 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:41.251363 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:49:41.259649 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:41.259633 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:49:41.282735 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:41.282702 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:49:41.324776 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:41.324725 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:49:41.334299 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:41.334278 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:49:41.343022 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:41.342990 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:49:42.745849 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:42.745820 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:49:42.755302 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:42.755279 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:49:42.767532 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:42.767512 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:49:42.776334 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:42.776314 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:49:42.787800 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:42.787777 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:49:42.796294 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:42.796279 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:49:42.812725 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:42.812707 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:49:42.818398 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:42.818382 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:49:42.863281 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:42.863260 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:49:42.868895 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:42.868880 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:49:42.941647 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:42.941629 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:49:42.949941 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:42.949923 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:49:42.961441 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:42.961427 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:49:42.970306 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:42.970289 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:49:43.004080 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:43.004024 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:49:43.012218 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:43.012183 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:49:43.048998 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:43.048975 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:49:43.057513 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:43.057469 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:49:43.313330 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:43.313261 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:49:43.323631 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:43.323608 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:49:43.445613 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:43.445590 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:49:43.453306 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:43.453287 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:49:43.486265 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:43.486243 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:49:43.493993 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:43.493979 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:49:43.518438 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:43.518417 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:49:43.525865 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:43.525848 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:49:43.538310 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:43.538288 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:49:43.546346 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:43.546326 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:49:44.912437 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:44.912412 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:49:44.921567 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:44.921520 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:49:44.933210 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:44.933190 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:49:44.941834 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:44.941812 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:49:44.953523 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:44.953503 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:49:44.963706 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:44.963689 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:49:44.978381 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:44.978361 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:49:44.984104 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:44.984089 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:49:45.032560 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:45.032540 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:49:45.038091 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:45.038069 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:49:45.110989 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:45.110966 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:49:45.119658 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:45.119642 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:49:45.130995 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:45.130981 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:49:45.139730 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:45.139716 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:49:45.170491 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:45.170438 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:49:45.178900 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:45.178878 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:49:45.221713 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:45.221697 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:49:45.231056 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:45.231039 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:49:45.479899 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:45.479834 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:49:45.488110 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:45.488088 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:49:45.602257 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:45.602239 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:49:45.610306 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:45.610287 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:49:45.642436 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:45.642415 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:49:45.650551 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:45.650535 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:49:45.674583 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:45.674562 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:49:45.682198 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:45.682178 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:49:45.695132 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:45.695115 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:49:45.703063 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:45.703046 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:49:47.064754 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:47.064726 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:49:47.075397 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:47.075372 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:49:47.089103 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:47.089083 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:49:47.098940 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:47.098925 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:49:47.111027 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:47.111011 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:49:47.120911 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:47.120894 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:49:47.136524 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:47.136502 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:49:47.142154 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:47.142138 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:49:47.189424 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:47.189407 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:49:47.195862 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:47.195844 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:49:47.269767 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:47.269748 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:49:47.278663 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:47.278631 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:49:47.291536 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:47.291508 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:49:47.300538 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:47.300518 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:49:47.332948 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:47.332886 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:49:47.342315 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:47.342293 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:49:47.377728 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:47.377709 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:49:47.386813 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:47.386794 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:49:47.644846 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:47.644817 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:49:47.660138 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:47.660117 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:49:47.779379 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:47.779354 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:49:47.787917 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:47.787895 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:49:47.823496 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:47.823474 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:49:47.832182 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:47.832165 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:49:47.856245 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:47.856228 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:49:47.877408 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:47.877358 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:49:47.889745 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:47.889727 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:49:47.898565 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:47.898486 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:49:49.257012 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:49.256969 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:49:49.266262 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:49.266236 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:49:49.278293 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:49.278263 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:49:49.287303 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:49.287285 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:49:49.298688 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:49.298667 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:49:49.307047 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:49.307019 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:49:49.321349 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:49.321322 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:49:49.327790 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:49.327775 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:49:49.373491 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:49.373469 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:49:49.379637 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:49.379620 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:49:49.449648 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:49.449621 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:49:49.458164 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:49.458143 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:49:49.469135 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:49.469116 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:49:49.477615 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:49.477580 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:49:49.508079 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:49.507994 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:49:49.515843 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:49.515813 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:49:49.548896 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:49.548874 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:49:49.557626 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:49.557612 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:49:49.813439 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:49.813376 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:49:49.821769 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:49.821753 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:49:49.931348 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:49.931325 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:49:49.938784 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:49.938763 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:49:49.970361 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:49.970340 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:49:49.978390 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:49.978372 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:49:50.003796 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:50.003774 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:49:50.012011 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:50.011984 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:49:50.023927 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:50.023903 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:49:50.032067 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:50.032049 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:49:51.457333 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:51.457303 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:49:51.471769 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:51.471744 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:49:51.486594 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:51.486570 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:49:51.496111 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:51.496091 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:49:51.508773 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:51.508755 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:49:51.520265 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:51.520249 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:49:51.538895 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:51.538874 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:49:51.546387 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:51.546368 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:49:51.592969 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:51.592949 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:49:51.601236 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:51.601197 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:49:51.680274 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:51.680250 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:49:51.689091 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:51.689061 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:49:51.701613 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:51.701589 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:49:51.710337 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:51.710288 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:49:51.743695 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:51.743673 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:49:51.752756 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:51.752735 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:49:51.788476 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:51.788452 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:49:51.797271 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:51.797253 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:49:52.070128 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:52.070032 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:49:52.077818 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:52.077792 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:49:52.201950 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:52.201921 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:49:52.210196 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:52.210170 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:49:52.245977 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:52.245958 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:49:52.260775 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:52.260757 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:49:52.354284 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:52.354265 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:49:52.368479 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:52.368458 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:49:52.383770 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:52.383750 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:49:52.395034 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:52.395016 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:49:53.832137 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:53.832109 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:49:53.843294 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:53.843271 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:49:53.856408 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:53.856391 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:49:53.866959 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:53.866938 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:49:53.880400 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:53.880382 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:49:53.890438 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:53.890415 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:49:53.905235 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:53.905219 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:49:53.911654 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:53.911628 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:49:53.974942 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:53.974923 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:49:53.982217 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:53.982184 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:49:54.065046 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:54.065028 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:49:54.074323 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:54.074284 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:49:54.085616 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:54.085568 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:49:54.096942 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:54.096920 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:49:54.136760 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:54.136735 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:49:54.146659 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:54.146639 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:49:54.186044 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:54.186013 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:49:54.197602 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:54.197583 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:49:54.471963 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:54.471935 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:49:54.481140 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:54.481110 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:49:54.639639 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:54.639613 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:49:54.650691 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:54.650668 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:49:54.694243 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:54.694218 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:49:54.703874 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:54.703857 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:49:54.741854 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:54.741805 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:49:54.751951 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:54.751933 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:49:54.767468 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:54.767447 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:49:54.777122 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:54.777102 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:49:56.169844 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:56.169812 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:49:56.179692 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:56.179673 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:49:56.194249 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:56.194223 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:49:56.206833 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:56.206812 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:49:56.221185 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:56.221165 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:49:56.233315 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:56.233295 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:49:56.250368 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:56.250344 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:49:56.259484 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:56.259468 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:49:56.314531 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:56.314506 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:49:56.320922 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:56.320897 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:49:56.406710 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:56.406668 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:49:56.417401 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:56.417386 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:49:56.431698 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:56.431647 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:49:56.441407 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:56.441390 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:49:56.477352 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:56.477335 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:49:56.487370 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:56.487340 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:49:56.525983 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:56.525941 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:49:56.536336 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:56.536305 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:49:56.849827 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:56.849728 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:49:56.857749 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:56.857725 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:49:56.986672 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:56.986638 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:49:56.999346 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:56.999322 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:49:57.037992 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:57.037968 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:49:57.047109 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:57.047086 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:49:57.076588 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:57.076559 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:49:57.086720 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:57.086689 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:49:57.099930 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:57.099874 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:49:57.109766 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:57.109732 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:49:58.493115 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:58.493062 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:49:58.505233 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:58.505191 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:49:58.519788 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:58.519769 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:49:58.530580 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:58.530562 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:49:58.542063 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:58.542044 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:49:58.552734 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:58.552717 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:49:58.569327 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:58.569312 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:49:58.576014 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:58.575997 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:49:58.646453 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:58.646433 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:49:58.654881 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:58.654868 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:49:58.738177 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:58.738163 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:49:58.750322 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:58.750271 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:49:58.764311 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:58.764292 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:49:58.775129 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:58.775108 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:49:58.808147 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:58.808128 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:49:58.818357 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:58.818341 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:49:58.860676 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:58.860660 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:49:58.870171 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:58.870155 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:49:59.181379 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:59.181352 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:49:59.189931 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:59.189911 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:49:59.322173 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:59.322146 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:49:59.330851 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:59.330828 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:49:59.381088 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:59.381046 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:49:59.390897 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:59.390877 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:49:59.418273 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:59.418251 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:49:59.429821 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:59.429804 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:49:59.444688 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:59.444633 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:49:59.453575 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:49:59.453559 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:50:00.836342 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:00.836314 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:50:00.847387 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:00.847367 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:50:00.871794 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:00.871775 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:50:00.895827 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:00.895809 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:50:00.911588 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:00.911567 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:50:00.922276 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:00.922260 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:50:00.941033 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:00.941011 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:50:00.948727 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:00.948702 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:50:00.997452 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:00.997436 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:50:01.005629 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:01.005607 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:50:01.110933 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:01.110918 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:50:01.123727 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:01.123711 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:50:01.138935 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:01.138917 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:50:01.151484 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:01.151466 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:50:01.193982 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:01.193963 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:50:01.203538 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:01.203519 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:50:01.257168 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:01.257149 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:50:01.271594 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:01.271576 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:50:01.567346 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:01.567262 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:50:01.577344 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:01.577322 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:50:01.704918 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:01.704896 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:50:01.714367 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:01.714346 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:50:01.751434 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:01.751401 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:50:01.763494 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:01.763476 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:50:01.788509 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:01.788490 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:50:01.797909 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:01.797890 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:50:01.812309 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:01.812292 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:50:01.821953 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:01.821904 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:50:03.236461 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:03.236437 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:50:03.248253 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:03.248231 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:50:03.265906 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:03.265879 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:50:03.275740 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:03.275721 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:50:03.287002 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:03.286981 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:50:03.296381 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:03.296363 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:50:03.311550 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:03.311530 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:50:03.316897 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:03.316882 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:50:03.363364 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:03.363341 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:50:03.368906 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:03.368887 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:50:03.445534 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:03.445514 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:50:03.454493 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:03.454478 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:50:03.465824 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:03.465809 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:50:03.475477 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:03.475460 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:50:03.515270 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:03.515198 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:50:03.525183 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:03.525164 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:50:03.574835 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:03.574818 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:50:03.585104 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:03.585084 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:50:03.849158 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:03.849062 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:50:03.856493 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:03.856475 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:50:03.993280 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:03.993253 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:50:04.005179 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:04.005160 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:50:04.043624 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:04.043606 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:50:04.053033 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:04.053016 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:50:04.081065 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:04.081042 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:50:04.091426 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:04.091380 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:50:04.104788 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:04.104769 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:50:04.112681 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:04.112662 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:50:05.490899 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:05.490867 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:50:05.500926 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:05.500906 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:50:05.512169 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:05.512145 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:50:05.521656 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:05.521634 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:50:05.533659 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:05.533642 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:50:05.545374 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:05.545353 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:50:05.560631 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:05.560614 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:50:05.568024 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:05.568007 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:50:05.615076 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:05.615054 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:50:05.621500 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:05.621482 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:50:05.705359 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:05.705338 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:50:05.713764 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:05.713750 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:50:05.725098 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:05.725080 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:50:05.735568 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:05.735552 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:50:05.769850 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:05.769798 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:50:05.778659 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:05.778632 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:50:05.813403 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:05.813386 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:50:05.830057 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:05.830033 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:50:06.091655 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:06.091589 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:50:06.102081 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:06.102040 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:50:06.235296 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:06.235273 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:50:06.243664 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:06.243644 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:50:06.278637 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:06.278619 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:50:06.286884 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:06.286866 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:50:06.311663 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:06.311644 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:50:06.319345 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:06.319329 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:50:06.332272 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:06.332252 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:50:06.340148 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:06.340126 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:50:07.714779 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:07.714751 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:50:07.724526 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:07.724503 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:50:07.736515 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:07.736494 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:50:07.745957 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:07.745939 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:50:07.758650 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:07.758630 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:50:07.767891 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:07.767875 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:50:07.783160 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:07.783139 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:50:07.791907 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:07.791888 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:50:07.844275 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:07.844245 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:50:07.849724 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:07.849701 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:50:07.948458 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:07.948413 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:50:07.958097 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:07.958077 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:50:07.970919 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:07.970843 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:50:07.980326 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:07.980310 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:50:08.012929 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:08.012902 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:50:08.021872 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:08.021853 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:50:08.063513 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:08.063494 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:50:08.075818 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:08.075800 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:50:08.340212 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:08.340117 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:50:08.348532 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:08.348515 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:50:08.490407 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:08.490374 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:50:08.499128 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:08.499102 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:50:08.535689 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:08.535664 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:50:08.544165 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:08.544149 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:50:08.571347 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:08.571323 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:50:08.580075 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:08.580058 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:50:08.593375 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:08.593319 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:50:08.603551 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:08.603530 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:50:09.999965 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:09.999940 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:50:10.008873 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:10.008851 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:50:10.019865 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:10.019845 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:50:10.028283 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:10.028267 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:50:10.039337 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:10.039316 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:50:10.047715 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:10.047699 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:50:10.062035 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:10.062012 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:50:10.068109 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:10.068089 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:50:10.114084 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:10.114064 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:50:10.119591 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:10.119571 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:50:10.191521 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:10.191495 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:50:10.203327 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:10.203303 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:50:10.214375 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:10.214353 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:50:10.222676 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:10.222657 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:50:10.253234 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:10.253148 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:50:10.261364 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:10.261347 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:50:10.295992 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:10.295970 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:50:10.304700 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:10.304677 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:50:10.562130 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:10.562046 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:50:10.569021 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:10.569000 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:50:10.687908 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:10.687887 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:50:10.695195 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:10.695174 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:50:10.725938 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:10.725911 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:50:10.733561 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:10.733528 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:50:10.757137 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:10.757107 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:50:10.764300 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:10.764278 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:50:10.776274 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:10.776254 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:50:10.783898 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:10.783878 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:50:12.140435 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:12.140402 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:50:12.152942 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:12.152908 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:50:12.169146 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:12.169115 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:50:12.182016 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:12.182000 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:50:12.200083 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:12.200062 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:50:12.215875 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:12.215857 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:50:12.235272 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:12.235253 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:50:12.247833 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:12.247814 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:50:12.311005 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:12.310984 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:50:12.321599 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:12.321582 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:50:12.418151 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:12.418053 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:50:12.427860 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:12.427845 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:50:12.443894 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:12.443871 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:50:12.455892 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:12.455876 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:50:12.499149 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:12.499105 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:50:12.514871 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:12.514855 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:50:12.575551 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:12.575527 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:50:12.584642 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:12.584623 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:50:12.921313 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:12.921286 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:50:12.931891 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:12.931872 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:50:13.099029 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:13.099001 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:50:13.110173 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:13.110151 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:50:13.147842 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:13.147813 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:50:13.156118 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:13.156096 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:50:13.180419 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:13.180358 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:50:13.188317 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:13.188302 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:50:13.201343 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:13.201324 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:50:13.209905 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:13.209882 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:50:14.591381 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:14.591353 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:50:14.603611 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:14.603592 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:50:14.615110 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:14.615091 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:50:14.624333 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:14.624310 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:50:14.635734 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:14.635713 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:50:14.644783 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:14.644768 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:50:14.660999 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:14.660982 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:50:14.666094 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:14.666080 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:50:14.714692 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:14.714673 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:50:14.721132 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:14.721110 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:50:14.796802 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:14.796782 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:50:14.806115 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:14.806099 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:50:14.817568 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:14.817551 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:50:14.826621 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:14.826603 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:50:14.861752 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:14.861734 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:50:14.871106 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:14.871087 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:50:14.904877 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:14.904853 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:50:14.913383 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:14.913351 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:50:15.162936 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:15.162835 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:50:15.171015 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:15.170997 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:50:15.325487 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:15.325440 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:50:15.333967 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:15.333916 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:50:15.381600 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:15.381581 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:50:15.390329 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:15.390307 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:50:15.415291 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:15.415226 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:50:15.422885 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:15.422867 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:50:15.434966 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:15.434948 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:50:15.443198 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:15.443152 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:50:16.828887 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:16.828858 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:50:16.838342 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:16.838320 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:50:16.851754 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:16.851735 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:50:16.863024 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:16.863002 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:50:16.874938 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:16.874910 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:50:16.884133 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:16.884115 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:50:16.899405 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:16.899389 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:50:16.905675 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:16.905659 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:50:16.951132 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:16.951113 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:50:16.957189 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:16.957169 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:50:17.035743 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:17.035724 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:50:17.045509 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:17.045474 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:50:17.057373 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:17.057357 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:50:17.067602 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:17.067584 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:50:17.101953 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:17.101935 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:50:17.111212 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:17.111189 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:50:17.146829 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:17.146806 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:50:17.155907 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:17.155890 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:50:17.421929 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:17.421836 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:50:17.429154 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:17.429133 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:50:17.542682 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:17.542654 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:50:17.550682 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:17.550662 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:50:17.582720 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:17.582697 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:50:17.590991 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:17.590973 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:50:17.614780 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:17.614757 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:50:17.622682 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:17.622663 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:50:17.634901 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:17.634882 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:50:17.643095 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:17.643076 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:50:19.013979 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:19.013942 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:50:19.023159 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:19.023131 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:50:19.036189 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:19.036165 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:50:19.044872 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:19.044853 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:50:19.056088 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:19.056070 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:50:19.064570 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:19.064534 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:50:19.078860 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:19.078825 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:50:19.084497 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:19.084454 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:50:19.131663 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:19.131644 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:50:19.138843 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:19.138816 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:50:19.212776 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:19.212752 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:50:19.221994 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:19.221980 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:50:19.233396 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:19.233366 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:50:19.242982 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:19.242966 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:50:19.277826 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:19.277763 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:50:19.288634 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:19.288588 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:50:19.326363 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:19.326338 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:50:19.335793 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:19.335746 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:50:19.581430 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:19.581315 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:50:19.588521 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:19.588500 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:50:19.707616 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:19.707587 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:50:19.715470 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:19.715449 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:50:19.749104 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:19.749079 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:50:19.758425 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:19.758399 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:50:19.783639 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:19.783623 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:50:19.791100 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:19.791083 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:50:19.803023 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:19.803001 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:50:19.810552 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:19.810515 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:50:21.183447 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:21.183420 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:50:21.193049 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:21.193029 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:50:21.205353 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:21.205335 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:50:21.214908 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:21.214886 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:50:21.226568 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:21.226550 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:50:21.235577 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:21.235557 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:50:21.250909 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:21.250888 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:50:21.257264 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:21.257249 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:50:21.304064 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:21.304049 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:50:21.310338 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:21.310306 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:50:21.389300 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:21.389284 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:50:21.400971 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:21.400956 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:50:21.413866 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:21.413845 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:50:21.426171 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:21.426153 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:50:21.466021 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:21.465925 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:50:21.477665 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:21.477619 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:50:21.512483 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:21.512462 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:50:21.521787 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:21.521716 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:50:21.785750 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:21.785651 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:50:21.794004 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:21.793966 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:50:21.915082 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:21.915048 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:50:21.923816 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:21.923792 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:50:21.956959 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:21.956938 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:50:21.965613 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:21.965596 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:50:21.991563 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:21.991543 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:50:21.998583 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:21.998568 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:50:22.012219 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:22.012175 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:50:22.021187 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:22.021166 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:50:23.408256 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:23.408225 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:50:23.417013 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:23.416990 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:50:23.429161 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:23.429141 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:50:23.438347 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:23.438329 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:50:23.449344 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:23.449311 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:50:23.457555 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:23.457537 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:50:23.472273 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:23.472225 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:50:23.481342 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:23.481326 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:50:23.529339 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:23.529319 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:50:23.535812 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:23.535786 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:50:23.613695 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:23.613676 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:50:23.623001 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:23.622981 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:50:23.634556 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:23.634533 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:50:23.642737 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:23.642719 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:50:23.673530 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:23.673467 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:50:23.681926 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:23.681907 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:50:23.718050 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:23.718027 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:50:23.726939 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:23.726919 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:50:24.001476 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:24.001384 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:50:24.020386 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:24.020355 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:50:24.146713 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:24.146673 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:50:24.154445 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:24.154424 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:50:24.196048 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:24.196025 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:50:24.205196 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:24.205174 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:50:24.231941 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:24.231922 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:50:24.239468 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:24.239453 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:50:24.253562 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:24.253516 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:50:24.261412 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:24.261394 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:50:25.620194 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:25.620164 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:50:25.629519 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:25.629499 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:50:25.640447 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:25.640429 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:50:25.648951 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:25.648930 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:50:25.659943 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:25.659910 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:50:25.668471 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:25.668452 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:50:25.683086 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:25.683065 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:50:25.688872 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:25.688854 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:50:25.735188 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:25.735165 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:50:25.741672 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:25.741656 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:50:25.821333 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:25.821295 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:50:25.832079 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:25.832056 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:50:25.845570 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:25.845551 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:50:25.854425 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:25.854406 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:50:25.886061 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:25.885993 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:50:25.894392 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:25.894374 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:50:25.926902 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:25.926884 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:50:25.935909 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:25.935894 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:50:26.187881 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:26.187771 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:50:26.195154 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:26.195108 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:50:26.309979 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:26.309958 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:50:26.317949 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:26.317929 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:50:26.350195 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:26.350178 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:50:26.358266 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:26.358252 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:50:26.382065 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:26.382047 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:50:26.391069 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:26.391054 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:50:26.404662 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:26.404641 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:50:26.413194 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:26.413176 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:50:27.804310 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:27.804286 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:50:27.814668 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:27.814649 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:50:27.826111 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:27.826089 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:50:27.834872 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:27.834855 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:50:27.845993 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:27.845975 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:50:27.854592 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:27.854576 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:50:27.871136 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:27.871115 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:50:27.877013 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:27.876995 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:50:27.930730 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:27.930711 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:50:27.939432 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:27.939414 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:50:28.021967 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:28.021946 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:50:28.032334 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:28.032315 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:50:28.043795 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:28.043774 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:50:28.052641 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:28.052622 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:50:28.084598 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:28.084551 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:50:28.092989 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:28.092971 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:50:28.127991 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:28.127964 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:50:28.136459 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:28.136438 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:50:28.402903 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:28.402877 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:50:28.410333 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:28.410313 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:50:28.523940 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:28.523915 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:50:28.532125 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:28.532103 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:50:28.563985 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:28.563963 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:50:28.571695 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:28.571675 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:50:28.601421 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:28.601403 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:50:28.613956 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:28.613935 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:50:28.626171 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:28.626151 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:50:28.635576 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:28.635555 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:50:29.997094 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:29.997062 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:50:30.006451 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:30.006424 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:50:30.018210 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:30.018187 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:50:30.027499 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:30.027477 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:50:30.039013 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:30.038992 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:50:30.047693 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:30.047675 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:50:30.063048 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:30.063028 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:50:30.069490 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:30.069474 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:50:30.120454 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:30.120430 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:50:30.126530 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:30.126512 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:50:30.207968 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:30.207949 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:50:30.217176 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:30.217157 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:50:30.228734 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:30.228719 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:50:30.237918 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:30.237899 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:50:30.269741 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:30.269682 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:50:30.278148 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:30.278133 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:50:30.312433 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:30.312413 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:50:30.321147 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:30.321131 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:50:30.565675 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:30.565606 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:50:30.573570 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:30.573546 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:50:30.696620 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:30.696594 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:50:30.705367 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:30.705348 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:50:30.742107 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:30.742086 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:50:30.750775 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:30.750752 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:50:30.777841 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:30.777820 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:50:30.785881 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:30.785866 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:50:30.798297 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:30.798277 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:50:30.805996 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:30.805976 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:50:32.161701 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:32.161675 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:50:32.170627 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:32.170603 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:50:32.182605 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:32.182582 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:50:32.190796 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:32.190777 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:50:32.201941 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:32.201921 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:50:32.209945 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:32.209927 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:50:32.225035 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:32.225014 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:50:32.232075 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:32.232060 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:50:32.274598 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:32.274582 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:50:32.280061 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:32.280045 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:50:32.358261 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:32.358241 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:50:32.367362 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:32.367344 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:50:32.379366 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:32.379346 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:50:32.390957 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:32.390941 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:50:32.430370 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:32.430314 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:50:32.440991 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:32.440973 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:50:32.476396 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:32.476377 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:50:32.486264 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:32.486244 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:50:32.739074 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:32.739004 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:50:32.746277 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:32.746257 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:50:32.878579 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:32.878555 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:50:32.886525 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:32.886508 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:50:32.918156 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:32.918136 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:50:32.926807 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:32.926792 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:50:32.951681 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:32.951662 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:50:32.959755 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:32.959740 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:50:32.971687 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:32.971671 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:50:32.979267 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:32.979246 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:50:34.326089 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:34.326062 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:50:34.335593 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:34.335572 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:50:34.346880 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:34.346856 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:50:34.355999 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:34.355979 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:50:34.368238 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:34.368197 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:50:34.378040 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:34.378012 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:50:34.394867 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:34.394844 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:50:34.400858 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:34.400842 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:50:34.445098 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:34.445067 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:50:34.451128 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:34.451111 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:50:34.527739 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:34.527720 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:50:34.537060 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:34.537033 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:50:34.548374 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:34.548357 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:50:34.557384 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:34.557368 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:50:34.590137 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:34.590085 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:50:34.600383 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:34.600365 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:50:34.639340 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:34.639322 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:50:34.647882 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:34.647862 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:50:34.905853 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:34.905827 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:50:34.913846 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:34.913829 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:50:35.038340 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:35.038321 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:50:35.046390 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:35.046375 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:50:35.079056 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:35.079035 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:50:35.087163 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:35.087145 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:50:35.110544 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:35.110525 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:50:35.118164 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:35.118145 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:50:35.130609 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:35.130591 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:50:35.158170 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:35.158124 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:50:36.523599 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:36.523575 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:50:36.532883 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:36.532861 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:50:36.544613 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:36.544593 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:50:36.553810 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:36.553787 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:50:36.566611 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:36.566592 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:50:36.575306 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:36.575290 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:50:36.590509 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:36.590491 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:50:36.601809 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:36.601789 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:50:36.647517 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:36.647497 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:50:36.653704 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:36.653689 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:50:36.731790 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:36.731770 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:50:36.740410 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:36.740394 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:50:36.751251 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:36.751232 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:50:36.759606 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:36.759587 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:50:36.790585 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:36.790536 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:50:36.799535 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:36.799519 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:50:36.835112 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:36.835094 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:50:36.844576 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:36.844561 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:50:37.098812 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:37.098748 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:50:37.106048 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:37.106017 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:50:37.218408 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:37.218385 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:50:37.225902 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:37.225880 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:50:37.258146 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:37.258126 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:50:37.266077 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:37.266055 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:50:37.289976 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:37.289960 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:50:37.297148 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:37.297133 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:50:37.309032 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:37.309011 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:50:37.317122 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:37.317103 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:50:38.711752 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:38.711721 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:50:38.720954 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:38.720932 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:50:38.733560 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:38.733541 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:50:38.742459 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:38.742445 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:50:38.754317 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:38.754302 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:50:38.765647 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:38.765632 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:50:38.781621 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:38.781602 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:50:38.787050 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:38.787026 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:50:38.831243 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:38.831219 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:50:38.837225 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:38.837192 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:50:38.913266 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:38.913249 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:50:38.921978 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:38.921959 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:50:38.933238 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:38.933217 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:50:38.943654 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:38.943633 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:50:38.975855 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:38.975804 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:50:38.984635 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:38.984619 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:50:39.022522 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:39.022503 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:50:39.031903 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:39.031887 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:50:39.282292 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:39.282196 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:50:39.289594 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:39.289573 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:50:39.406639 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:39.406614 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:50:39.414422 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:39.414400 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:50:39.448354 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:39.448335 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:50:39.456309 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:39.456285 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:50:39.480035 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:39.480016 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:50:39.487220 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:39.487182 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:50:39.499146 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:39.499113 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:50:39.506745 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:39.506730 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:50:40.872398 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:40.872364 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:50:40.881748 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:40.881724 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:50:40.893511 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:40.893489 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:50:40.902769 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:40.902754 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:50:40.914463 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:40.914443 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:50:40.923197 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:40.923180 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:50:40.938264 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:40.938242 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:50:40.944535 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:40.944519 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:50:40.993099 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:40.993079 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:50:40.999020 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:40.999007 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:50:41.079769 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:41.079749 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:50:41.089059 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:41.089038 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:50:41.100696 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:41.100680 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:50:41.109921 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:41.109903 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:50:41.151959 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:41.151904 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:50:41.161186 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:41.161169 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:50:41.196243 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:41.196222 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:50:41.205812 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:41.205796 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:50:41.463424 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:41.463349 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:50:41.473652 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:41.473633 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:50:41.597133 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:41.597104 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:50:41.606035 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:41.606014 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:50:41.638126 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:41.638106 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:50:41.646266 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:41.646247 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:50:41.670095 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:41.670077 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:50:41.678932 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:41.678910 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:50:41.697954 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:41.693569 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:50:41.704688 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:41.704663 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:50:43.085349 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:43.085318 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:50:43.093880 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:43.093854 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:50:43.104560 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:43.104540 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:50:43.113118 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:43.113101 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:50:43.123811 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:43.123789 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:50:43.132307 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:43.132289 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:50:43.146330 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:43.146310 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:50:43.152099 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:43.152081 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:50:43.199336 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:43.199312 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:50:43.204820 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:43.204802 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:50:43.275100 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:43.275077 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:50:43.283597 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:43.283580 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:50:43.294802 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:43.294788 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:50:43.303008 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:43.302988 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:50:43.333399 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:43.333381 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:50:43.341935 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:43.341881 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:50:43.375015 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:43.374993 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:50:43.384051 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:43.384033 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:50:43.633003 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:43.632977 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:50:43.639580 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:43.639551 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:50:43.757129 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:43.757102 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:50:43.765888 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:43.765867 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:50:43.800321 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:43.800302 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:50:43.809872 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:43.809848 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:50:43.832897 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:43.832875 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:50:43.841428 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:43.841410 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:50:43.853413 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:43.853393 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:50:43.860769 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:43.860750 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:50:45.221600 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:45.221571 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:50:45.233050 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:45.233030 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:50:45.245646 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:45.245630 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:50:45.254499 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:45.254480 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:50:45.266161 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:45.266140 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:50:45.275016 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:45.274996 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:50:45.312990 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:45.312973 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:50:45.318657 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:45.318638 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:50:45.365906 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:45.365883 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:50:45.373123 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:45.373102 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:50:45.447969 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:45.447950 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:50:45.456617 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:45.456595 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:50:45.467782 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:45.467758 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:50:45.476460 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:45.476401 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:50:45.509372 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:45.509353 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:50:45.517490 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:45.517472 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:50:45.550469 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:45.550449 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:50:45.559344 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:45.559325 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:50:45.808367 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:45.808293 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:50:45.815331 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:45.815301 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:50:45.937091 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:45.937061 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:50:45.945834 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:45.945815 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:50:45.978652 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:45.978633 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:50:45.986656 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:45.986638 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:50:46.010162 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:46.010142 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:50:46.018551 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:46.018533 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:50:46.031977 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:46.031957 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:50:46.040525 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:46.040508 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:50:47.408134 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:47.408106 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:50:47.420186 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:47.420164 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:50:47.432918 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:47.432903 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:50:47.441972 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:47.441951 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:50:47.453685 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:47.453667 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:50:47.462185 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:47.462169 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:50:47.480049 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:47.480031 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:50:47.486485 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:47.486469 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:50:47.532841 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:47.532821 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:50:47.538506 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:47.538489 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:50:47.612026 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:47.611996 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:50:47.620899 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:47.620877 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:50:47.632335 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:47.632318 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:50:47.640776 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:47.640761 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:50:47.673235 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:47.673145 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:50:47.681625 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:47.681607 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:50:47.715608 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:47.715587 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:50:47.725101 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:47.725069 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:50:47.979086 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:47.978998 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:50:47.986713 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:47.986694 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:50:48.145567 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:48.145537 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:50:48.156773 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:48.156754 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:50:48.189513 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:48.189488 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:50:48.197805 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:48.197782 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:50:48.222087 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:48.222066 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:50:48.229657 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:48.229602 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:50:48.246299 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:48.246278 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:50:48.254162 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:48.254129 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:50:49.625070 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:49.625044 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:50:49.634318 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:49.634297 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:50:49.646160 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:49.646143 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:50:49.655348 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:49.655332 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:50:49.666660 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:49.666646 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:50:49.675483 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:49.675464 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:50:49.690771 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:49.690751 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:50:49.695847 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:49.695830 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:50:49.741055 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:49.741035 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:50:49.746964 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:49.746944 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:50:49.822054 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:49.822034 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:50:49.830576 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:49.830561 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:50:49.842223 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:49.842187 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:50:49.850553 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:49.850538 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:50:49.881922 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:49.881866 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:50:49.890543 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:49.890528 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:50:49.928012 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:49.927993 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:50:49.936692 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:49.936672 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:50:50.180882 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:50.180808 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:50:50.188530 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:50.188509 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:50:50.303322 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:50.303298 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:50:50.311351 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:50.311330 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:50:50.348930 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:50.348908 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:50:50.356741 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:50.356724 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:50:50.380282 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:50.380267 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:50:50.388860 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:50.388844 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:50:50.401100 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:50.401073 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:50:50.408688 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:50.408670 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:50:51.772431 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:51.772393 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:50:51.782685 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:51.782654 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:50:51.794871 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:51.794853 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:50:51.804519 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:51.804499 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:50:51.816137 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:51.816118 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:50:51.828980 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:51.828965 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:50:51.854972 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:51.854952 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:50:51.868076 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:51.868058 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:50:51.923853 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:51.923834 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:50:51.936902 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:51.936884 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:50:52.041325 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:52.041256 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:50:52.052955 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:52.052932 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:50:52.067931 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:52.067908 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:50:52.079974 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:52.079952 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:50:52.127185 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:52.127165 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:50:52.142972 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:52.142950 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:50:52.181748 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:52.181720 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:50:52.190798 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:52.190777 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:50:52.505247 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:52.505191 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:50:52.512389 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:52.512369 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:50:52.627039 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:52.627014 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:50:52.635012 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:52.634988 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:50:52.673380 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:52.673358 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:50:52.681275 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:52.681259 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:50:52.706126 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:52.706103 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:50:52.713673 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:52.713659 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:50:52.725486 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:52.725462 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:50:52.734674 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:52.734658 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:50:54.085544 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:54.085511 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:50:54.094100 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:54.094082 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:50:54.104971 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:54.104952 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:50:54.113286 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:54.113268 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:50:54.124429 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:54.124413 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:50:54.132609 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:54.132589 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:50:54.149038 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:54.149016 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:50:54.154581 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:54.154566 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:50:54.201008 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:54.200989 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:50:54.206375 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:54.206349 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:50:54.276421 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:54.276406 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:50:54.285756 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:54.285741 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:50:54.298930 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:54.298915 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:50:54.306703 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:54.306686 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:50:54.335726 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:54.335660 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:50:54.348087 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:54.348071 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:50:54.380438 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:54.380410 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:50:54.388695 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:54.388671 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:50:54.628994 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:54.628968 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:50:54.635824 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:54.635804 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:50:54.753414 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:54.753395 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:50:54.761269 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:54.761249 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:50:54.795406 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:54.795386 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:50:54.803328 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:54.803313 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:50:54.827941 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:54.827917 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:50:54.835111 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:54.835095 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:50:54.847799 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:54.847778 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:50:54.862057 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:54.862032 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:50:56.220811 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:56.220777 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:50:56.230195 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:56.230174 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:50:56.241435 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:56.241415 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:50:56.250008 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:56.249992 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:50:56.261457 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:56.261434 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:50:56.270492 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:56.270473 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:50:56.284889 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:56.284868 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:50:56.291224 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:56.291182 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:50:56.339559 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:56.339544 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:50:56.345580 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:56.345565 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:50:56.422097 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:56.422080 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:50:56.430873 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:56.430847 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:50:56.441934 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:56.441919 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:50:56.450666 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:56.450653 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:50:56.482455 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:56.482402 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:50:56.491381 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:56.491356 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:50:56.527647 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:56.527624 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:50:56.536208 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:56.536189 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:50:56.784100 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:56.784017 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:50:56.791134 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:56.791114 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:50:56.904809 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:56.904782 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:50:56.912916 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:56.912900 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:50:56.945406 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:56.945383 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:50:56.953429 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:56.953408 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:50:56.976716 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:56.976694 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:50:56.984914 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:56.984893 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:50:56.999722 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:56.999702 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:50:57.007510 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:57.007494 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:50:58.371978 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:58.371948 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:50:58.381390 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:58.381368 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:50:58.393109 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:58.393072 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:50:58.401374 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:58.401352 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:50:58.417060 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:58.417043 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:50:58.425904 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:58.425888 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:50:58.441700 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:58.441677 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:50:58.450069 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:58.450049 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:50:58.495024 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:58.495003 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:50:58.501018 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:58.500998 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:50:58.575829 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:58.575812 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:50:58.584531 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:58.584512 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:50:58.595411 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:58.595382 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:50:58.605583 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:58.605558 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:50:58.637712 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:58.637656 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:50:58.646266 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:58.646249 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:50:58.681882 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:58.681859 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:50:58.690638 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:58.690618 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:50:58.932244 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:58.932167 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:50:58.939937 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:58.939918 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:50:59.068516 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:59.068498 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:50:59.076620 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:59.076602 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:50:59.111538 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:59.111522 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:50:59.119806 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:59.119787 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:50:59.146407 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:59.146386 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:50:59.153869 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:59.153853 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:50:59.166073 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:59.166053 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:50:59.174055 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:50:59.174037 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:51:00.538534 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:00.538504 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:51:00.547323 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:00.547301 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:51:00.559074 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:00.559056 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:51:00.566935 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:00.566917 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:51:00.578377 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:00.578357 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:51:00.608249 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:00.608227 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:51:00.622910 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:00.622876 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:51:00.627858 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:00.627842 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:51:00.674410 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:00.674391 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:51:00.679494 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:00.679476 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:51:00.745888 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:00.745868 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:51:00.753908 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:00.753877 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:51:00.764901 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:00.764865 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:51:00.772560 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:00.772544 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:51:00.801444 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:00.801395 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:51:00.809308 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:00.809285 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:51:00.841441 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:00.841427 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:51:00.849499 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:00.849480 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:51:01.082681 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:01.082618 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:51:01.089841 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:01.089823 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:51:01.199589 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:01.199561 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:51:01.206923 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:01.206899 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:51:01.237289 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:01.237268 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:51:01.244642 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:01.244625 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:51:01.266569 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:01.266554 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:51:01.273096 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:01.273071 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:51:01.284731 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:01.284713 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:51:01.291875 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:01.291854 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:51:02.633353 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:02.633327 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:51:02.643182 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:02.643161 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:51:02.655553 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:02.655534 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:51:02.664971 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:02.664951 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:51:02.676824 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:02.676802 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:51:02.685793 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:02.685778 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:51:02.701166 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:02.701146 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:51:02.707869 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:02.707854 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:51:02.755754 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:02.755734 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:51:02.762224 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:02.762194 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:51:02.843450 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:02.843433 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:51:02.852574 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:02.852558 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:51:02.863773 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:02.863756 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:51:02.872577 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:02.872558 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:51:02.903811 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:02.903763 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:51:02.912743 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:02.912724 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:51:02.946346 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:02.946329 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:51:02.955459 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:02.955435 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:51:03.209994 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:03.209915 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:51:03.217928 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:03.217902 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:51:03.344763 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:03.344735 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:51:03.353109 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:03.353086 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:51:03.387049 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:03.387023 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:51:03.403443 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:03.403425 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:51:03.428889 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:03.428872 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:51:03.436854 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:03.436834 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:51:03.451664 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:03.451646 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:51:03.460065 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:03.460014 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:51:04.812005 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:04.811978 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:51:04.820795 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:04.820774 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:51:04.832461 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:04.832439 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:51:04.841463 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:04.841448 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:51:04.852922 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:04.852896 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:51:04.861421 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:04.861399 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:51:04.876536 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:04.876513 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:51:04.883313 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:04.883290 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:51:04.928966 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:04.928942 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:51:04.934114 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:04.934099 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:51:05.006134 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:05.006114 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:51:05.014974 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:05.014956 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:51:05.025929 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:05.025909 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:51:05.033901 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:05.033883 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:51:05.065184 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:05.065125 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:51:05.073611 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:05.073595 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:51:05.109604 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:05.109586 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:51:05.118821 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:05.118802 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:51:05.379244 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:05.379222 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:51:05.386109 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:05.386092 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:51:05.505669 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:05.505648 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:51:05.513281 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:05.513257 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:51:05.543965 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:05.543945 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:51:05.552106 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:05.552086 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:51:05.574791 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:05.574776 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:51:05.582031 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:05.582011 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:51:05.593475 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:05.593456 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:51:05.601000 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:05.600981 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:51:06.943612 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:06.943586 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:51:06.953296 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:06.953266 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:51:06.964983 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:06.964955 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:51:06.973666 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:06.973649 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:51:06.985483 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:06.985464 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:51:06.994226 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:06.994192 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:51:07.009728 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:07.009712 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:51:07.015874 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:07.015858 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:51:07.073890 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:07.073862 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:51:07.083164 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:07.083142 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:51:07.157179 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:07.157155 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:51:07.165739 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:07.165712 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:51:07.177375 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:07.177356 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:51:07.187367 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:07.187348 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:51:07.218476 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:07.218407 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:51:07.226948 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:07.226932 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:51:07.261412 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:07.261390 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:51:07.270235 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:07.270192 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:51:07.527977 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:07.527912 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:51:07.535359 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:07.535335 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:51:07.659343 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:07.659319 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:51:07.667294 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:07.667274 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:51:07.699505 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:07.699484 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:51:07.707592 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:07.707560 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:51:07.733296 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:07.733277 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:51:07.746999 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:07.746983 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:51:07.760118 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:07.760103 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:51:07.768628 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:07.768610 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:51:09.148005 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:09.147978 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:51:09.157064 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:09.157045 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:51:09.168392 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:09.168369 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:51:09.177431 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:09.177410 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:51:09.189304 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:09.189284 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:51:09.198220 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:09.198183 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:51:09.212934 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:09.212908 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:51:09.219722 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:09.219706 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:51:09.266955 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:09.266934 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:51:09.272980 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:09.272962 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:51:09.348970 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:09.348954 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:51:09.357102 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:09.357088 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:51:09.368034 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:09.368017 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:51:09.376280 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:09.376263 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:51:09.407250 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:09.407177 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:51:09.417829 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:09.417813 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:51:09.453568 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:09.453542 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:51:09.461761 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:09.461747 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:51:09.711914 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:09.711855 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:51:09.718549 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:09.718527 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:51:09.837911 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:09.837886 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:51:09.845645 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:09.845620 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:51:09.881506 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:09.881482 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:51:09.889014 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:09.888995 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:51:09.912875 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:09.912845 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:51:09.920906 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:09.920886 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:51:09.933997 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:09.933977 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:51:09.942259 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:09.942238 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:51:11.369075 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:11.369046 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:51:11.378140 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:11.378122 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:51:11.389958 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:11.389941 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:51:11.400696 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:11.400680 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:51:11.412325 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:11.412303 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:51:11.421292 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:11.421274 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:51:11.436659 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:11.436632 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:51:11.442484 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:11.442470 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:51:11.491429 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:11.491414 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:51:11.497855 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:11.497842 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:51:11.577421 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:11.577407 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:51:11.587073 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:11.587060 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:51:11.599421 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:11.599403 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:51:11.608952 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:11.608932 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:51:11.642363 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:11.642307 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:51:11.651292 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:11.651274 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:51:11.686693 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:11.686673 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:51:11.695590 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:11.695571 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:51:11.956830 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:11.956755 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:51:11.964892 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:11.964868 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:51:12.090630 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:12.090599 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:51:12.098951 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:12.098927 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:51:12.140308 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:12.140287 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:51:12.152302 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:12.152283 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:51:12.178702 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:12.178678 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:51:12.186450 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:12.186426 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:51:12.200566 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:12.200551 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:51:12.209885 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:12.209833 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:51:13.599332 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:13.599306 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:51:13.610031 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:13.610010 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:51:13.622591 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:13.622571 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:51:13.632385 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:13.632365 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:51:13.645870 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:13.645847 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:51:13.656727 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:13.656709 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:51:13.673718 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:13.673699 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:51:13.682327 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:13.682310 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:51:13.739789 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:13.739769 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:51:13.746482 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:13.746460 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:51:13.832559 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:13.832540 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:51:13.842329 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:13.842313 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:51:13.854482 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:13.854462 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:51:13.864957 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:13.864939 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:51:13.905605 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:13.905585 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:51:13.916479 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:13.916461 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:51:13.957542 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:13.957520 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:51:13.971988 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:13.971969 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:51:14.337937 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:14.337862 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:51:14.347325 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:14.347307 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:51:14.492680 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:14.492655 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:51:14.502387 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:14.502362 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:51:14.547290 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:14.547266 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:51:14.558222 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:14.558173 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:51:14.588576 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:14.588520 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:51:14.601637 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:14.601596 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:51:14.616050 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:14.616028 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:51:14.625528 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:14.625512 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:51:16.046635 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:16.046608 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:51:16.055952 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:16.055930 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:51:16.067450 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:16.067433 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:51:16.076977 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:16.076959 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:51:16.088932 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:16.088916 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:51:16.099718 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:16.099702 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:51:16.117563 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:16.117538 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:51:16.123925 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:16.123910 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:51:16.169008 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:16.168986 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:51:16.175360 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:16.175344 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:51:16.257558 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:16.257540 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:51:16.266417 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:16.266396 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:51:16.278393 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:16.278375 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:51:16.287664 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:16.287647 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:51:16.323293 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:16.323240 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:51:16.333326 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:16.333306 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:51:16.368796 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:16.368777 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:51:16.378164 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:16.378146 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:51:16.649577 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:16.649552 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:51:16.657477 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:16.657456 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:51:16.790073 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:16.785050 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:51:16.796377 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:16.796354 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:51:16.832846 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:16.832820 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:51:16.843349 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:16.843331 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:51:16.872592 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:16.872571 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:51:16.880494 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:16.880475 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:51:16.894375 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:16.894355 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:51:16.903319 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:16.903254 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:51:18.339679 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:18.339653 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:51:18.349378 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:18.349355 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:51:18.361711 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:18.361690 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:51:18.372927 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:18.372911 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:51:18.384760 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:18.384727 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:51:18.393725 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:18.393705 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:51:18.410281 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:18.410240 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:51:18.416846 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:18.416821 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:51:18.463573 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:18.463551 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:51:18.470410 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:18.470390 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:51:18.545342 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:18.545324 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:51:18.554459 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:18.554444 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:51:18.567032 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:18.567015 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:51:18.576322 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:18.576302 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:51:18.614385 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:18.614363 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:51:18.623701 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:18.623684 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:51:18.659016 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:18.658996 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:51:18.669302 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:18.669280 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:51:18.939269 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:18.939185 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:51:18.948980 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:18.948957 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:51:19.081033 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:19.081012 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:51:19.089606 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:19.089581 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:51:19.124698 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:19.124660 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:51:19.133143 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:19.133126 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:51:19.159499 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:19.159466 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:51:19.166781 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:19.166765 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:51:19.179925 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:19.179906 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:51:19.190939 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:19.190848 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:51:20.652339 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:20.652315 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:51:20.667396 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:20.667373 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:51:20.678874 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:20.678853 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:51:20.687847 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:20.687808 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:51:20.699059 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:20.699043 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:51:20.707693 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:20.707673 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:51:20.722660 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:20.722640 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:51:20.728567 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:20.728553 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:51:20.773783 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:20.773763 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:51:20.779996 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:20.779980 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:51:20.856953 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:20.856930 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:51:20.865854 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:20.865833 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:51:20.877534 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:20.877513 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:51:20.886584 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:20.886569 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:51:20.919876 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:20.919809 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:51:20.928624 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:20.928606 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:51:20.962438 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:20.962417 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:51:20.970957 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:20.970930 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:51:21.226601 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:21.226530 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:51:21.234049 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:21.234033 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:51:21.350187 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:21.350163 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:51:21.357994 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:21.357976 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:51:21.390944 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:21.390923 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:51:21.399425 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:21.399407 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:51:21.425126 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:21.425107 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:51:21.432491 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:21.432471 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:51:21.444565 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:21.444547 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:51:21.452041 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:21.452025 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:51:22.859434 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:22.859408 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:51:22.868787 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:22.868762 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:51:22.880675 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:22.880655 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:51:22.889927 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:22.889909 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:51:22.902330 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:22.902311 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:51:22.924134 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:22.924117 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:51:22.940501 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:22.940478 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:51:22.946239 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:22.946224 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:51:22.994658 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:22.994629 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:51:23.003067 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:23.003044 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:51:23.078612 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:23.078590 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:51:23.087886 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:23.087866 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:51:23.099281 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:23.099263 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:51:23.108727 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:23.108709 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:51:23.142702 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:23.142654 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:51:23.151690 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:23.151673 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:51:23.194413 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:23.194394 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:51:23.204712 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:23.204680 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:51:23.475603 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:23.473178 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:51:23.480360 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:23.480332 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:51:23.611194 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:23.611170 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:51:23.619013 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:23.618996 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:51:23.655129 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:23.655109 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:51:23.664120 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:23.664100 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:51:23.691339 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:23.691322 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:51:23.699092 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:23.699072 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:51:23.712917 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:23.712899 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:51:23.721257 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:23.721240 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:51:25.136787 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:25.136754 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:51:25.145632 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:25.145610 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:51:25.157921 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:25.157904 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:51:25.167513 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:25.167495 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:51:25.178713 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:25.178691 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:51:25.187677 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:25.187659 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:51:25.203756 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:25.203741 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:51:25.208861 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:25.208844 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:51:25.254545 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:25.254526 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:51:25.260330 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:25.260313 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:51:25.351534 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:25.351515 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:51:25.360629 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:25.360610 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:51:25.371804 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:25.371788 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:51:25.380557 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:25.380539 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:51:25.412467 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:25.412408 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:51:25.420780 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:25.420764 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:51:25.455123 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:25.455104 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:51:25.463964 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:25.463948 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:51:25.731672 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:25.731575 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:51:25.738361 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:25.738330 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:51:25.856297 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:25.856271 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:51:25.864296 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:25.864278 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:51:25.898992 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:25.898971 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:51:25.907883 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:25.907859 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:51:25.935162 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:25.935146 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:51:25.943425 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:25.943408 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:51:25.955954 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:25.955932 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:51:25.964074 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:25.964045 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:51:27.347711 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:27.347683 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/kserve-container/0.log" Apr 16 17:51:27.357788 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:27.357762 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-predictor-78c8d484d6-hrnjg_2f62a498-f7fb-46e3-b63a-9e7cee1d3375/storage-initializer/0.log" Apr 16 17:51:27.370024 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:27.370000 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/kserve-container/0.log" Apr 16 17:51:27.380586 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:27.380564 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-runtime-predictor-64984c7cb-jwfk4_902aabef-a060-4f86-9c71-5913e069001e/storage-initializer/0.log" Apr 16 17:51:27.394105 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:27.394085 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/kserve-container/0.log" Apr 16 17:51:27.406104 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:27.406055 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-kserve-predictor-68fc4f5b49-q4wtp_312dd04f-11ce-4a90-afea-d471128c9fd3/storage-initializer/0.log" Apr 16 17:51:27.422310 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:27.422293 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/kserve-container/0.log" Apr 16 17:51:27.428294 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:27.428275 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-lightgbm-v2-runtime-predictor-8d44c64dc-d5l4v_34163a51-b7e0-40f5-aa47-7a39addab1a4/storage-initializer/0.log" Apr 16 17:51:27.474859 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:27.474833 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/kserve-container/0.log" Apr 16 17:51:27.480785 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:27.480771 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-mlflow-v2-runtime-predictor-5497fc5f68-m4c55_21c509c7-82b6-4e6e-af25-b691fc24a898/storage-initializer/0.log" Apr 16 17:51:27.550098 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:27.550082 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/kserve-container/0.log" Apr 16 17:51:27.558356 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:27.558340 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-predictor-5584ffd8c9-zvfhk_b09b7f8f-a1fa-4062-8de9-a929f3af5713/storage-initializer/0.log" Apr 16 17:51:27.571759 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:27.571743 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/kserve-container/0.log" Apr 16 17:51:27.580237 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:27.580220 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-pmml-runtime-predictor-7576f6b69f-7dqpk_15fdc27b-a100-4955-96f3-a033027e5c44/storage-initializer/0.log" Apr 16 17:51:27.611723 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:27.611707 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/kserve-container/0.log" Apr 16 17:51:27.620429 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:27.620412 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-lightgbm-predictor-669896799c-54xhl_fbeb89a1-7819-4f6c-b1e4-f25fbf54b19b/storage-initializer/0.log" Apr 16 17:51:27.653132 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:27.653105 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/kserve-container/0.log" Apr 16 17:51:27.661890 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:27.661865 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-predictive-sklearn-predictor-85bccb8945-5ppfc_f2948b8d-7d4a-4c39-83e2-09cf271d48d1/storage-initializer/0.log" Apr 16 17:51:27.918589 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:27.918526 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/kserve-container/0.log" Apr 16 17:51:27.925599 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:27.925578 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-sklearn-v2-runtime-predictor-7dd9b85c64-4qjt2_059eab9b-8d48-4612-a432-a19551e35a31/storage-initializer/0.log" Apr 16 17:51:28.040450 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:28.040407 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/kserve-container/0.log" Apr 16 17:51:28.047911 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:28.047886 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-mlserver-predictor-58b7db6668-h8q77_59ee056f-54da-42d8-8925-48fa8e215f7a/storage-initializer/0.log" Apr 16 17:51:28.080022 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:28.080001 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/kserve-container/0.log" Apr 16 17:51:28.087693 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:28.087673 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_isvc-xgboost-v2-runtime-predictor-b5d6966c7-87qdg_13d15bf0-ed3e-456a-b5f5-ee62c25e3220/storage-initializer/0.log" Apr 16 17:51:28.111519 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:28.111497 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/kserve-container/0.log" Apr 16 17:51:28.118776 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:28.118758 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_sklearn-v2-mlserver-predictor-695d5f5568-fvc2w_6e510c2a-a7c8-4c1a-8754-018a372a6279/storage-initializer/0.log" Apr 16 17:51:28.131121 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:28.131104 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/kserve-container/0.log" Apr 16 17:51:28.138747 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:28.138733 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_xgboost-v2-mlserver-predictor-c58d48f-f64lm_306d7e24-f7b2-4a97-9e1d-8ded6e6ef290/storage-initializer/0.log" Apr 16 17:51:33.920956 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:33.920919 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mtpht_43838dea-81df-4027-b3a7-1f13ebc11e45/global-pull-secret-syncer/0.log" Apr 16 17:51:34.081041 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:34.081011 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wtvc6_b7ff6bde-0c70-4148-bec9-b41af34740f1/konnectivity-agent/0.log" Apr 16 17:51:34.148167 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:34.148142 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-49.ec2.internal_c0e6e3b61a3e8cfc6394e27b3b2ac0f3/haproxy/0.log" Apr 16 17:51:37.779426 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:37.779398 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-54997c8455-sxv9m_616efe3e-e6dd-427e-b9ee-db6ac7365ae8/metrics-server/0.log" Apr 16 17:51:37.959762 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:37.959724 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q9hmc_196eabc2-d9ba-4b5e-987e-d3ba87cc1037/node-exporter/0.log" Apr 16 17:51:37.978337 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:37.978304 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q9hmc_196eabc2-d9ba-4b5e-987e-d3ba87cc1037/kube-rbac-proxy/0.log" Apr 16 17:51:37.996582 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:37.996564 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q9hmc_196eabc2-d9ba-4b5e-987e-d3ba87cc1037/init-textfile/0.log" Apr 16 17:51:38.019135 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:38.019113 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-5669946b84-dzpt2_29fd6fae-df7e-47f5-b4cb-d7102b1b13ac/kube-rbac-proxy-main/0.log" Apr 16 17:51:38.037964 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:38.037909 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-5669946b84-dzpt2_29fd6fae-df7e-47f5-b4cb-d7102b1b13ac/kube-rbac-proxy-self/0.log" Apr 16 17:51:38.057326 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:38.057307 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-5669946b84-dzpt2_29fd6fae-df7e-47f5-b4cb-d7102b1b13ac/openshift-state-metrics/0.log" Apr 16 17:51:40.103001 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:40.102964 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/1.log" Apr 16 17:51:40.108224 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:40.108190 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-d87b8d5fc-s6s57_3e7b3fa7-ba27-4076-b195-19d14c437d6a/console-operator/2.log" Apr 16 17:51:40.511518 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:40.511445 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-66488bc844-rqprq_7881d457-3a14-41d2-879b-0e812fe08ef2/console/0.log" Apr 16 17:51:40.542987 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:40.542966 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-586b57c7b4-4pm8j_5a9d593a-f4f5-4786-849a-1330254f9b1c/download-server/0.log" Apr 16 17:51:40.937445 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:40.937397 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-7d955d5dd4-ss8kp_c1529d61-c793-4fc5-977d-5e976290f774/volume-data-source-validator/0.log" Apr 16 17:51:41.202555 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:41.202468 2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dtnls/perf-node-gather-daemonset-rd72q"] Apr 16 17:51:41.206540 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:41.206516 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dtnls/perf-node-gather-daemonset-rd72q" Apr 16 17:51:41.209088 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:41.209056 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dtnls\"/\"openshift-service-ca.crt\"" Apr 16 17:51:41.209194 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:41.209107 2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dtnls\"/\"kube-root-ca.crt\"" Apr 16 17:51:41.209194 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:41.209158 2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dtnls\"/\"default-dockercfg-zt7d5\"" Apr 16 17:51:41.212998 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:41.212977 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dtnls/perf-node-gather-daemonset-rd72q"] Apr 16 17:51:41.353687 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:41.353661 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/60c2ea57-e9b9-49f9-9ac2-d2b141271bf1-lib-modules\") pod \"perf-node-gather-daemonset-rd72q\" (UID: \"60c2ea57-e9b9-49f9-9ac2-d2b141271bf1\") " pod="openshift-must-gather-dtnls/perf-node-gather-daemonset-rd72q" Apr 16 17:51:41.353825 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:41.353714 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8s9r\" (UniqueName: \"kubernetes.io/projected/60c2ea57-e9b9-49f9-9ac2-d2b141271bf1-kube-api-access-r8s9r\") pod \"perf-node-gather-daemonset-rd72q\" (UID: \"60c2ea57-e9b9-49f9-9ac2-d2b141271bf1\") " pod="openshift-must-gather-dtnls/perf-node-gather-daemonset-rd72q" Apr 16 17:51:41.353825 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:41.353749 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/60c2ea57-e9b9-49f9-9ac2-d2b141271bf1-podres\") pod \"perf-node-gather-daemonset-rd72q\" (UID: \"60c2ea57-e9b9-49f9-9ac2-d2b141271bf1\") " pod="openshift-must-gather-dtnls/perf-node-gather-daemonset-rd72q" Apr 16 17:51:41.353825 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:41.353777 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/60c2ea57-e9b9-49f9-9ac2-d2b141271bf1-sys\") pod \"perf-node-gather-daemonset-rd72q\" (UID: \"60c2ea57-e9b9-49f9-9ac2-d2b141271bf1\") " pod="openshift-must-gather-dtnls/perf-node-gather-daemonset-rd72q" Apr 16 17:51:41.353825 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:41.353801 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/60c2ea57-e9b9-49f9-9ac2-d2b141271bf1-proc\") pod \"perf-node-gather-daemonset-rd72q\" (UID: \"60c2ea57-e9b9-49f9-9ac2-d2b141271bf1\") " pod="openshift-must-gather-dtnls/perf-node-gather-daemonset-rd72q" Apr 16 17:51:41.454282 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:41.454190 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r8s9r\" (UniqueName: \"kubernetes.io/projected/60c2ea57-e9b9-49f9-9ac2-d2b141271bf1-kube-api-access-r8s9r\") pod \"perf-node-gather-daemonset-rd72q\" (UID: \"60c2ea57-e9b9-49f9-9ac2-d2b141271bf1\") " pod="openshift-must-gather-dtnls/perf-node-gather-daemonset-rd72q" Apr 16 17:51:41.454282 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:41.454256 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/60c2ea57-e9b9-49f9-9ac2-d2b141271bf1-podres\") pod \"perf-node-gather-daemonset-rd72q\" (UID: \"60c2ea57-e9b9-49f9-9ac2-d2b141271bf1\") " pod="openshift-must-gather-dtnls/perf-node-gather-daemonset-rd72q" Apr 16 17:51:41.454463 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:41.454281 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/60c2ea57-e9b9-49f9-9ac2-d2b141271bf1-sys\") pod \"perf-node-gather-daemonset-rd72q\" (UID: \"60c2ea57-e9b9-49f9-9ac2-d2b141271bf1\") " pod="openshift-must-gather-dtnls/perf-node-gather-daemonset-rd72q" Apr 16 17:51:41.454463 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:41.454309 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/60c2ea57-e9b9-49f9-9ac2-d2b141271bf1-proc\") pod \"perf-node-gather-daemonset-rd72q\" (UID: \"60c2ea57-e9b9-49f9-9ac2-d2b141271bf1\") " pod="openshift-must-gather-dtnls/perf-node-gather-daemonset-rd72q" Apr 16 17:51:41.454463 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:41.454341 2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/60c2ea57-e9b9-49f9-9ac2-d2b141271bf1-lib-modules\") pod \"perf-node-gather-daemonset-rd72q\" (UID: \"60c2ea57-e9b9-49f9-9ac2-d2b141271bf1\") " pod="openshift-must-gather-dtnls/perf-node-gather-daemonset-rd72q" Apr 16 17:51:41.454463 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:41.454370 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/60c2ea57-e9b9-49f9-9ac2-d2b141271bf1-podres\") pod \"perf-node-gather-daemonset-rd72q\" (UID: \"60c2ea57-e9b9-49f9-9ac2-d2b141271bf1\") " pod="openshift-must-gather-dtnls/perf-node-gather-daemonset-rd72q" Apr 16 17:51:41.454463 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:41.454401 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/60c2ea57-e9b9-49f9-9ac2-d2b141271bf1-sys\") pod \"perf-node-gather-daemonset-rd72q\" (UID: \"60c2ea57-e9b9-49f9-9ac2-d2b141271bf1\") " pod="openshift-must-gather-dtnls/perf-node-gather-daemonset-rd72q" Apr 16 17:51:41.454463 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:41.454405 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/60c2ea57-e9b9-49f9-9ac2-d2b141271bf1-proc\") pod \"perf-node-gather-daemonset-rd72q\" (UID: \"60c2ea57-e9b9-49f9-9ac2-d2b141271bf1\") " pod="openshift-must-gather-dtnls/perf-node-gather-daemonset-rd72q" Apr 16 17:51:41.454463 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:41.454464 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/60c2ea57-e9b9-49f9-9ac2-d2b141271bf1-lib-modules\") pod \"perf-node-gather-daemonset-rd72q\" (UID: \"60c2ea57-e9b9-49f9-9ac2-d2b141271bf1\") " pod="openshift-must-gather-dtnls/perf-node-gather-daemonset-rd72q" Apr 16 17:51:41.463068 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:41.463041 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8s9r\" (UniqueName: \"kubernetes.io/projected/60c2ea57-e9b9-49f9-9ac2-d2b141271bf1-kube-api-access-r8s9r\") pod \"perf-node-gather-daemonset-rd72q\" (UID: \"60c2ea57-e9b9-49f9-9ac2-d2b141271bf1\") " pod="openshift-must-gather-dtnls/perf-node-gather-daemonset-rd72q" Apr 16 17:51:41.517719 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:41.517695 2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dtnls/perf-node-gather-daemonset-rd72q" Apr 16 17:51:41.570595 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:41.570571 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gk5x8_430f1690-8090-4eaa-903e-a5aab4c24146/dns/0.log" Apr 16 17:51:41.590668 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:41.590646 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gk5x8_430f1690-8090-4eaa-903e-a5aab4c24146/kube-rbac-proxy/0.log" Apr 16 17:51:41.648470 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:41.648446 2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dtnls/perf-node-gather-daemonset-rd72q"] Apr 16 17:51:41.650871 ip-10-0-140-49 kubenswrapper[2574]: W0416 17:51:41.650839 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod60c2ea57_e9b9_49f9_9ac2_d2b141271bf1.slice/crio-81a3e5022c30832d2db220bfbe9c5a69c41609d188f891f44aa84ed51a4191af WatchSource:0}: Error finding container 81a3e5022c30832d2db220bfbe9c5a69c41609d188f891f44aa84ed51a4191af: Status 404 returned error can't find the container with id 81a3e5022c30832d2db220bfbe9c5a69c41609d188f891f44aa84ed51a4191af Apr 16 17:51:41.652654 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:41.652636 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 16 17:51:41.718481 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:41.718422 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5zftw_8eb081ed-d37b-42a9-a3c9-8549bcb2da0d/dns-node-resolver/0.log" Apr 16 17:51:41.719773 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:41.719750 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dtnls/perf-node-gather-daemonset-rd72q" event={"ID":"60c2ea57-e9b9-49f9-9ac2-d2b141271bf1","Type":"ContainerStarted","Data":"81a3e5022c30832d2db220bfbe9c5a69c41609d188f891f44aa84ed51a4191af"} Apr 16 17:51:42.115010 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:42.114975 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-67db985cf-skv8z_dc5c6566-f0a7-42ab-ae2f-8a5b0846b043/registry/0.log" Apr 16 17:51:42.154435 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:42.154403 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mx85v_29ee980f-ef4e-416f-a7db-2b23ef70873e/node-ca/0.log" Apr 16 17:51:42.724270 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:42.724232 2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dtnls/perf-node-gather-daemonset-rd72q" event={"ID":"60c2ea57-e9b9-49f9-9ac2-d2b141271bf1","Type":"ContainerStarted","Data":"1e47825bd2860fcae405a37929ce61e0713c8fc1d4932a1b798045e64e2947c8"} Apr 16 17:51:42.724618 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:42.724355 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dtnls/perf-node-gather-daemonset-rd72q" Apr 16 17:51:42.744056 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:42.744012 2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-dtnls/perf-node-gather-daemonset-rd72q" podStartSLOduration=1.743997926 podStartE2EDuration="1.743997926s" podCreationTimestamp="2026-04-16 17:51:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 17:51:42.743360224 +0000 UTC m=+3836.825094438" watchObservedRunningTime="2026-04-16 17:51:42.743997926 +0000 UTC m=+3836.825732142" Apr 16 17:51:43.206528 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:43.206499 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8kc47_a5cf9cec-6008-46d4-9c84-74af0d3ff9c7/serve-healthcheck-canary/0.log" Apr 16 17:51:43.593623 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:43.593547 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5785d4fcdd-tfwxh_3ef9470d-4a29-4aa3-aa5d-21af5d523f6f/insights-operator/1.log" Apr 16 17:51:43.627219 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:43.627185 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5785d4fcdd-tfwxh_3ef9470d-4a29-4aa3-aa5d-21af5d523f6f/insights-operator/0.log" Apr 16 17:51:43.704172 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:43.704147 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4ljtp_cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7/kube-rbac-proxy/0.log" Apr 16 17:51:43.723826 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:43.723801 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4ljtp_cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7/exporter/0.log" Apr 16 17:51:43.743533 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:43.743510 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4ljtp_cbbcedd1-7dfb-4f57-b692-6fb3ccd071a7/extractor/0.log" Apr 16 17:51:45.745841 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:45.745808 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-68cc5db7c4-vht6t_b290ecac-fda9-416d-92b4-926c2fe09ee7/manager/0.log" Apr 16 17:51:46.090456 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:46.090381 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-696fc77849-rc4qq_9d9df816-3acd-4fd8-a61c-27820fe76f58/manager/0.log" Apr 16 17:51:46.193196 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:46.193163 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-tls-custom-5c88b85bb7-wmrmt_99c4631c-42c7-4173-b33e-055c2de79f60/seaweedfs-tls-custom/0.log" Apr 16 17:51:48.737550 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:48.737526 2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dtnls/perf-node-gather-daemonset-rd72q" Apr 16 17:51:49.795906 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:49.795878 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-64d4d94569-66kjg_4013146b-f68d-4de5-a76e-dc6d4bba1a81/migrator/0.log" Apr 16 17:51:49.814213 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:49.814183 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-64d4d94569-66kjg_4013146b-f68d-4de5-a76e-dc6d4bba1a81/graceful-termination/0.log" Apr 16 17:51:51.172281 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:51.172256 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4k9v9_d655ba61-fb76-4e4b-bbef-f9cfca4b8495/kube-multus/0.log" Apr 16 17:51:51.340998 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:51.340972 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kfh7g_fa3397b8-3244-4c47-bf40-735b4b386fb5/kube-multus-additional-cni-plugins/0.log" Apr 16 17:51:51.359580 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:51.359557 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kfh7g_fa3397b8-3244-4c47-bf40-735b4b386fb5/egress-router-binary-copy/0.log" Apr 16 17:51:51.377241 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:51.377220 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kfh7g_fa3397b8-3244-4c47-bf40-735b4b386fb5/cni-plugins/0.log" Apr 16 17:51:51.394336 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:51.394306 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kfh7g_fa3397b8-3244-4c47-bf40-735b4b386fb5/bond-cni-plugin/0.log" Apr 16 17:51:51.412536 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:51.412517 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kfh7g_fa3397b8-3244-4c47-bf40-735b4b386fb5/routeoverride-cni/0.log" Apr 16 17:51:51.430005 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:51.429932 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kfh7g_fa3397b8-3244-4c47-bf40-735b4b386fb5/whereabouts-cni-bincopy/0.log" Apr 16 17:51:51.447570 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:51.447545 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kfh7g_fa3397b8-3244-4c47-bf40-735b4b386fb5/whereabouts-cni/0.log" Apr 16 17:51:51.699026 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:51.698930 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-kfxzj_d2114257-06f1-4070-ad8d-afce484cc205/network-metrics-daemon/0.log" Apr 16 17:51:51.715632 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:51.715608 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-kfxzj_d2114257-06f1-4070-ad8d-afce484cc205/kube-rbac-proxy/0.log" Apr 16 17:51:52.479317 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:52.479225 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hfbd8_5ec5cd46-2ab4-4b8a-961e-22cad6381d66/ovn-controller/0.log" Apr 16 17:51:52.516080 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:52.516038 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hfbd8_5ec5cd46-2ab4-4b8a-961e-22cad6381d66/ovn-acl-logging/0.log" Apr 16 17:51:52.536776 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:52.536753 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hfbd8_5ec5cd46-2ab4-4b8a-961e-22cad6381d66/kube-rbac-proxy-node/0.log" Apr 16 17:51:52.555642 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:52.555615 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hfbd8_5ec5cd46-2ab4-4b8a-961e-22cad6381d66/kube-rbac-proxy-ovn-metrics/0.log" Apr 16 17:51:52.571352 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:52.571333 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hfbd8_5ec5cd46-2ab4-4b8a-961e-22cad6381d66/northd/0.log" Apr 16 17:51:52.590302 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:52.590282 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hfbd8_5ec5cd46-2ab4-4b8a-961e-22cad6381d66/nbdb/0.log" Apr 16 17:51:52.608828 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:52.608806 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hfbd8_5ec5cd46-2ab4-4b8a-961e-22cad6381d66/sbdb/0.log" Apr 16 17:51:52.711106 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:52.711081 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hfbd8_5ec5cd46-2ab4-4b8a-961e-22cad6381d66/ovnkube-controller/0.log" Apr 16 17:51:54.256338 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:54.256313 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-6kmtd_1d844889-4f54-47ae-8c62-03b86abb6d39/network-check-target-container/0.log" Apr 16 17:51:55.100589 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:55.100560 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-66wx7_a63483e4-ee18-41f8-b36c-45b8cb789405/iptables-alerter/0.log" Apr 16 17:51:55.754995 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:55.754970 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-2m2qc_d4923458-dd09-4ed9-877f-7223efd21713/tuned/0.log" Apr 16 17:51:57.457222 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:57.457130 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-667775844f-l9td2_a2bd7515-2313-46a8-8961-28f13c30996b/cluster-samples-operator/0.log" Apr 16 17:51:57.472930 ip-10-0-140-49 kubenswrapper[2574]: I0416 17:51:57.472908 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-667775844f-l9td2_a2bd7515-2313-46a8-8961-28f13c30996b/cluster-samples-operator-watch/0.log"