Jul 02 14:13:22.466226 ip-10-0-138-160 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 02 14:13:22.466240 ip-10-0-138-160 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 02 14:13:22.466249 ip-10-0-138-160 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 02 14:13:22.466544 ip-10-0-138-160 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 02 14:13:32.536309 ip-10-0-138-160 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 02 14:13:32.536329 ip-10-0-138-160 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 71f644af33e54a1a8db6130fb051f142 --
Jul 02 14:15:37.937251 ip-10-0-138-160 systemd[1]: Starting Kubernetes Kubelet...
Jul 02 14:15:38.317060 ip-10-0-138-160 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 02 14:15:38.317060 ip-10-0-138-160 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 02 14:15:38.317060 ip-10-0-138-160 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 02 14:15:38.317060 ip-10-0-138-160 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 02 14:15:38.317060 ip-10-0-138-160 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 02 14:15:38.318534 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.318402    2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 02 14:15:38.323944 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.323927    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 14:15:38.323944 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.323943    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 14:15:38.324028 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.323947    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 14:15:38.324028 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.323950    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 14:15:38.324028 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.323967    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 14:15:38.324028 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.323970    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 14:15:38.324028 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.323974    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 14:15:38.324028 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.323977    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 14:15:38.324028 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.323980    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 14:15:38.324028 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.323982    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 14:15:38.324028 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.323985    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 14:15:38.324028 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.323988    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 14:15:38.324028 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.323990    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 14:15:38.324028 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.323993    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 14:15:38.324028 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.323996    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 14:15:38.324028 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.323998    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 14:15:38.324028 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324001    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 14:15:38.324028 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324003    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 14:15:38.324028 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324006    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 14:15:38.324028 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324009    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 14:15:38.324028 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324012    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 14:15:38.324028 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324015    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 14:15:38.324497 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324017    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 14:15:38.324497 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324019    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 14:15:38.324497 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324022    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 14:15:38.324497 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324024    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 14:15:38.324497 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324027    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 14:15:38.324497 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324029    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 14:15:38.324497 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324032    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 14:15:38.324497 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324035    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 14:15:38.324497 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324038    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 14:15:38.324497 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324041    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 14:15:38.324497 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324043    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 14:15:38.324497 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324046    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 14:15:38.324497 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324048    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 14:15:38.324497 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324051    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 14:15:38.324497 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324053    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 14:15:38.324497 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324056    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 14:15:38.324497 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324058    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 14:15:38.324497 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324060    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 14:15:38.324497 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324063    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 14:15:38.324497 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324065    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 14:15:38.324979 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324069    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 14:15:38.324979 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324072    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 14:15:38.324979 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324074    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 14:15:38.324979 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324079    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 14:15:38.324979 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324082    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 14:15:38.324979 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324085    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 14:15:38.324979 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324088    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 14:15:38.324979 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324091    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 14:15:38.324979 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324093    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 14:15:38.324979 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324096    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 14:15:38.324979 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324099    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 14:15:38.324979 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324101    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 14:15:38.324979 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324104    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 14:15:38.324979 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324109    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 14:15:38.324979 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324113    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 14:15:38.324979 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324115    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 14:15:38.324979 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324118    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 14:15:38.324979 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324121    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 14:15:38.324979 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324124    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 14:15:38.325457 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324126    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 14:15:38.325457 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324129    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 14:15:38.325457 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324131    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 14:15:38.325457 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324133    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 14:15:38.325457 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324136    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 14:15:38.325457 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324139    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 14:15:38.325457 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324141    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 14:15:38.325457 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324144    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 14:15:38.325457 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324148    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 14:15:38.325457 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324150    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 14:15:38.325457 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324153    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 14:15:38.325457 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324155    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 14:15:38.325457 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324158    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 14:15:38.325457 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324160    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 14:15:38.325457 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324164    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 14:15:38.325457 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324166    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 14:15:38.325457 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324169    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 14:15:38.325457 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324171    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 14:15:38.325457 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324175    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 14:15:38.325457 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324179    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 14:15:38.325941 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324183    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 14:15:38.325941 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324185    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 14:15:38.325941 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324190    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 14:15:38.325941 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324193    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 14:15:38.325941 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324196    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 14:15:38.325941 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324198    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 14:15:38.325941 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324201    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 14:15:38.325941 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324203    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 14:15:38.325941 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324206    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 14:15:38.325941 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324208    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 14:15:38.325941 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324211    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 14:15:38.325941 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324213    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 14:15:38.325941 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324216    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 14:15:38.325941 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324218    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 14:15:38.325941 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324221    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 14:15:38.325941 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324223    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 14:15:38.325941 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324225    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 14:15:38.325941 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.324228    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 14:15:38.325941 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325557    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 14:15:38.325941 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325565    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 14:15:38.326445 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325568    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 14:15:38.326445 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325572    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 14:15:38.326445 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325574    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 14:15:38.326445 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325577    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 14:15:38.326445 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325580    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 14:15:38.326445 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325583    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 14:15:38.326445 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325585    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 14:15:38.326445 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325588    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 14:15:38.326445 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325591    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 14:15:38.326445 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325594    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 14:15:38.326445 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325597    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 14:15:38.326445 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325599    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 14:15:38.326445 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325602    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 14:15:38.326445 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325605    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 14:15:38.326445 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325608    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 14:15:38.326445 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325611    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 14:15:38.326445 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325613    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 14:15:38.326445 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325616    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 14:15:38.326445 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325618    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 14:15:38.326445 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325621    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 14:15:38.326964 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325624    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 14:15:38.326964 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325626    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 14:15:38.326964 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325629    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 14:15:38.326964 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325631    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 14:15:38.326964 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325633    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 14:15:38.326964 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325637    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 14:15:38.326964 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325639    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 14:15:38.326964 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325643    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 14:15:38.326964 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325647    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 14:15:38.326964 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325650    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 14:15:38.326964 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325652    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 14:15:38.326964 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325655    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 14:15:38.326964 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325657    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 14:15:38.326964 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325659    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 14:15:38.326964 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325662    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 14:15:38.326964 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325664    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 14:15:38.326964 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325667    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 14:15:38.326964 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325669    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 14:15:38.326964 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325672    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 14:15:38.327484 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325675    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 14:15:38.327484 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325678    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 14:15:38.327484 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325680    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 14:15:38.327484 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325683    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 14:15:38.327484 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325686    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 14:15:38.327484 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325689    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 14:15:38.327484 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325693    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 14:15:38.327484 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325696    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 14:15:38.327484 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325698    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 14:15:38.327484 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325700    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 14:15:38.327484 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325703    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 14:15:38.327484 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325706    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 14:15:38.327484 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325709    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 14:15:38.327484 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325711    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 14:15:38.327484 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325713    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 14:15:38.327484 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325716    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 14:15:38.327484 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325719    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 14:15:38.327484 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325721    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 14:15:38.327484 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325724    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 14:15:38.327484 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325726    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 14:15:38.328013 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325728    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 14:15:38.328013 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325732    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 14:15:38.328013 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325736    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 14:15:38.328013 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325739    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 14:15:38.328013 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325743    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 14:15:38.328013 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325746    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 14:15:38.328013 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325749    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 14:15:38.328013 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325751    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 14:15:38.328013 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325754    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 14:15:38.328013 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325757    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 14:15:38.328013 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325759    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 14:15:38.328013 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325762    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 14:15:38.328013 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325765    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 14:15:38.328013 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325768    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 14:15:38.328013 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325771    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 14:15:38.328013 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325773    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 14:15:38.328013 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325776    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 14:15:38.328013 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325779    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 14:15:38.328013 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325781    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 14:15:38.328481 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325784    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 14:15:38.328481 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325788    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 14:15:38.328481 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325791    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 14:15:38.328481 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325794    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 14:15:38.328481 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325797    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 14:15:38.328481 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325799    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 14:15:38.328481 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325802    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 14:15:38.328481 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325804    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 14:15:38.328481 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325807    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 14:15:38.328481 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325810    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 14:15:38.328481 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325813    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 14:15:38.328481 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325815    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 14:15:38.328481 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325818    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 14:15:38.328481 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325820    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 14:15:38.328481 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325823    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 14:15:38.328481 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325825    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 14:15:38.328481 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325828    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 14:15:38.328481 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325831    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 14:15:38.328481 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.325833    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 14:15:38.328938 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326225    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 14:15:38.328938 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326231    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 14:15:38.328938 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326234    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 14:15:38.328938 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326237    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 14:15:38.328938 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326240    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 14:15:38.328938 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326243    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 14:15:38.328938 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326246    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 14:15:38.328938 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326249    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 14:15:38.328938 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326252    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 14:15:38.328938 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326255    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 14:15:38.328938 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326257    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 14:15:38.328938 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326260    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 14:15:38.328938 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326264    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 14:15:38.328938 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326266    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 14:15:38.328938 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326269    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 14:15:38.328938 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326272    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 14:15:38.328938 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326274    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 14:15:38.328938 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326276    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 14:15:38.328938 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326279    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 14:15:38.328938 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326281    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 14:15:38.329462 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326284    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 14:15:38.329462 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326286    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 14:15:38.329462 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326288    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 14:15:38.329462 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326291    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 14:15:38.329462 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326293    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 14:15:38.329462 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326296    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 14:15:38.329462 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326298    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 14:15:38.329462 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326301    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 14:15:38.329462 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326304    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 14:15:38.329462 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326306    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 14:15:38.329462 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326309    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 14:15:38.329462 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326311    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 14:15:38.329462 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326314    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 14:15:38.329462 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326316    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 14:15:38.329462 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326318    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 14:15:38.329462 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326321    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 14:15:38.329462 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326325    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 14:15:38.329462 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326328    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 14:15:38.329462 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326330    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 14:15:38.329462 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326333    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 14:15:38.329980 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326336    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 14:15:38.329980 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326339    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 14:15:38.329980 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326341    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 14:15:38.329980 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326344    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 14:15:38.329980 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326346    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 14:15:38.329980 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326350    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 14:15:38.329980 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326354    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 14:15:38.329980 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326358    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 14:15:38.329980 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326363    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 14:15:38.329980 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326366    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 14:15:38.329980 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326370    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 14:15:38.329980 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326372    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 14:15:38.329980 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326375    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 14:15:38.329980 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326378    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 14:15:38.329980 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326381    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 14:15:38.329980 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326384    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 14:15:38.329980 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326386    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 14:15:38.329980 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326389    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 14:15:38.329980 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326392    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 14:15:38.330465 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326395    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 14:15:38.330465 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326398    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 14:15:38.330465 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326401    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 14:15:38.330465 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326404    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 14:15:38.330465 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326406    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 14:15:38.330465 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326409    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 14:15:38.330465 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326411    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 14:15:38.330465 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326414    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 14:15:38.330465 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326416    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 14:15:38.330465 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326420    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 14:15:38.330465 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326423    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 14:15:38.330465 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326426    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 14:15:38.330465 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326428    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 14:15:38.330465 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326431    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 14:15:38.330465 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326434    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 14:15:38.330465 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326436    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 14:15:38.330465 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326442    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 14:15:38.330465 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326444    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 14:15:38.330465 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326447    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 14:15:38.330465 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326449    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 14:15:38.330996 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326452    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 14:15:38.330996 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326455    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 14:15:38.330996 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326457    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 14:15:38.330996 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326460    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 14:15:38.330996 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326462    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 14:15:38.330996 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326465    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 14:15:38.330996 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326468    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 14:15:38.330996 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326470    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 14:15:38.330996 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326473    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 14:15:38.330996 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326475    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 14:15:38.330996 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326478    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 14:15:38.330996 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326480    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 14:15:38.330996 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326483    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 14:15:38.330996 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326485    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 14:15:38.330996 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326488    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 14:15:38.330996 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326490    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 14:15:38.330996 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326492    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 14:15:38.330996 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326495    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 14:15:38.330996 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326497    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 14:15:38.330996 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.326499    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 14:15:38.331477 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326583    2582 flags.go:64] FLAG: --address="0.0.0.0"
Jul 02 14:15:38.331477 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326596    2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 02 14:15:38.331477 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326605    2582 flags.go:64] FLAG: --anonymous-auth="true"
Jul 02 14:15:38.331477 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326611    2582 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 02 14:15:38.331477 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326617    2582 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 02 14:15:38.331477 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326622    2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 02 14:15:38.331477 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326628    2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 02 14:15:38.331477 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326633    2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 02 14:15:38.331477 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326637    2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 02 14:15:38.331477 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326640    2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 02 14:15:38.331477 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326644    2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 02 14:15:38.331477 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326647    2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 02 14:15:38.331477 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326650    2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 02 14:15:38.331477 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326653    2582 flags.go:64] FLAG: --cgroup-root=""
Jul 02 14:15:38.331477 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326656    2582 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 02 14:15:38.331477 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326658    2582 flags.go:64] FLAG: --client-ca-file=""
Jul 02 14:15:38.331477 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326661    2582 flags.go:64] FLAG: --cloud-provider="external"
Jul 02 14:15:38.331477 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326664    2582 flags.go:64] FLAG: --cluster-dns="[]"
Jul 02 14:15:38.331477 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326668    2582 flags.go:64] FLAG: --cluster-domain=""
Jul 02 14:15:38.331477 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326671    2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 02 14:15:38.331477 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326674    2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 02 14:15:38.331477 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326677    2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 02 14:15:38.331477 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326680    2582 flags.go:64] FLAG: --container-log-max-files="5"
Jul 02 14:15:38.331477 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326683    2582 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 02 14:15:38.332073 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326687    2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 02 14:15:38.332073 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326690    2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 02 14:15:38.332073 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326693    2582 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 02 14:15:38.332073 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326696    2582 flags.go:64] FLAG: --contention-profiling="false"
Jul 02 14:15:38.332073 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326698    2582 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 02 14:15:38.332073 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326701    2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 02 14:15:38.332073 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326704    2582 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 02 14:15:38.332073 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326707    2582 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 02 14:15:38.332073 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326711    2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 02 14:15:38.332073 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326716    2582 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 02 14:15:38.332073 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326719    2582 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 02 14:15:38.332073 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326722    2582 flags.go:64] FLAG: --enable-load-reader="false"
Jul 02 14:15:38.332073 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326725    2582 flags.go:64] FLAG: --enable-server="true"
Jul 02 14:15:38.332073 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326728    2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 02 14:15:38.332073 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326732    2582 flags.go:64] FLAG: --event-burst="100"
Jul 02 14:15:38.332073 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326735    2582 flags.go:64] FLAG: --event-qps="50"
Jul 02 14:15:38.332073 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326738    2582 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 02 14:15:38.332073 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326744    2582 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 02 14:15:38.332073 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326748    2582 flags.go:64] FLAG: --eviction-hard=""
Jul 02 14:15:38.332073 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326751    2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 02 14:15:38.332073 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326754    2582 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 02 14:15:38.332073 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326757    2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 02 14:15:38.332073 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326760    2582 flags.go:64] FLAG: --eviction-soft=""
Jul 02 14:15:38.332073 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326762    2582 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 02 14:15:38.332073 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326765    2582 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 02 14:15:38.332741 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326768    2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 02 14:15:38.332741 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326771    2582 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 02 14:15:38.332741 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326773    2582 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 02 14:15:38.332741 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326776    2582 flags.go:64] FLAG: --fail-swap-on="true"
Jul 02 14:15:38.332741 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326779    2582 flags.go:64] FLAG: --feature-gates=""
Jul 02 14:15:38.332741 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326783    2582 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 02 14:15:38.332741 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326786    2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 02 14:15:38.332741 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326789    2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 02 14:15:38.332741 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326792    2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 02 14:15:38.332741 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326795    2582 flags.go:64] FLAG: --healthz-port="10248"
Jul 02 14:15:38.332741 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326798    2582 flags.go:64] FLAG: --help="false"
Jul 02 14:15:38.332741 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326800    2582 flags.go:64] FLAG: --hostname-override="ip-10-0-138-160.ec2.internal"
Jul 02 14:15:38.332741 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326803    2582 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 02 14:15:38.332741 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326806    2582 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 02 14:15:38.332741 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326809    2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 02 14:15:38.332741 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326812    2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 02 14:15:38.332741 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326816    2582 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 02 14:15:38.332741 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326819    2582 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 02 14:15:38.332741 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326822    2582 flags.go:64] FLAG: --image-service-endpoint=""
Jul 02 14:15:38.332741 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326825    2582 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 02 14:15:38.332741 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326828    2582 flags.go:64] FLAG: --kube-api-burst="100"
Jul 02 14:15:38.332741 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326831    2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 02 14:15:38.332741 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326834    2582 flags.go:64] FLAG: --kube-api-qps="50"
Jul 02 14:15:38.332741 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326837    2582 flags.go:64] FLAG: --kube-reserved=""
Jul 02 14:15:38.333753 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326840    2582 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 02 14:15:38.333753 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326843    2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 02 14:15:38.333753 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326846    2582 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 02 14:15:38.333753 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326849    2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 02 14:15:38.333753 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326852    2582 flags.go:64] FLAG: --lock-file=""
Jul 02 14:15:38.333753 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326854    2582 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 02 14:15:38.333753 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326857    2582 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 02 14:15:38.333753 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326860    2582 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 02 14:15:38.333753 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326869    2582 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 02 14:15:38.333753 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326872    2582 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 02 14:15:38.333753 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326875    2582 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 02 14:15:38.333753 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326878    2582 flags.go:64] FLAG: --logging-format="text"
Jul 02 14:15:38.333753 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326880    2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 02 14:15:38.333753 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326884    2582 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 02 14:15:38.333753 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326886    2582 flags.go:64] FLAG: --manifest-url=""
Jul 02 14:15:38.333753 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326889    2582 flags.go:64] FLAG: --manifest-url-header=""
Jul 02 14:15:38.333753 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326894    2582 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 02 14:15:38.333753 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326896    2582 flags.go:64] FLAG: --max-open-files="1000000"
Jul 02 14:15:38.333753 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326900    2582 flags.go:64] FLAG: --max-pods="110"
Jul 02 14:15:38.333753 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326903    2582 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 02 14:15:38.333753 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326906    2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 02 14:15:38.333753 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326909    2582 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 02 14:15:38.333753 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326911    2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 02 14:15:38.333753 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326914    2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 02 14:15:38.333753 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326918    2582 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 02 14:15:38.334826 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326921    2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 02 14:15:38.334826 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326928    2582 flags.go:64] FLAG: --node-status-max-images="50"
Jul 02 14:15:38.334826 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326931    2582 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 02 14:15:38.334826 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326934    2582 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 02 14:15:38.334826 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326936    2582 flags.go:64] FLAG: --pod-cidr=""
Jul 02 14:15:38.334826 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326939    2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8a2632cd31d78133be7b5fb6999469b2acf9b98d077c7f6ae5d0d94083d76b04"
Jul 02 14:15:38.334826 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326945    2582 flags.go:64] FLAG: --pod-manifest-path=""
Jul 02 14:15:38.334826 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326948    2582 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 02 14:15:38.334826 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326965    2582 flags.go:64] FLAG: --pods-per-core="0"
Jul 02 14:15:38.334826 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326969    2582 flags.go:64] FLAG: --port="10250"
Jul 02 14:15:38.334826 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326972    2582 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 02 14:15:38.334826 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326975    2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a8c0b92743328e1b"
Jul 02 14:15:38.334826 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326979    2582 flags.go:64] FLAG: --qos-reserved=""
Jul 02 14:15:38.334826 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326981    2582 flags.go:64] FLAG: --read-only-port="10255"
Jul 02 14:15:38.334826 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326984    2582 flags.go:64] FLAG: --register-node="true"
Jul 02 14:15:38.334826 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326987    2582 flags.go:64] FLAG: --register-with-taints=""
Jul 02 14:15:38.334826 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326990    2582 flags.go:64] FLAG: --registry-burst="10"
Jul 02 14:15:38.334826 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326993    2582 flags.go:64] FLAG: --registry-qps="5"
Jul 02 14:15:38.334826 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326996    2582 flags.go:64] FLAG: --reserved-cpus=""
Jul 02 14:15:38.334826 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.326999    2582 flags.go:64] FLAG: --reserved-memory=""
Jul 02 14:15:38.334826 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327002    2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 02 14:15:38.334826 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327005    2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 02 14:15:38.334826 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327008    2582 flags.go:64] FLAG: --rotate-certificates="false"
Jul 02 14:15:38.334826 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327011    2582 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 02 14:15:38.334826 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327014    2582 flags.go:64] FLAG: --runonce="false"
Jul 02 14:15:38.336072 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327017    2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 02 14:15:38.336072 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327020    2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 02 14:15:38.336072 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327023    2582 flags.go:64] FLAG: --seccomp-default="false"
Jul 02 14:15:38.336072 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327026    2582 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 02 14:15:38.336072 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327028    2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 02 14:15:38.336072 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327033    2582 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 02 14:15:38.336072 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327036    2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 02 14:15:38.336072 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327040    2582 flags.go:64] FLAG: --storage-driver-password="root"
Jul 02 14:15:38.336072 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327043    2582 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 02 14:15:38.336072 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327045    2582 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 02 14:15:38.336072 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327048    2582 flags.go:64] FLAG: --storage-driver-user="root"
Jul 02 14:15:38.336072 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327051    2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 02 14:15:38.336072 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327054    2582 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 02 14:15:38.336072 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327057    2582 flags.go:64] FLAG: --system-cgroups=""
Jul 02 14:15:38.336072 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327059    2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 02 14:15:38.336072 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327065    2582 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 02 14:15:38.336072 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327067    2582 flags.go:64] FLAG: --tls-cert-file=""
Jul 02 14:15:38.336072 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327071    2582 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 02 14:15:38.336072 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327076    2582 flags.go:64] FLAG: --tls-min-version=""
Jul 02 14:15:38.336072 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327078    2582 flags.go:64] FLAG: --tls-private-key-file=""
Jul 02 14:15:38.336072 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327081    2582 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 02 14:15:38.336072 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327084    2582 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 02 14:15:38.336072 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327087    2582 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 02 14:15:38.336072 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327089    2582 flags.go:64] FLAG: --v="2"
Jul 02 14:15:38.336072 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327093    2582 flags.go:64] FLAG: --version="false"
Jul 02 14:15:38.336943 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327097    2582 flags.go:64] FLAG: --vmodule=""
Jul 02 14:15:38.336943 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327100    2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 02 14:15:38.336943 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327104    2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 02 14:15:38.336943 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327218    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 14:15:38.336943 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327223    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 14:15:38.336943 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327227    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 14:15:38.336943 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327230    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 14:15:38.336943 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327234    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 14:15:38.336943 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327237    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 14:15:38.336943 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327240    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 14:15:38.336943 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327242    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 14:15:38.336943 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327245    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 14:15:38.336943 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327249    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 14:15:38.336943 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327252    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 14:15:38.336943 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327256    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 14:15:38.336943 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327258    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 14:15:38.336943 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327261    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 14:15:38.336943 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327264    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 14:15:38.336943 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327266    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 14:15:38.337889 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327269    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 14:15:38.337889 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327272    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 14:15:38.337889 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327274    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 14:15:38.337889 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327277    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 14:15:38.337889 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327279    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 14:15:38.337889 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327282    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 14:15:38.337889 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327285    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 14:15:38.337889 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327288    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 14:15:38.337889 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327290    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 14:15:38.337889 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327293    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 14:15:38.337889 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327295    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 14:15:38.337889 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327298    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 14:15:38.337889 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327300    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 14:15:38.337889 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327303    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 14:15:38.337889 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327305    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 14:15:38.337889 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327307    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 14:15:38.337889 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327310    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 14:15:38.337889 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327312    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 14:15:38.337889 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327315    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 14:15:38.337889 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327317    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 14:15:38.338727 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327320    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 14:15:38.338727 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327323    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 14:15:38.338727 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327325    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 14:15:38.338727 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327328    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 14:15:38.338727 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327330    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 14:15:38.338727 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327334    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 14:15:38.338727 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327336    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 14:15:38.338727 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327341    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 14:15:38.338727 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327343    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 14:15:38.338727 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327345    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 14:15:38.338727 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327348    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 14:15:38.338727 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327350    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 14:15:38.338727 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327353    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 14:15:38.338727 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327355    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 14:15:38.338727 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327358    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 14:15:38.338727 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327360    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 14:15:38.338727 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327363    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 14:15:38.338727 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327365    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 14:15:38.338727 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327368    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 14:15:38.338727 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327371    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 14:15:38.339322 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327373    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 14:15:38.339322 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327376    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 14:15:38.339322 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327380    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 14:15:38.339322 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327383    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 14:15:38.339322 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327385    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 14:15:38.339322 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327388    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 14:15:38.339322 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327390    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 14:15:38.339322 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327393    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 14:15:38.339322 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327395    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 14:15:38.339322 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327397    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 14:15:38.339322 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327400    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 14:15:38.339322 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327402    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 14:15:38.339322 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327405    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 14:15:38.339322 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327408    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 14:15:38.339322 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327410    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 14:15:38.339322 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327412    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 14:15:38.339322 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327415    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 14:15:38.339322 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327418    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 14:15:38.339322 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327421    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 14:15:38.339857 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327425    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 14:15:38.339857 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327427    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 14:15:38.339857 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327430    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 14:15:38.339857 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327432    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 14:15:38.339857 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327435    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 14:15:38.339857 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327437    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 14:15:38.339857 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327440    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 14:15:38.339857 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327442    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 14:15:38.339857 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327444    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 14:15:38.339857 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327447    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 14:15:38.339857 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327449    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 14:15:38.339857 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327452    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 14:15:38.339857 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327454    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 14:15:38.339857 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327457    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 14:15:38.339857 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327460    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 14:15:38.339857 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327462    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 14:15:38.339857 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327464    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 14:15:38.339857 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327467    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 14:15:38.339857 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327470    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 14:15:38.339857 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327472    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 14:15:38.340385 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327475    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 14:15:38.340385 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327477    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 14:15:38.340385 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327480    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 14:15:38.340385 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.327482    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 14:15:38.340385 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.327488    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 02 14:15:38.340385 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.335129    2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 02 14:15:38.340385 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.335155    2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 02 14:15:38.340385 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.335180    2582 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 02 14:15:38.340385 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.335190    2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 02 14:15:38.340385 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335291    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 14:15:38.340385 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335300    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 14:15:38.340385 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335305    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 14:15:38.340385 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335310    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 14:15:38.340385 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335315    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 14:15:38.340385 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335319    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 14:15:38.340826 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335323    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 14:15:38.340826 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335328    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 14:15:38.340826 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335332    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 14:15:38.340826 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335336    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 14:15:38.340826 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335341    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 14:15:38.340826 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335345    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 14:15:38.340826 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335349    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 14:15:38.340826 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335354    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 14:15:38.340826 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335358    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 14:15:38.340826 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335362    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 14:15:38.340826 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335366    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 14:15:38.340826 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335370    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 14:15:38.340826 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335377    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 14:15:38.340826 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335383    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 14:15:38.340826 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335388    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 14:15:38.340826 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335392    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 14:15:38.340826 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335396    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 14:15:38.340826 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335403    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 14:15:38.341340 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335409    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 14:15:38.341340 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335414    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 14:15:38.341340 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335418    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 14:15:38.341340 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335423    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 14:15:38.341340 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335427    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 14:15:38.341340 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335437    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 14:15:38.341340 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335443    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 14:15:38.341340 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335448    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 14:15:38.341340 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335452    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 14:15:38.341340 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335457    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 14:15:38.341340 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335463    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 14:15:38.341340 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335467    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 14:15:38.341340 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335472    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 14:15:38.341340 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335476    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 14:15:38.341340 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335480    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 14:15:38.341340 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335485    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 14:15:38.341340 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335489    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 14:15:38.341340 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335493    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 14:15:38.341340 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335497    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 14:15:38.341340 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335502    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 14:15:38.341866 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335506    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 14:15:38.341866 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335510    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 14:15:38.341866 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335514    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 14:15:38.341866 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335518    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 14:15:38.341866 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335522    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 14:15:38.341866 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335526    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 14:15:38.341866 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335531    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 14:15:38.341866 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335535    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 14:15:38.341866 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335539    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 14:15:38.341866 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335543    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 14:15:38.341866 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335548    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 14:15:38.341866 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335552    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 14:15:38.341866 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335556    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 14:15:38.341866 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335561    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 14:15:38.341866 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335565    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 14:15:38.341866 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335569    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 14:15:38.341866 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335573    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 14:15:38.341866 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335578    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 14:15:38.341866 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335583    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 14:15:38.341866 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335587    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 14:15:38.342516 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335592    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 14:15:38.342516 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335596    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 14:15:38.342516 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335600    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 14:15:38.342516 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335605    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 14:15:38.342516 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335609    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 14:15:38.342516 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335613    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 14:15:38.342516 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335617    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 14:15:38.342516 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335621    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 14:15:38.342516 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335625    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 14:15:38.342516 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335629    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 14:15:38.342516 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335633    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 14:15:38.342516 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335637    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 14:15:38.342516 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335642    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 14:15:38.342516 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335646    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 14:15:38.342516 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335650    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 14:15:38.342516 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335654    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 14:15:38.342516 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335659    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 14:15:38.342516 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335663    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 14:15:38.342516 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335667    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 14:15:38.342516 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335672    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 14:15:38.343347 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335676    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 14:15:38.343347 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335680    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 14:15:38.343347 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335684    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 14:15:38.343347 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335688    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 14:15:38.343347 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335692    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 14:15:38.343347 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335697    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 14:15:38.343347 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335701    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 14:15:38.343347 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335705    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 14:15:38.343347 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335709    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 14:15:38.343347 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335714    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 14:15:38.343347 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335718    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 14:15:38.343347 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335723    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 14:15:38.343347 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335727    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 14:15:38.343347 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335732    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 14:15:38.343347 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335736    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 14:15:38.343762 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.335744    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 02 14:15:38.343762 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335933    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 14:15:38.343762 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335942    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 14:15:38.343762 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335947    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 14:15:38.343762 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335971    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 14:15:38.343762 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335976    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 14:15:38.343762 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335983    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 14:15:38.343762 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335990    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 14:15:38.343762 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.335995    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 14:15:38.343762 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336000    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 14:15:38.343762 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336005    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 14:15:38.343762 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336009    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 14:15:38.343762 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336014    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 14:15:38.343762 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336019    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 14:15:38.344342 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336023    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 14:15:38.344342 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336029    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 14:15:38.344342 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336033    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 14:15:38.344342 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336037    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 14:15:38.344342 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336042    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 14:15:38.344342 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336046    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 14:15:38.344342 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336051    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 14:15:38.344342 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336055    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 14:15:38.344342 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336060    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 14:15:38.344342 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336064    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 14:15:38.344342 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336067    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 14:15:38.344342 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336071    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 14:15:38.344342 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336075    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 14:15:38.344342 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336079    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 14:15:38.344342 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336083    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 14:15:38.344342 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336088    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 14:15:38.344342 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336092    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 14:15:38.344342 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336097    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 14:15:38.344342 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336101    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 14:15:38.344342 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336105    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 14:15:38.344911 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336109    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 14:15:38.344911 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336113    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 14:15:38.344911 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336118    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 14:15:38.344911 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336122    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 14:15:38.344911 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336126    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 14:15:38.344911 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336131    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 14:15:38.344911 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336135    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 14:15:38.344911 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336140    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 14:15:38.344911 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336144    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 14:15:38.344911 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336148    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 14:15:38.344911 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336152    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 14:15:38.344911 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336156    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 14:15:38.344911 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336160    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 14:15:38.344911 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336165    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 14:15:38.344911 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336169    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 14:15:38.344911 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336173    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 14:15:38.344911 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336177    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 14:15:38.344911 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336181    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 14:15:38.344911 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336185    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 14:15:38.344911 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336189    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 14:15:38.345446 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336193    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 14:15:38.345446 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336198    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 14:15:38.345446 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336202    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 14:15:38.345446 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336206    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 14:15:38.345446 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336210    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 14:15:38.345446 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336215    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 14:15:38.345446 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336219    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 14:15:38.345446 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336223    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 14:15:38.345446 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336228    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 14:15:38.345446 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336233    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 14:15:38.345446 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336237    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 14:15:38.345446 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336240    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 14:15:38.345446 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336245    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 14:15:38.345446 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336249    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 14:15:38.345446 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336253    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 14:15:38.345446 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336258    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 14:15:38.345446 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336262    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 14:15:38.345446 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336266    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 14:15:38.345446 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336270    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 14:15:38.345446 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336275    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 14:15:38.345932 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336280    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 14:15:38.345932 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336285    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 14:15:38.345932 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336289    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 14:15:38.345932 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336293    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 14:15:38.345932 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336297    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 14:15:38.345932 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336301    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 14:15:38.345932 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336306    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 14:15:38.345932 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336310    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 14:15:38.345932 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336314    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 14:15:38.345932 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336318    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 14:15:38.345932 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336322    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 14:15:38.345932 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336326    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 14:15:38.345932 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336331    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 14:15:38.345932 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336335    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 14:15:38.345932 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336339    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 14:15:38.345932 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336343    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 14:15:38.345932 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336347    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 14:15:38.345932 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336354    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 14:15:38.345932 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336359    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 14:15:38.346410 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336363    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 14:15:38.346410 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336369    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 14:15:38.346410 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336374    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 14:15:38.346410 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336378    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 14:15:38.346410 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336382    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 14:15:38.346410 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336386    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 14:15:38.346410 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:38.336390    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 14:15:38.346410 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.336398    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 02 14:15:38.346410 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.336551    2582 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 02 14:15:38.346410 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.339231    2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 02 14:15:38.346410 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.340699    2582 server.go:1019] "Starting client certificate rotation"
Jul 02 14:15:38.346410 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.340805    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 02 14:15:38.346410 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.342439    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 02 14:15:38.346745 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.345854    2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 02 14:15:38.349077 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.349060    2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 02 14:15:38.360814 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.360796    2582 log.go:25] "Validated CRI v1 runtime API"
Jul 02 14:15:38.362751 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.362731    2582 log.go:25] "Validated CRI v1 image API"
Jul 02 14:15:38.363569 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.363555    2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 02 14:15:38.366512 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.366492    2582 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 83300c1b-c879-4b7b-a745-f2e8e9bacf0f:/dev/nvme0n1p4 8e9ef38d-50cd-4953-86b9-4ebff2a8c1b6:/dev/nvme0n1p3]
Jul 02 14:15:38.366573 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.366512    2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 02 14:15:38.371027 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.371010    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 02 14:15:38.371864 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.371749    2582 manager.go:216] Machine: {Timestamp:2026-07-02 14:15:38.370402884 +0000 UTC m=+0.321740522 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3149995 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec205ba040c71a058e60804e88b8ae13 SystemUUID:ec205ba0-40c7-1a05-8e60-804e88b8ae13 BootID:71f644af-33e5-4a1a-8db6-130fb051f142 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:1c:a4:82:93:cf Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:1c:a4:82:93:cf Speed:0 Mtu:9001} {Name:ovs-system MacAddress:52:21:db:3b:2d:07 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 02 14:15:38.371864 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.371861    2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 02 14:15:38.371994 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.371942    2582 manager.go:232] Version: {KernelVersion:5.14.0-570.124.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260623-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 02 14:15:38.374702 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.374669    2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 02 14:15:38.374876 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.374705    2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-160.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 02 14:15:38.375413 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.375401    2582 topology_manager.go:138] "Creating topology manager with none policy"
Jul 02 14:15:38.375461 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.375420    2582 container_manager_linux.go:309] "Creating device plugin manager"
Jul 02 14:15:38.375794 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.375782    2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 02 14:15:38.376386 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.376375    2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 02 14:15:38.377256 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.377239    2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 02 14:15:38.380923 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.380908    2582 state_mem.go:36] "Initialized new in-memory state store"
Jul 02 14:15:38.381133 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.381121    2582 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 02 14:15:38.381193 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.381161    2582 kubelet.go:486] "Attempting to sync node with API server"
Jul 02 14:15:38.381193 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.381175    2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 02 14:15:38.381277 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.381202    2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 02 14:15:38.381277 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.381223    2582 kubelet.go:398] "Adding apiserver pod source"
Jul 02 14:15:38.381277 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.381236    2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 02 14:15:38.381689 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.381674    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 02 14:15:38.381757 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.381698    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 02 14:15:38.383798 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.383782    2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 02 14:15:38.387305 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.387288    2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 02 14:15:38.387381 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.387320    2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 02 14:15:38.387986 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.387974    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 02 14:15:38.388020 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.387991    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 02 14:15:38.388020 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.387998    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 02 14:15:38.388020 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.388003    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 02 14:15:38.388020 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.388011    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 02 14:15:38.388020 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.388017    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 02 14:15:38.388160 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.388025    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 02 14:15:38.388160 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.388030    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 02 14:15:38.388160 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.388037    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 02 14:15:38.388160 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.388042    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 02 14:15:38.388160 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.388051    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 02 14:15:38.388160 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.388058    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 02 14:15:38.388868 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.388857    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 02 14:15:38.388896 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.388869    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 02 14:15:38.392202 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.392190    2582 server.go:1268] "Started kubelet"
Jul 02 14:15:38.392381 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.392286    2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 02 14:15:38.392381 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.392342    2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 02 14:15:38.392485 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.392403    2582 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 02 14:15:38.393127 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.393106    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-160.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 02 14:15:38.393738 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.393724    2582 server.go:310] "Adding debug handlers to kubelet server"
Jul 02 14:15:38.393795 ip-10-0-138-160 systemd[1]: Started Kubernetes Kubelet.
Jul 02 14:15:38.393921 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:38.393893    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-160.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 02 14:15:38.393992 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:38.393893    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 02 14:15:38.394855 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.394840    2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 02 14:15:38.397704 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:38.396830    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-160.ec2.internal.18be7ea262c4c67c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-160.ec2.internal,UID:ip-10-0-138-160.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-160.ec2.internal,},FirstTimestamp:2026-07-02 14:15:38.392159868 +0000 UTC m=+0.343497505,LastTimestamp:2026-07-02 14:15:38.392159868 +0000 UTC m=+0.343497505,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-160.ec2.internal,}"
Jul 02 14:15:38.398572 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:38.398555    2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 02 14:15:38.401528 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.401503    2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 02 14:15:38.401620 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.401526    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 02 14:15:38.401700 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:38.401679    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-160.ec2.internal\" not found"
Jul 02 14:15:38.402278 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.402261    2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 02 14:15:38.402368 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.402308    2582 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 02 14:15:38.402368 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.402325    2582 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 02 14:15:38.402470 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.402447    2582 reconstruct.go:97] "Volume reconstruction finished"
Jul 02 14:15:38.402470 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.402457    2582 reconciler.go:29] "Reconciler: start to sync state"
Jul 02 14:15:38.402553 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.402495    2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 02 14:15:38.402553 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.402508    2582 factory.go:55] Registering systemd factory
Jul 02 14:15:38.402553 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.402517    2582 factory.go:223] Registration of the systemd container factory successfully
Jul 02 14:15:38.402767 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.402749    2582 factory.go:153] Registering CRI-O factory
Jul 02 14:15:38.402767 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.402767    2582 factory.go:223] Registration of the crio container factory successfully
Jul 02 14:15:38.402930 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.402788    2582 factory.go:103] Registering Raw factory
Jul 02 14:15:38.402930 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.402802    2582 manager.go:1196] Started watching for new ooms in manager
Jul 02 14:15:38.403452 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.403439    2582 manager.go:349] Starting recovery of all containers
Jul 02 14:15:38.403712 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:38.403692    2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-160.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 02 14:15:38.403895 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:38.403852    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 02 14:15:38.406634 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.406611    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-k9h8t"
Jul 02 14:15:38.413426 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.413281    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-k9h8t"
Jul 02 14:15:38.413683 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.413659    2582 manager.go:354] Recovery completed
Jul 02 14:15:38.415690 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:38.415667    2582 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 02 14:15:38.418575 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.418559    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 14:15:38.420251 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.420235    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 14:15:38.420312 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.420268    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 14:15:38.420312 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.420280    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasSufficientPID"
Jul 02 14:15:38.420776 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.420760    2582 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 02 14:15:38.420853 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.420773    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 14:15:38.420853 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.420776    2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 02 14:15:38.420853 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.420811    2582 state_mem.go:36] "Initialized new in-memory state store"
Jul 02 14:15:38.421417 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.421399    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 14:15:38.421486 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.421428    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 14:15:38.421486 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.421439    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasSufficientPID"
Jul 02 14:15:38.422760 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.422749    2582 policy_none.go:49] "None policy: Start"
Jul 02 14:15:38.422806 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.422764    2582 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 02 14:15:38.422806 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.422774    2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 02 14:15:38.424697 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.424681    2582 policy_none.go:47] "Start"
Jul 02 14:15:38.483472 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.467687    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 02 14:15:38.483472 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.469040    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 02 14:15:38.483472 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.469083    2582 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 02 14:15:38.483472 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.469107    2582 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 02 14:15:38.483472 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:38.469147    2582 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Jul 02 14:15:38.483472 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.470641    2582 manager.go:337] "Starting Device Plugin manager"
Jul 02 14:15:38.483472 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:38.470672    2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 02 14:15:38.483472 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.470685    2582 server.go:87] "Starting device plugin registration server"
Jul 02 14:15:38.483472 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.470944    2582 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 02 14:15:38.483472 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.470974    2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 02 14:15:38.483472 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.471191    2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 02 14:15:38.483472 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.471262    2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 02 14:15:38.483472 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.471270    2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 02 14:15:38.483472 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:38.471668    2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 02 14:15:38.483472 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:38.471728    2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-160.ec2.internal\" not found"
Jul 02 14:15:38.483472 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.471776    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 14:15:38.569642 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.569567    2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-160.ec2.internal"]
Jul 02 14:15:38.569760 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.569649    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 14:15:38.570602 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.570586    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 14:15:38.570685 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.570615    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 14:15:38.570685 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.570633    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasSufficientPID"
Jul 02 14:15:38.571648 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.571635    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 14:15:38.572308 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.572293    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 14:15:38.572380 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.572324    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 14:15:38.572380 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.572338    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasSufficientPID"
Jul 02 14:15:38.572380 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.572363    2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-160.ec2.internal"
Jul 02 14:15:38.572853 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.572842    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 14:15:38.572976 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.572950    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal"
Jul 02 14:15:38.573013 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.572991    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 14:15:38.573486 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.573469    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 14:15:38.573572 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.573497    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 14:15:38.573572 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.573507    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasSufficientPID"
Jul 02 14:15:38.573572 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.573478    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 14:15:38.573572 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.573569    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 14:15:38.573692 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.573606    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasSufficientPID"
Jul 02 14:15:38.576223 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.576207    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-160.ec2.internal"
Jul 02 14:15:38.576296 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.576231    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 14:15:38.580375 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.580361    2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-160.ec2.internal"
Jul 02 14:15:38.580444 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:38.580380    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-160.ec2.internal\": node \"ip-10-0-138-160.ec2.internal\" not found"
Jul 02 14:15:38.580920 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.580903    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 14:15:38.581015 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.580930    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 14:15:38.581015 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.580941    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeHasSufficientPID"
Jul 02 14:15:38.594531 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:38.594510    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-160.ec2.internal\" not found" node="ip-10-0-138-160.ec2.internal"
Jul 02 14:15:38.599259 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:38.599237    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-160.ec2.internal\" not found"
Jul 02 14:15:38.600313 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:38.600301    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-160.ec2.internal\" not found" node="ip-10-0-138-160.ec2.internal"
Jul 02 14:15:38.604344 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.604328    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e0538fe0bb3712feb96e32d2faab018b-config\") pod \"kube-apiserver-proxy-ip-10-0-138-160.ec2.internal\" (UID: \"e0538fe0bb3712feb96e32d2faab018b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-160.ec2.internal"
Jul 02 14:15:38.604397 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.604352    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d870243e2cd87d194db7d4e9abd907fd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal\" (UID: \"d870243e2cd87d194db7d4e9abd907fd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal"
Jul 02 14:15:38.604397 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.604371    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d870243e2cd87d194db7d4e9abd907fd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal\" (UID: \"d870243e2cd87d194db7d4e9abd907fd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal"
Jul 02 14:15:38.700365 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:38.700322    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-160.ec2.internal\" not found"
Jul 02 14:15:38.704693 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.704667    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d870243e2cd87d194db7d4e9abd907fd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal\" (UID: \"d870243e2cd87d194db7d4e9abd907fd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal"
Jul 02 14:15:38.704771 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.704701    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d870243e2cd87d194db7d4e9abd907fd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal\" (UID: \"d870243e2cd87d194db7d4e9abd907fd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal"
Jul 02 14:15:38.704771 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.704718    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e0538fe0bb3712feb96e32d2faab018b-config\") pod \"kube-apiserver-proxy-ip-10-0-138-160.ec2.internal\" (UID: \"e0538fe0bb3712feb96e32d2faab018b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-160.ec2.internal"
Jul 02 14:15:38.704771 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.704753    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d870243e2cd87d194db7d4e9abd907fd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal\" (UID: \"d870243e2cd87d194db7d4e9abd907fd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal"
Jul 02 14:15:38.704868 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.704771    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d870243e2cd87d194db7d4e9abd907fd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal\" (UID: \"d870243e2cd87d194db7d4e9abd907fd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal"
Jul 02 14:15:38.704868 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.704751    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e0538fe0bb3712feb96e32d2faab018b-config\") pod \"kube-apiserver-proxy-ip-10-0-138-160.ec2.internal\" (UID: \"e0538fe0bb3712feb96e32d2faab018b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-160.ec2.internal"
Jul 02 14:15:38.800439 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:38.800405    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-160.ec2.internal\" not found"
Jul 02 14:15:38.897203 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.897130    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal"
Jul 02 14:15:38.901147 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:38.901125    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-160.ec2.internal\" not found"
Jul 02 14:15:38.903810 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:38.903795    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-160.ec2.internal"
Jul 02 14:15:39.001561 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:39.001525    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-160.ec2.internal\" not found"
Jul 02 14:15:39.102197 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:39.102165    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-160.ec2.internal\" not found"
Jul 02 14:15:39.205106 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:39.205008    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-160.ec2.internal\" not found"
Jul 02 14:15:39.309416 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:39.309368    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-160.ec2.internal\" not found"
Jul 02 14:15:39.339903 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:39.339879    2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 02 14:15:39.340461 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:39.340046    2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 02 14:15:39.409738 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:39.409704    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-160.ec2.internal\" not found"
Jul 02 14:15:39.420006 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:39.419977    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-01 14:10:39 +0000 UTC" deadline="2028-01-21 21:46:06.213127768 +0000 UTC"
Jul 02 14:15:39.420006 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:39.420004    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13639h30m26.79312783s"
Jul 02 14:15:39.510384 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:39.510345    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-160.ec2.internal\" not found"
Jul 02 14:15:39.547678 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:39.547647    2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 14:15:39.611214 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:39.611178    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-160.ec2.internal\" not found"
Jul 02 14:15:39.632772 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:39.632737    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0538fe0bb3712feb96e32d2faab018b.slice/crio-a90ad4aa75df9ebb965bad31ddda73dc87b2ef512a16bfeb3d5215ef660e1aa9 WatchSource:0}: Error finding container a90ad4aa75df9ebb965bad31ddda73dc87b2ef512a16bfeb3d5215ef660e1aa9: Status 404 returned error can't find the container with id a90ad4aa75df9ebb965bad31ddda73dc87b2ef512a16bfeb3d5215ef660e1aa9
Jul 02 14:15:39.638616 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:39.638593    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 14:15:39.712219 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:39.712177    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-160.ec2.internal\" not found"
Jul 02 14:15:39.740786 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:39.740752    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd870243e2cd87d194db7d4e9abd907fd.slice/crio-fa4bb2711ffc38f763e88b52555ce199dffeb5a3c4656e1d1271e04f6848bb18 WatchSource:0}: Error finding container fa4bb2711ffc38f763e88b52555ce199dffeb5a3c4656e1d1271e04f6848bb18: Status 404 returned error can't find the container with id fa4bb2711ffc38f763e88b52555ce199dffeb5a3c4656e1d1271e04f6848bb18
Jul 02 14:15:39.812653 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:39.812563    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-160.ec2.internal\" not found"
Jul 02 14:15:39.816937 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:39.816918    2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 14:15:39.896462 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:39.896431    2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 14:15:39.902282 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:39.902263    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal"
Jul 02 14:15:39.919197 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:39.919173    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 02 14:15:39.920134 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:39.920119    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-160.ec2.internal"
Jul 02 14:15:39.928357 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:39.928340    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 02 14:15:40.382679 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.382637    2582 apiserver.go:52] "Watching apiserver"
Jul 02 14:15:40.394801 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.394763    2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 02 14:15:40.395392 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.395359    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-f8pbz","openshift-multus/multus-r2rq9","openshift-multus/network-metrics-daemon-lpdzr","openshift-ovn-kubernetes/ovnkube-node-cnwb5","kube-system/global-pull-secret-syncer-p64q5","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4vgf6","openshift-image-registry/node-ca-rtn2p","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal","openshift-multus/multus-additional-cni-plugins-p96q8","openshift-network-diagnostics/network-check-target-7gvk9","openshift-network-operator/iptables-alerter-v5829","kube-system/konnectivity-agent-r6m5m","kube-system/kube-apiserver-proxy-ip-10-0-138-160.ec2.internal","openshift-cluster-node-tuning-operator/tuned-q6zgx"]
Jul 02 14:15:40.398078 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.398045    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rtn2p"
Jul 02 14:15:40.400399 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.400375    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-v5829"
Jul 02 14:15:40.400998 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.400953    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 02 14:15:40.400998 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.400978    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 02 14:15:40.401166 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.401110    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 02 14:15:40.401215 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.401184    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zbbnn\""
Jul 02 14:15:40.402703 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.402681    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 02 14:15:40.403658 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.403636    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 02 14:15:40.403844 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.403828    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 02 14:15:40.404086 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.404068    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 02 14:15:40.404288 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.404270    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-js5bc\""
Jul 02 14:15:40.404494 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.404480    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:15:40.404584 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:40.404560    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpdzr" podUID="ada3d319-cd6a-42ad-b8f9-c04da51705ef"
Jul 02 14:15:40.405395 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.405375    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.407952 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.407933    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:15:40.408092 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:40.408010    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p64q5" podUID="2bb3f329-4286-46e2-8a7d-450769d1aa15"
Jul 02 14:15:40.408182 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.408165    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 02 14:15:40.408815 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.408791    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 02 14:15:40.409160 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.409138    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 02 14:15:40.409318 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.409186    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 02 14:15:40.409318 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.409283    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gkrvk\""
Jul 02 14:15:40.409437 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.409425    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 02 14:15:40.410525 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.410506    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4vgf6"
Jul 02 14:15:40.410800 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.410782    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 02 14:15:40.413267 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.413246    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p96q8"
Jul 02 14:15:40.413365 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.413344    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 02 14:15:40.413597 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.413574    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 02 14:15:40.413852 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.413826    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 02 14:15:40.413954 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.413840    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9z9cp\""
Jul 02 14:15:40.415719 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.415680    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.416004 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.415973    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 02 14:15:40.416226 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.416189    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ada3d319-cd6a-42ad-b8f9-c04da51705ef-metrics-certs\") pod \"network-metrics-daemon-lpdzr\" (UID: \"ada3d319-cd6a-42ad-b8f9-c04da51705ef\") " pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:15:40.416328 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.416234    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4cb188f8-101f-4d6f-b035-90cf16106a5e-host\") pod \"node-ca-rtn2p\" (UID: \"4cb188f8-101f-4d6f-b035-90cf16106a5e\") " pod="openshift-image-registry/node-ca-rtn2p"
Jul 02 14:15:40.416328 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.416293    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 02 14:15:40.416328 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.416296    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjm84\" (UniqueName: \"kubernetes.io/projected/4cb188f8-101f-4d6f-b035-90cf16106a5e-kube-api-access-cjm84\") pod \"node-ca-rtn2p\" (UID: \"4cb188f8-101f-4d6f-b035-90cf16106a5e\") " pod="openshift-image-registry/node-ca-rtn2p"
Jul 02 14:15:40.416512 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.416333    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b96a56ee-b8ff-41b1-8470-13914745bc35-host-slash\") pod \"iptables-alerter-v5829\" (UID: \"b96a56ee-b8ff-41b1-8470-13914745bc35\") " pod="openshift-network-operator/iptables-alerter-v5829"
Jul 02 14:15:40.416512 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.416340    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pldt7\""
Jul 02 14:15:40.416512 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.416361    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kxgw\" (UniqueName: \"kubernetes.io/projected/b96a56ee-b8ff-41b1-8470-13914745bc35-kube-api-access-2kxgw\") pod \"iptables-alerter-v5829\" (UID: \"b96a56ee-b8ff-41b1-8470-13914745bc35\") " pod="openshift-network-operator/iptables-alerter-v5829"
Jul 02 14:15:40.416512 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.416374    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 02 14:15:40.416512 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.416377    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bb5z\" (UniqueName: \"kubernetes.io/projected/ada3d319-cd6a-42ad-b8f9-c04da51705ef-kube-api-access-8bb5z\") pod \"network-metrics-daemon-lpdzr\" (UID: \"ada3d319-cd6a-42ad-b8f9-c04da51705ef\") " pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:15:40.416512 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.416480    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4cb188f8-101f-4d6f-b035-90cf16106a5e-serviceca\") pod \"node-ca-rtn2p\" (UID: \"4cb188f8-101f-4d6f-b035-90cf16106a5e\") " pod="openshift-image-registry/node-ca-rtn2p"
Jul 02 14:15:40.416714 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.416499    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b96a56ee-b8ff-41b1-8470-13914745bc35-iptables-alerter-script\") pod \"iptables-alerter-v5829\" (UID: \"b96a56ee-b8ff-41b1-8470-13914745bc35\") " pod="openshift-network-operator/iptables-alerter-v5829"
Jul 02 14:15:40.416714 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.416603    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 02 14:15:40.416714 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.416632    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 02 14:15:40.418293 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.418274    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.419027 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.419008    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 02 14:15:40.419466 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.419444    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6bftp\""
Jul 02 14:15:40.420180 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.420154    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 02 14:15:40.420944 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.420927    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 02 14:15:40.421063 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.421041    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:15:40.421137 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:40.421106    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7gvk9" podUID="81c88401-37f6-493c-80ad-c65003c62e57"
Jul 02 14:15:40.421704 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.421688    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-252dm\""
Jul 02 14:15:40.423829 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.423807    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-f8pbz"
Jul 02 14:15:40.426346 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.426326    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-r6m5m"
Jul 02 14:15:40.426932 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.426914    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 02 14:15:40.427045 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.426929    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-rstj9\""
Jul 02 14:15:40.427118 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.427098    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 02 14:15:40.430489 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.430471    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 02 14:15:40.430570 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.430548    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hnw9s\""
Jul 02 14:15:40.430828 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.430815    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 02 14:15:40.458760 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.458726    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 02 14:15:40.473927 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.473883    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal" event={"ID":"d870243e2cd87d194db7d4e9abd907fd","Type":"ContainerStarted","Data":"fa4bb2711ffc38f763e88b52555ce199dffeb5a3c4656e1d1271e04f6848bb18"}
Jul 02 14:15:40.474978 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.474954    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-160.ec2.internal" event={"ID":"e0538fe0bb3712feb96e32d2faab018b","Type":"ContainerStarted","Data":"a90ad4aa75df9ebb965bad31ddda73dc87b2ef512a16bfeb3d5215ef660e1aa9"}
Jul 02 14:15:40.503430 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.503391    2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 02 14:15:40.516878 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.516815    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-etc-sysctl-d\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.516878 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.516859    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-host-run-netns\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.517098 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.516888    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/929e19e8-c3ec-4f7a-810f-cd805cf058f6-multus-daemon-config\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.517098 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.516920    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2ff40ff7-e770-4d86-b327-b437a9cfa15a-system-cni-dir\") pod \"multus-additional-cni-plugins-p96q8\" (UID: \"2ff40ff7-e770-4d86-b327-b437a9cfa15a\") " pod="openshift-multus/multus-additional-cni-plugins-p96q8"
Jul 02 14:15:40.517098 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.516950    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzzfl\" (UniqueName: \"kubernetes.io/projected/76237012-8c88-464a-a19a-dfc41536ea5c-kube-api-access-gzzfl\") pod \"node-resolver-f8pbz\" (UID: \"76237012-8c88-464a-a19a-dfc41536ea5c\") " pod="openshift-dns/node-resolver-f8pbz"
Jul 02 14:15:40.517098 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.516976    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-host-kubelet\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.517098 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517018    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-multus-conf-dir\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.517098 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517058    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bb5z\" (UniqueName: \"kubernetes.io/projected/ada3d319-cd6a-42ad-b8f9-c04da51705ef-kube-api-access-8bb5z\") pod \"network-metrics-daemon-lpdzr\" (UID: \"ada3d319-cd6a-42ad-b8f9-c04da51705ef\") " pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:15:40.517098 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517085    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2ff40ff7-e770-4d86-b327-b437a9cfa15a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p96q8\" (UID: \"2ff40ff7-e770-4d86-b327-b437a9cfa15a\") " pod="openshift-multus/multus-additional-cni-plugins-p96q8"
Jul 02 14:15:40.517343 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517110    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/76237012-8c88-464a-a19a-dfc41536ea5c-hosts-file\") pod \"node-resolver-f8pbz\" (UID: \"76237012-8c88-464a-a19a-dfc41536ea5c\") " pod="openshift-dns/node-resolver-f8pbz"
Jul 02 14:15:40.517343 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517135    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-var-lib-openvswitch\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.517343 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517159    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-node-log\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.517343 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517183    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vmtx\" (UniqueName: \"kubernetes.io/projected/ca909057-dd63-451d-9c05-b671ca5ae671-kube-api-access-2vmtx\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.517343 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517208    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-log-socket\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.517343 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517234    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.517343 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517260    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-run\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.517343 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517284    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d68b1d02-d681-429f-913b-8eedb0835382-etc-selinux\") pod \"aws-ebs-csi-driver-node-4vgf6\" (UID: \"d68b1d02-d681-429f-913b-8eedb0835382\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4vgf6"
Jul 02 14:15:40.517343 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517310    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-host-cni-bin\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.517343 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517338    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-host-cni-netd\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.517681 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517362    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-host-run-k8s-cni-cncf-io\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.517681 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517394    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2ff40ff7-e770-4d86-b327-b437a9cfa15a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p96q8\" (UID: \"2ff40ff7-e770-4d86-b327-b437a9cfa15a\") " pod="openshift-multus/multus-additional-cni-plugins-p96q8"
Jul 02 14:15:40.517681 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517422    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ca909057-dd63-451d-9c05-b671ca5ae671-env-overrides\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.517681 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517447    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9b8d6940-2ec0-442f-ae0f-0e7221348432-tmp\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.517681 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517475    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btppq\" (UniqueName: \"kubernetes.io/projected/9b8d6940-2ec0-442f-ae0f-0e7221348432-kube-api-access-btppq\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.517681 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517505    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-multus-cni-dir\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.517681 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517535    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b96a56ee-b8ff-41b1-8470-13914745bc35-host-slash\") pod \"iptables-alerter-v5829\" (UID: \"b96a56ee-b8ff-41b1-8470-13914745bc35\") " pod="openshift-network-operator/iptables-alerter-v5829"
Jul 02 14:15:40.517681 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517562    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-var-lib-kubelet\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.517681 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517588    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq248\" (UniqueName: \"kubernetes.io/projected/81c88401-37f6-493c-80ad-c65003c62e57-kube-api-access-bq248\") pod \"network-check-target-7gvk9\" (UID: \"81c88401-37f6-493c-80ad-c65003c62e57\") " pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:15:40.517681 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517617    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4cb188f8-101f-4d6f-b035-90cf16106a5e-serviceca\") pod \"node-ca-rtn2p\" (UID: \"4cb188f8-101f-4d6f-b035-90cf16106a5e\") " pod="openshift-image-registry/node-ca-rtn2p"
Jul 02 14:15:40.517681 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517643    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b96a56ee-b8ff-41b1-8470-13914745bc35-iptables-alerter-script\") pod \"iptables-alerter-v5829\" (UID: \"b96a56ee-b8ff-41b1-8470-13914745bc35\") " pod="openshift-network-operator/iptables-alerter-v5829"
Jul 02 14:15:40.517681 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517669    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fd59e931-7137-4fa6-8b4a-3e759e2eebf1-konnectivity-ca\") pod \"konnectivity-agent-r6m5m\" (UID: \"fd59e931-7137-4fa6-8b4a-3e759e2eebf1\") " pod="kube-system/konnectivity-agent-r6m5m"
Jul 02 14:15:40.518117 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517711    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-host-run-ovn-kubernetes\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.518117 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517736    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-lib-modules\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.518117 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517764    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76237012-8c88-464a-a19a-dfc41536ea5c-tmp-dir\") pod \"node-resolver-f8pbz\" (UID: \"76237012-8c88-464a-a19a-dfc41536ea5c\") " pod="openshift-dns/node-resolver-f8pbz"
Jul 02 14:15:40.518117 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517793    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2bb3f329-4286-46e2-8a7d-450769d1aa15-original-pull-secret\") pod \"global-pull-secret-syncer-p64q5\" (UID: \"2bb3f329-4286-46e2-8a7d-450769d1aa15\") " pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:15:40.518117 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517818    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-etc-sysctl-conf\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.518117 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517854    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ada3d319-cd6a-42ad-b8f9-c04da51705ef-metrics-certs\") pod \"network-metrics-daemon-lpdzr\" (UID: \"ada3d319-cd6a-42ad-b8f9-c04da51705ef\") " pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:15:40.518117 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517878    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d68b1d02-d681-429f-913b-8eedb0835382-device-dir\") pod \"aws-ebs-csi-driver-node-4vgf6\" (UID: \"d68b1d02-d681-429f-913b-8eedb0835382\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4vgf6"
Jul 02 14:15:40.518117 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517907    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-host-slash\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.518117 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517935    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-host-run-netns\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.518117 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517961    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2bb3f329-4286-46e2-8a7d-450769d1aa15-dbus\") pod \"global-pull-secret-syncer-p64q5\" (UID: \"2bb3f329-4286-46e2-8a7d-450769d1aa15\") " pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:15:40.518117 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.517987    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-hostroot\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.518117 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518012    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fd59e931-7137-4fa6-8b4a-3e759e2eebf1-agent-certs\") pod \"konnectivity-agent-r6m5m\" (UID: \"fd59e931-7137-4fa6-8b4a-3e759e2eebf1\") " pod="kube-system/konnectivity-agent-r6m5m"
Jul 02 14:15:40.518117 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518053    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-systemd-units\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.518117 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518079    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ca909057-dd63-451d-9c05-b671ca5ae671-ovnkube-config\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.518117 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518105    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-etc-sysconfig\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.518624 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518129    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-etc-kubernetes\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.518624 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518154    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-cnibin\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.518624 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518179    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-etc-kubernetes\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.518624 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518205    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcqv4\" (UniqueName: \"kubernetes.io/projected/d68b1d02-d681-429f-913b-8eedb0835382-kube-api-access-kcqv4\") pod \"aws-ebs-csi-driver-node-4vgf6\" (UID: \"d68b1d02-d681-429f-913b-8eedb0835382\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4vgf6"
Jul 02 14:15:40.518624 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518239    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2ff40ff7-e770-4d86-b327-b437a9cfa15a-cni-binary-copy\") pod \"multus-additional-cni-plugins-p96q8\" (UID: \"2ff40ff7-e770-4d86-b327-b437a9cfa15a\") " pod="openshift-multus/multus-additional-cni-plugins-p96q8"
Jul 02 14:15:40.518624 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518265    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-sys\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.518624 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518298    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9b8d6940-2ec0-442f-ae0f-0e7221348432-etc-tuned\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.518624 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518324    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-multus-socket-dir-parent\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.518624 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518350    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-host-var-lib-cni-multus\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.518624 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518383    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kxgw\" (UniqueName: \"kubernetes.io/projected/b96a56ee-b8ff-41b1-8470-13914745bc35-kube-api-access-2kxgw\") pod \"iptables-alerter-v5829\" (UID: \"b96a56ee-b8ff-41b1-8470-13914745bc35\") " pod="openshift-network-operator/iptables-alerter-v5829"
Jul 02 14:15:40.518624 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518411    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d68b1d02-d681-429f-913b-8eedb0835382-registration-dir\") pod \"aws-ebs-csi-driver-node-4vgf6\" (UID: \"d68b1d02-d681-429f-913b-8eedb0835382\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4vgf6"
Jul 02 14:15:40.518624 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518443    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2ff40ff7-e770-4d86-b327-b437a9cfa15a-cnibin\") pod \"multus-additional-cni-plugins-p96q8\" (UID: \"2ff40ff7-e770-4d86-b327-b437a9cfa15a\") " pod="openshift-multus/multus-additional-cni-plugins-p96q8"
Jul 02 14:15:40.518624 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518469    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ca909057-dd63-451d-9c05-b671ca5ae671-ovn-node-metrics-cert\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.518624 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518496    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ca909057-dd63-451d-9c05-b671ca5ae671-ovnkube-script-lib\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.518624 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518523    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2ff40ff7-e770-4d86-b327-b437a9cfa15a-os-release\") pod \"multus-additional-cni-plugins-p96q8\" (UID: \"2ff40ff7-e770-4d86-b327-b437a9cfa15a\") " pod="openshift-multus/multus-additional-cni-plugins-p96q8"
Jul 02 14:15:40.518624 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518549    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-run-ovn\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.519319 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518574    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-etc-modprobe-d\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.519319 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518598    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-host\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.519319 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518623    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-host-var-lib-cni-bin\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.519319 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518650    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gd8s\" (UniqueName: \"kubernetes.io/projected/929e19e8-c3ec-4f7a-810f-cd805cf058f6-kube-api-access-5gd8s\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.519319 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518678    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4cb188f8-101f-4d6f-b035-90cf16106a5e-host\") pod \"node-ca-rtn2p\" (UID: \"4cb188f8-101f-4d6f-b035-90cf16106a5e\") " pod="openshift-image-registry/node-ca-rtn2p"
Jul 02 14:15:40.519319 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518706    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjm84\" (UniqueName: \"kubernetes.io/projected/4cb188f8-101f-4d6f-b035-90cf16106a5e-kube-api-access-cjm84\") pod \"node-ca-rtn2p\" (UID: \"4cb188f8-101f-4d6f-b035-90cf16106a5e\") " pod="openshift-image-registry/node-ca-rtn2p"
Jul 02 14:15:40.519319 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518737    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d68b1d02-d681-429f-913b-8eedb0835382-socket-dir\") pod \"aws-ebs-csi-driver-node-4vgf6\" (UID: \"d68b1d02-d681-429f-913b-8eedb0835382\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4vgf6"
Jul 02 14:15:40.519319 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518763    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d68b1d02-d681-429f-913b-8eedb0835382-sys-fs\") pod \"aws-ebs-csi-driver-node-4vgf6\" (UID: \"d68b1d02-d681-429f-913b-8eedb0835382\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4vgf6"
Jul 02 14:15:40.519319 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518792    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2ff40ff7-e770-4d86-b327-b437a9cfa15a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p96q8\" (UID: \"2ff40ff7-e770-4d86-b327-b437a9cfa15a\") " pod="openshift-multus/multus-additional-cni-plugins-p96q8"
Jul 02 14:15:40.519319 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518821    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-system-cni-dir\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.519319 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518848    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-os-release\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.519319 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518875    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/929e19e8-c3ec-4f7a-810f-cd805cf058f6-cni-binary-copy\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.519319 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518900    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d68b1d02-d681-429f-913b-8eedb0835382-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4vgf6\" (UID: \"d68b1d02-d681-429f-913b-8eedb0835382\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4vgf6"
Jul 02 14:15:40.519319 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518930    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7xqs\" (UniqueName: \"kubernetes.io/projected/2ff40ff7-e770-4d86-b327-b437a9cfa15a-kube-api-access-s7xqs\") pod \"multus-additional-cni-plugins-p96q8\" (UID: \"2ff40ff7-e770-4d86-b327-b437a9cfa15a\") " pod="openshift-multus/multus-additional-cni-plugins-p96q8"
Jul 02 14:15:40.519319 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518956    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-run-systemd\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.519319 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.518982    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2bb3f329-4286-46e2-8a7d-450769d1aa15-kubelet-config\") pod \"global-pull-secret-syncer-p64q5\" (UID: \"2bb3f329-4286-46e2-8a7d-450769d1aa15\") " pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:15:40.519918 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.519008    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-etc-systemd\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.519918 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.519060    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-host-var-lib-kubelet\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.519918 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.519089    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-host-run-multus-certs\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.519918 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.519116    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-etc-openvswitch\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.519918 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.519142    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-run-openvswitch\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.519918 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.519733    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b96a56ee-b8ff-41b1-8470-13914745bc35-host-slash\") pod \"iptables-alerter-v5829\" (UID: \"b96a56ee-b8ff-41b1-8470-13914745bc35\") " pod="openshift-network-operator/iptables-alerter-v5829"
Jul 02 14:15:40.520327 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.520300    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4cb188f8-101f-4d6f-b035-90cf16106a5e-serviceca\") pod \"node-ca-rtn2p\" (UID: \"4cb188f8-101f-4d6f-b035-90cf16106a5e\") " pod="openshift-image-registry/node-ca-rtn2p"
Jul 02 14:15:40.520406 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.520350    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4cb188f8-101f-4d6f-b035-90cf16106a5e-host\") pod \"node-ca-rtn2p\" (UID: \"4cb188f8-101f-4d6f-b035-90cf16106a5e\") " pod="openshift-image-registry/node-ca-rtn2p"
Jul 02 14:15:40.520459 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:40.520440    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 14:15:40.520587 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:40.520572    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ada3d319-cd6a-42ad-b8f9-c04da51705ef-metrics-certs podName:ada3d319-cd6a-42ad-b8f9-c04da51705ef nodeName:}" failed. No retries permitted until 2026-07-02 14:15:41.020527306 +0000 UTC m=+2.971864958 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ada3d319-cd6a-42ad-b8f9-c04da51705ef-metrics-certs") pod "network-metrics-daemon-lpdzr" (UID: "ada3d319-cd6a-42ad-b8f9-c04da51705ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 14:15:40.520817 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.520770    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b96a56ee-b8ff-41b1-8470-13914745bc35-iptables-alerter-script\") pod \"iptables-alerter-v5829\" (UID: \"b96a56ee-b8ff-41b1-8470-13914745bc35\") " pod="openshift-network-operator/iptables-alerter-v5829"
Jul 02 14:15:40.532913 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.532883    2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 02 14:15:40.536906 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.536845    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjm84\" (UniqueName: \"kubernetes.io/projected/4cb188f8-101f-4d6f-b035-90cf16106a5e-kube-api-access-cjm84\") pod \"node-ca-rtn2p\" (UID: \"4cb188f8-101f-4d6f-b035-90cf16106a5e\") " pod="openshift-image-registry/node-ca-rtn2p"
Jul 02 14:15:40.537121 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.537095    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kxgw\" (UniqueName: \"kubernetes.io/projected/b96a56ee-b8ff-41b1-8470-13914745bc35-kube-api-access-2kxgw\") pod \"iptables-alerter-v5829\" (UID: \"b96a56ee-b8ff-41b1-8470-13914745bc35\") " pod="openshift-network-operator/iptables-alerter-v5829"
Jul 02 14:15:40.537476 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.537448    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bb5z\" (UniqueName: \"kubernetes.io/projected/ada3d319-cd6a-42ad-b8f9-c04da51705ef-kube-api-access-8bb5z\") pod \"network-metrics-daemon-lpdzr\" (UID: \"ada3d319-cd6a-42ad-b8f9-c04da51705ef\") " pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:15:40.538313 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.538283    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 14:15:40.566473 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.566442    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vlwv7"
Jul 02 14:15:40.576076 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.576042    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vlwv7"
Jul 02 14:15:40.620329 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.620289    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fd59e931-7137-4fa6-8b4a-3e759e2eebf1-konnectivity-ca\") pod \"konnectivity-agent-r6m5m\" (UID: \"fd59e931-7137-4fa6-8b4a-3e759e2eebf1\") " pod="kube-system/konnectivity-agent-r6m5m"
Jul 02 14:15:40.620517 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.620359    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-host-run-ovn-kubernetes\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.620517 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.620389    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-lib-modules\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.620517 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.620419    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76237012-8c88-464a-a19a-dfc41536ea5c-tmp-dir\") pod \"node-resolver-f8pbz\" (UID: \"76237012-8c88-464a-a19a-dfc41536ea5c\") " pod="openshift-dns/node-resolver-f8pbz"
Jul 02 14:15:40.620517 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.620473    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2bb3f329-4286-46e2-8a7d-450769d1aa15-original-pull-secret\") pod \"global-pull-secret-syncer-p64q5\" (UID: \"2bb3f329-4286-46e2-8a7d-450769d1aa15\") " pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:15:40.620740 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.620518    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-etc-sysctl-conf\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.620740 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.620556    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d68b1d02-d681-429f-913b-8eedb0835382-device-dir\") pod \"aws-ebs-csi-driver-node-4vgf6\" (UID: \"d68b1d02-d681-429f-913b-8eedb0835382\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4vgf6"
Jul 02 14:15:40.620740 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.620575    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-lib-modules\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.620740 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.620579    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-host-slash\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.620740 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.620605    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-host-run-netns\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.620740 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.620626    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2bb3f329-4286-46e2-8a7d-450769d1aa15-dbus\") pod \"global-pull-secret-syncer-p64q5\" (UID: \"2bb3f329-4286-46e2-8a7d-450769d1aa15\") " pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:15:40.620740 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.620642    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-hostroot\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.620740 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.620659    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fd59e931-7137-4fa6-8b4a-3e759e2eebf1-agent-certs\") pod \"konnectivity-agent-r6m5m\" (UID: \"fd59e931-7137-4fa6-8b4a-3e759e2eebf1\") " pod="kube-system/konnectivity-agent-r6m5m"
Jul 02 14:15:40.620740 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.620674    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-systemd-units\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.620740 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.620689    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ca909057-dd63-451d-9c05-b671ca5ae671-ovnkube-config\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.620740 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.620711    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-etc-sysconfig\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.620740 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.620736    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-etc-kubernetes\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.621471 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.620763    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-cnibin\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.621471 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.620787    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-etc-kubernetes\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.621471 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.620812    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcqv4\" (UniqueName: \"kubernetes.io/projected/d68b1d02-d681-429f-913b-8eedb0835382-kube-api-access-kcqv4\") pod \"aws-ebs-csi-driver-node-4vgf6\" (UID: \"d68b1d02-d681-429f-913b-8eedb0835382\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4vgf6"
Jul 02 14:15:40.621471 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.620836    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2ff40ff7-e770-4d86-b327-b437a9cfa15a-cni-binary-copy\") pod \"multus-additional-cni-plugins-p96q8\" (UID: \"2ff40ff7-e770-4d86-b327-b437a9cfa15a\") " pod="openshift-multus/multus-additional-cni-plugins-p96q8"
Jul 02 14:15:40.621471 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.620856    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-sys\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.621471 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.620884    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9b8d6940-2ec0-442f-ae0f-0e7221348432-etc-tuned\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.621471 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.620910    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-multus-socket-dir-parent\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.621471 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.620916    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76237012-8c88-464a-a19a-dfc41536ea5c-tmp-dir\") pod \"node-resolver-f8pbz\" (UID: \"76237012-8c88-464a-a19a-dfc41536ea5c\") " pod="openshift-dns/node-resolver-f8pbz"
Jul 02 14:15:40.621471 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.620927    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-host-var-lib-cni-multus\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.621471 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.620962    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fd59e931-7137-4fa6-8b4a-3e759e2eebf1-konnectivity-ca\") pod \"konnectivity-agent-r6m5m\" (UID: \"fd59e931-7137-4fa6-8b4a-3e759e2eebf1\") " pod="kube-system/konnectivity-agent-r6m5m"
Jul 02 14:15:40.621471 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.620969    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d68b1d02-d681-429f-913b-8eedb0835382-registration-dir\") pod \"aws-ebs-csi-driver-node-4vgf6\" (UID: \"d68b1d02-d681-429f-913b-8eedb0835382\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4vgf6"
Jul 02 14:15:40.621471 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.621051    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d68b1d02-d681-429f-913b-8eedb0835382-registration-dir\") pod \"aws-ebs-csi-driver-node-4vgf6\" (UID: \"d68b1d02-d681-429f-913b-8eedb0835382\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4vgf6"
Jul 02 14:15:40.621471 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.621068    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2ff40ff7-e770-4d86-b327-b437a9cfa15a-cnibin\") pod \"multus-additional-cni-plugins-p96q8\" (UID: \"2ff40ff7-e770-4d86-b327-b437a9cfa15a\") " pod="openshift-multus/multus-additional-cni-plugins-p96q8"
Jul 02 14:15:40.621471 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.621110    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-etc-sysconfig\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.621471 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.621100    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ca909057-dd63-451d-9c05-b671ca5ae671-ovn-node-metrics-cert\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.621471 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.621143    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ca909057-dd63-451d-9c05-b671ca5ae671-ovnkube-script-lib\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.621471 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.621149    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-etc-kubernetes\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.622385 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.621170    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2ff40ff7-e770-4d86-b327-b437a9cfa15a-os-release\") pod \"multus-additional-cni-plugins-p96q8\" (UID: \"2ff40ff7-e770-4d86-b327-b437a9cfa15a\") " pod="openshift-multus/multus-additional-cni-plugins-p96q8"
Jul 02 14:15:40.622385 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.621196    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-run-ovn\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.622385 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.621225    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-etc-kubernetes\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.622385 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.621227    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-etc-modprobe-d\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.622385 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.621255    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-host\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.622385 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.621280    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-host-var-lib-cni-bin\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.622385 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.621310    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gd8s\" (UniqueName: \"kubernetes.io/projected/929e19e8-c3ec-4f7a-810f-cd805cf058f6-kube-api-access-5gd8s\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.622385 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.621342    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d68b1d02-d681-429f-913b-8eedb0835382-socket-dir\") pod \"aws-ebs-csi-driver-node-4vgf6\" (UID: \"d68b1d02-d681-429f-913b-8eedb0835382\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4vgf6"
Jul 02 14:15:40.622385 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.621368    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d68b1d02-d681-429f-913b-8eedb0835382-sys-fs\") pod \"aws-ebs-csi-driver-node-4vgf6\" (UID: \"d68b1d02-d681-429f-913b-8eedb0835382\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4vgf6"
Jul 02 14:15:40.622385 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.621396    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2ff40ff7-e770-4d86-b327-b437a9cfa15a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p96q8\" (UID: \"2ff40ff7-e770-4d86-b327-b437a9cfa15a\") " pod="openshift-multus/multus-additional-cni-plugins-p96q8"
Jul 02 14:15:40.622385 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.621425    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-system-cni-dir\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.622385 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.621452    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-os-release\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.622385 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.621476    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ca909057-dd63-451d-9c05-b671ca5ae671-ovnkube-config\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.622385 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.621800    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2ff40ff7-e770-4d86-b327-b437a9cfa15a-cnibin\") pod \"multus-additional-cni-plugins-p96q8\" (UID: \"2ff40ff7-e770-4d86-b327-b437a9cfa15a\") " pod="openshift-multus/multus-additional-cni-plugins-p96q8"
Jul 02 14:15:40.622385 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.621897    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-sys\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.622385 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.622348    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-multus-socket-dir-parent\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.622385 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.622367    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-host-run-ovn-kubernetes\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.623176 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.622430    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-host-run-netns\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.623176 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.622485    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ca909057-dd63-451d-9c05-b671ca5ae671-ovnkube-script-lib\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.623176 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.622501    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d68b1d02-d681-429f-913b-8eedb0835382-socket-dir\") pod \"aws-ebs-csi-driver-node-4vgf6\" (UID: \"d68b1d02-d681-429f-913b-8eedb0835382\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4vgf6"
Jul 02 14:15:40.623176 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.621200    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-cnibin\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.623176 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:40.622543    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 14:15:40.623176 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.622597    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d68b1d02-d681-429f-913b-8eedb0835382-sys-fs\") pod \"aws-ebs-csi-driver-node-4vgf6\" (UID: \"d68b1d02-d681-429f-913b-8eedb0835382\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4vgf6"
Jul 02 14:15:40.623176 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.622610    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2ff40ff7-e770-4d86-b327-b437a9cfa15a-os-release\") pod \"multus-additional-cni-plugins-p96q8\" (UID: \"2ff40ff7-e770-4d86-b327-b437a9cfa15a\") " pod="openshift-multus/multus-additional-cni-plugins-p96q8"
Jul 02 14:15:40.623176 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:40.622688    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bb3f329-4286-46e2-8a7d-450769d1aa15-original-pull-secret podName:2bb3f329-4286-46e2-8a7d-450769d1aa15 nodeName:}" failed. No retries permitted until 2026-07-02 14:15:41.12266903 +0000 UTC m=+3.074006672 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2bb3f329-4286-46e2-8a7d-450769d1aa15-original-pull-secret") pod "global-pull-secret-syncer-p64q5" (UID: "2bb3f329-4286-46e2-8a7d-450769d1aa15") : object "kube-system"/"original-pull-secret" not registered
Jul 02 14:15:40.623176 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.622741    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-host\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.623176 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.622792    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-host-var-lib-cni-bin\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.623176 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.623052    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2ff40ff7-e770-4d86-b327-b437a9cfa15a-cni-binary-copy\") pod \"multus-additional-cni-plugins-p96q8\" (UID: \"2ff40ff7-e770-4d86-b327-b437a9cfa15a\") " pod="openshift-multus/multus-additional-cni-plugins-p96q8"
Jul 02 14:15:40.623176 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.623143    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-etc-sysctl-conf\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.623176 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.623169    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2ff40ff7-e770-4d86-b327-b437a9cfa15a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p96q8\" (UID: \"2ff40ff7-e770-4d86-b327-b437a9cfa15a\") " pod="openshift-multus/multus-additional-cni-plugins-p96q8"
Jul 02 14:15:40.623800 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.623211    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d68b1d02-d681-429f-913b-8eedb0835382-device-dir\") pod \"aws-ebs-csi-driver-node-4vgf6\" (UID: \"d68b1d02-d681-429f-913b-8eedb0835382\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4vgf6"
Jul 02 14:15:40.623800 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.623280    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-host-slash\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.623800 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.623322    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-etc-modprobe-d\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.623800 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.623380    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-run-ovn\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.623800 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.623441    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-system-cni-dir\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.623800 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.623442    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-host-var-lib-cni-multus\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.623800 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.623565    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-os-release\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.624225 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.624081    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-systemd-units\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.624225 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.624169    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-hostroot\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.624225 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.621476    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/929e19e8-c3ec-4f7a-810f-cd805cf058f6-cni-binary-copy\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.624366 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.624227    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d68b1d02-d681-429f-913b-8eedb0835382-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4vgf6\" (UID: \"d68b1d02-d681-429f-913b-8eedb0835382\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4vgf6"
Jul 02 14:15:40.624366 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.624264    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7xqs\" (UniqueName: \"kubernetes.io/projected/2ff40ff7-e770-4d86-b327-b437a9cfa15a-kube-api-access-s7xqs\") pod \"multus-additional-cni-plugins-p96q8\" (UID: \"2ff40ff7-e770-4d86-b327-b437a9cfa15a\") " pod="openshift-multus/multus-additional-cni-plugins-p96q8"
Jul 02 14:15:40.624366 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.624327    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2bb3f329-4286-46e2-8a7d-450769d1aa15-dbus\") pod \"global-pull-secret-syncer-p64q5\" (UID: \"2bb3f329-4286-46e2-8a7d-450769d1aa15\") " pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:15:40.624505 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.624361    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/929e19e8-c3ec-4f7a-810f-cd805cf058f6-cni-binary-copy\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.624505 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.624384    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d68b1d02-d681-429f-913b-8eedb0835382-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4vgf6\" (UID: \"d68b1d02-d681-429f-913b-8eedb0835382\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4vgf6"
Jul 02 14:15:40.624505 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.624292    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-run-systemd\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.624505 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.624439    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2bb3f329-4286-46e2-8a7d-450769d1aa15-kubelet-config\") pod \"global-pull-secret-syncer-p64q5\" (UID: \"2bb3f329-4286-46e2-8a7d-450769d1aa15\") " pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:15:40.624505 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.624471    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-etc-systemd\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.624729 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.624550    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-host-var-lib-kubelet\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.624729 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.624576    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-host-run-multus-certs\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.624729 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.624613    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-etc-openvswitch\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.624729 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.624712    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-run-openvswitch\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.624918 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.624743    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-etc-sysctl-d\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.624918 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.624812    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-host-var-lib-kubelet\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.624918 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.624777    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-host-run-netns\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.624918 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.624866    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/929e19e8-c3ec-4f7a-810f-cd805cf058f6-multus-daemon-config\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.624918 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.624897    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2ff40ff7-e770-4d86-b327-b437a9cfa15a-system-cni-dir\") pod \"multus-additional-cni-plugins-p96q8\" (UID: \"2ff40ff7-e770-4d86-b327-b437a9cfa15a\") " pod="openshift-multus/multus-additional-cni-plugins-p96q8"
Jul 02 14:15:40.625178 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.624948    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-run-systemd\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.625178 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.625009    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2bb3f329-4286-46e2-8a7d-450769d1aa15-kubelet-config\") pod \"global-pull-secret-syncer-p64q5\" (UID: \"2bb3f329-4286-46e2-8a7d-450769d1aa15\") " pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:15:40.625178 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.625075    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzzfl\" (UniqueName: \"kubernetes.io/projected/76237012-8c88-464a-a19a-dfc41536ea5c-kube-api-access-gzzfl\") pod \"node-resolver-f8pbz\" (UID: \"76237012-8c88-464a-a19a-dfc41536ea5c\") " pod="openshift-dns/node-resolver-f8pbz"
Jul 02 14:15:40.625178 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.625077    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-etc-systemd\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.625178 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.625122    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-host-kubelet\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.625410 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.625149    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-multus-conf-dir\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.625410 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.625284    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2ff40ff7-e770-4d86-b327-b437a9cfa15a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p96q8\" (UID: \"2ff40ff7-e770-4d86-b327-b437a9cfa15a\") " pod="openshift-multus/multus-additional-cni-plugins-p96q8"
Jul 02 14:15:40.625410 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.625312    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/76237012-8c88-464a-a19a-dfc41536ea5c-hosts-file\") pod \"node-resolver-f8pbz\" (UID: \"76237012-8c88-464a-a19a-dfc41536ea5c\") " pod="openshift-dns/node-resolver-f8pbz"
Jul 02 14:15:40.625557 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.625484    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-var-lib-openvswitch\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.625557 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.625517    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-node-log\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.625557 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.625522    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-host-run-multus-certs\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.625557 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.625542    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vmtx\" (UniqueName: \"kubernetes.io/projected/ca909057-dd63-451d-9c05-b671ca5ae671-kube-api-access-2vmtx\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.625745 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.625570    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-log-socket\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.625745 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.625591    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-etc-openvswitch\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.625745 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.625599    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.625745 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.625661    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-run-openvswitch\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.625745 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.625627    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-run\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.625745 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.625743    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d68b1d02-d681-429f-913b-8eedb0835382-etc-selinux\") pod \"aws-ebs-csi-driver-node-4vgf6\" (UID: \"d68b1d02-d681-429f-913b-8eedb0835382\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4vgf6"
Jul 02 14:15:40.626108 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.625767    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-host-cni-bin\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.626108 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.625783    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-etc-sysctl-d\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.626108 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.625795    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-host-cni-netd\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.626108 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.625824    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-host-run-k8s-cni-cncf-io\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.626108 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.625832    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-node-log\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.626108 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.625862    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-host-run-netns\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.626108 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.625917    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-host-kubelet\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.626108 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.625971    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-multus-conf-dir\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.626108 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.625903    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2ff40ff7-e770-4d86-b327-b437a9cfa15a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p96q8\" (UID: \"2ff40ff7-e770-4d86-b327-b437a9cfa15a\") " pod="openshift-multus/multus-additional-cni-plugins-p96q8"
Jul 02 14:15:40.626108 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.626020    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ca909057-dd63-451d-9c05-b671ca5ae671-env-overrides\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.626108 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.626078    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9b8d6940-2ec0-442f-ae0f-0e7221348432-tmp\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.626108 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.626107    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btppq\" (UniqueName: \"kubernetes.io/projected/9b8d6940-2ec0-442f-ae0f-0e7221348432-kube-api-access-btppq\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.626686 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.626142    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-multus-cni-dir\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.626686 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.626178    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-var-lib-kubelet\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.626686 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.626213    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq248\" (UniqueName: \"kubernetes.io/projected/81c88401-37f6-493c-80ad-c65003c62e57-kube-api-access-bq248\") pod \"network-check-target-7gvk9\" (UID: \"81c88401-37f6-493c-80ad-c65003c62e57\") " pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:15:40.626686 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.626423    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/929e19e8-c3ec-4f7a-810f-cd805cf058f6-multus-daemon-config\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.626686 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.626484    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-host-run-k8s-cni-cncf-io\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.626686 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.626487    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2ff40ff7-e770-4d86-b327-b437a9cfa15a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p96q8\" (UID: \"2ff40ff7-e770-4d86-b327-b437a9cfa15a\") " pod="openshift-multus/multus-additional-cni-plugins-p96q8"
Jul 02 14:15:40.626686 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.626567    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2ff40ff7-e770-4d86-b327-b437a9cfa15a-system-cni-dir\") pod \"multus-additional-cni-plugins-p96q8\" (UID: \"2ff40ff7-e770-4d86-b327-b437a9cfa15a\") " pod="openshift-multus/multus-additional-cni-plugins-p96q8"
Jul 02 14:15:40.627005 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.626722    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2ff40ff7-e770-4d86-b327-b437a9cfa15a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p96q8\" (UID: \"2ff40ff7-e770-4d86-b327-b437a9cfa15a\") " pod="openshift-multus/multus-additional-cni-plugins-p96q8"
Jul 02 14:15:40.627005 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.626917    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-log-socket\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.627005 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.626976    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.627197 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.627025    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-run\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.627197 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.627139    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d68b1d02-d681-429f-913b-8eedb0835382-etc-selinux\") pod \"aws-ebs-csi-driver-node-4vgf6\" (UID: \"d68b1d02-d681-429f-913b-8eedb0835382\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4vgf6"
Jul 02 14:15:40.627197 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.627182    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-host-cni-bin\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.627345 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.627226    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-host-cni-netd\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.627345 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.627249    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/929e19e8-c3ec-4f7a-810f-cd805cf058f6-multus-cni-dir\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.627345 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.627268    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ca909057-dd63-451d-9c05-b671ca5ae671-var-lib-openvswitch\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.627345 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.627334    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/76237012-8c88-464a-a19a-dfc41536ea5c-hosts-file\") pod \"node-resolver-f8pbz\" (UID: \"76237012-8c88-464a-a19a-dfc41536ea5c\") " pod="openshift-dns/node-resolver-f8pbz"
Jul 02 14:15:40.627527 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.627365    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b8d6940-2ec0-442f-ae0f-0e7221348432-var-lib-kubelet\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.627527 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.627405    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ca909057-dd63-451d-9c05-b671ca5ae671-ovn-node-metrics-cert\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.627728 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.627709    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ca909057-dd63-451d-9c05-b671ca5ae671-env-overrides\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.632118 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.631759    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9b8d6940-2ec0-442f-ae0f-0e7221348432-tmp\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.632118 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.631970    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9b8d6940-2ec0-442f-ae0f-0e7221348432-etc-tuned\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.632118 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.632061    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fd59e931-7137-4fa6-8b4a-3e759e2eebf1-agent-certs\") pod \"konnectivity-agent-r6m5m\" (UID: \"fd59e931-7137-4fa6-8b4a-3e759e2eebf1\") " pod="kube-system/konnectivity-agent-r6m5m"
Jul 02 14:15:40.635056 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.634960    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzzfl\" (UniqueName: \"kubernetes.io/projected/76237012-8c88-464a-a19a-dfc41536ea5c-kube-api-access-gzzfl\") pod \"node-resolver-f8pbz\" (UID: \"76237012-8c88-464a-a19a-dfc41536ea5c\") " pod="openshift-dns/node-resolver-f8pbz"
Jul 02 14:15:40.635056 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.634965    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7xqs\" (UniqueName: \"kubernetes.io/projected/2ff40ff7-e770-4d86-b327-b437a9cfa15a-kube-api-access-s7xqs\") pod \"multus-additional-cni-plugins-p96q8\" (UID: \"2ff40ff7-e770-4d86-b327-b437a9cfa15a\") " pod="openshift-multus/multus-additional-cni-plugins-p96q8"
Jul 02 14:15:40.635543 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.635520    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcqv4\" (UniqueName: \"kubernetes.io/projected/d68b1d02-d681-429f-913b-8eedb0835382-kube-api-access-kcqv4\") pod \"aws-ebs-csi-driver-node-4vgf6\" (UID: \"d68b1d02-d681-429f-913b-8eedb0835382\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4vgf6"
Jul 02 14:15:40.637193 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.637168    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gd8s\" (UniqueName: \"kubernetes.io/projected/929e19e8-c3ec-4f7a-810f-cd805cf058f6-kube-api-access-5gd8s\") pod \"multus-r2rq9\" (UID: \"929e19e8-c3ec-4f7a-810f-cd805cf058f6\") " pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.647662 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:40.647636    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 14:15:40.647662 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:40.647665    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 14:15:40.647662 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:40.647679    2582 projected.go:196] Error preparing data for projected volume kube-api-access-bq248 for pod openshift-network-diagnostics/network-check-target-7gvk9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 14:15:40.647942 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:40.647764    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/81c88401-37f6-493c-80ad-c65003c62e57-kube-api-access-bq248 podName:81c88401-37f6-493c-80ad-c65003c62e57 nodeName:}" failed. No retries permitted until 2026-07-02 14:15:41.147742761 +0000 UTC m=+3.099080407 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bq248" (UniqueName: "kubernetes.io/projected/81c88401-37f6-493c-80ad-c65003c62e57-kube-api-access-bq248") pod "network-check-target-7gvk9" (UID: "81c88401-37f6-493c-80ad-c65003c62e57") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 14:15:40.650390 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.650369    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btppq\" (UniqueName: \"kubernetes.io/projected/9b8d6940-2ec0-442f-ae0f-0e7221348432-kube-api-access-btppq\") pod \"tuned-q6zgx\" (UID: \"9b8d6940-2ec0-442f-ae0f-0e7221348432\") " pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.650665 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.650645    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vmtx\" (UniqueName: \"kubernetes.io/projected/ca909057-dd63-451d-9c05-b671ca5ae671-kube-api-access-2vmtx\") pod \"ovnkube-node-cnwb5\" (UID: \"ca909057-dd63-451d-9c05-b671ca5ae671\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.710458 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.710420    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rtn2p"
Jul 02 14:15:40.719203 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.719177    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-v5829"
Jul 02 14:15:40.728538 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.728513    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:15:40.738143 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.738111    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4vgf6"
Jul 02 14:15:40.744299 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.744274    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p96q8"
Jul 02 14:15:40.751237 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.751210    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q6zgx"
Jul 02 14:15:40.758503 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.758464    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-r2rq9"
Jul 02 14:15:40.763501 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.763477    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-f8pbz"
Jul 02 14:15:40.768331 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:40.768309    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-r6m5m"
Jul 02 14:15:41.030702 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:41.030665    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ada3d319-cd6a-42ad-b8f9-c04da51705ef-metrics-certs\") pod \"network-metrics-daemon-lpdzr\" (UID: \"ada3d319-cd6a-42ad-b8f9-c04da51705ef\") " pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:15:41.030860 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:41.030836    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 14:15:41.030943 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:41.030915    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ada3d319-cd6a-42ad-b8f9-c04da51705ef-metrics-certs podName:ada3d319-cd6a-42ad-b8f9-c04da51705ef nodeName:}" failed. No retries permitted until 2026-07-02 14:15:42.030896029 +0000 UTC m=+3.982233657 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ada3d319-cd6a-42ad-b8f9-c04da51705ef-metrics-certs") pod "network-metrics-daemon-lpdzr" (UID: "ada3d319-cd6a-42ad-b8f9-c04da51705ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 14:15:41.132806 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:41.132760    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2bb3f329-4286-46e2-8a7d-450769d1aa15-original-pull-secret\") pod \"global-pull-secret-syncer-p64q5\" (UID: \"2bb3f329-4286-46e2-8a7d-450769d1aa15\") " pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:15:41.133004 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:41.132921    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 14:15:41.133048 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:41.133014    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bb3f329-4286-46e2-8a7d-450769d1aa15-original-pull-secret podName:2bb3f329-4286-46e2-8a7d-450769d1aa15 nodeName:}" failed. No retries permitted until 2026-07-02 14:15:42.132995932 +0000 UTC m=+4.084333559 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2bb3f329-4286-46e2-8a7d-450769d1aa15-original-pull-secret") pod "global-pull-secret-syncer-p64q5" (UID: "2bb3f329-4286-46e2-8a7d-450769d1aa15") : object "kube-system"/"original-pull-secret" not registered
Jul 02 14:15:41.233388 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:41.233328    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq248\" (UniqueName: \"kubernetes.io/projected/81c88401-37f6-493c-80ad-c65003c62e57-kube-api-access-bq248\") pod \"network-check-target-7gvk9\" (UID: \"81c88401-37f6-493c-80ad-c65003c62e57\") " pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:15:41.233572 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:41.233504    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 14:15:41.233572 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:41.233537    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 14:15:41.233572 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:41.233552    2582 projected.go:196] Error preparing data for projected volume kube-api-access-bq248 for pod openshift-network-diagnostics/network-check-target-7gvk9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 14:15:41.233734 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:41.233617    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/81c88401-37f6-493c-80ad-c65003c62e57-kube-api-access-bq248 podName:81c88401-37f6-493c-80ad-c65003c62e57 nodeName:}" failed. No retries permitted until 2026-07-02 14:15:42.233596844 +0000 UTC m=+4.184934493 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq248" (UniqueName: "kubernetes.io/projected/81c88401-37f6-493c-80ad-c65003c62e57-kube-api-access-bq248") pod "network-check-target-7gvk9" (UID: "81c88401-37f6-493c-80ad-c65003c62e57") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 14:15:41.470360 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:41.470317    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:15:41.470957 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:41.470444    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpdzr" podUID="ada3d319-cd6a-42ad-b8f9-c04da51705ef"
Jul 02 14:15:41.580119 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:41.580070    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 14:10:41 +0000 UTC" deadline="2028-01-10 17:14:24.974167499 +0000 UTC"
Jul 02 14:15:41.580119 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:41.580106    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13370h58m43.394064949s"
Jul 02 14:15:41.768202 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:41.768168    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb96a56ee_b8ff_41b1_8470_13914745bc35.slice/crio-b3bef2ff05ac3f1717e4ba0a86f36f414280a3dc232c7433776cf6676064cde3 WatchSource:0}: Error finding container b3bef2ff05ac3f1717e4ba0a86f36f414280a3dc232c7433776cf6676064cde3: Status 404 returned error can't find the container with id b3bef2ff05ac3f1717e4ba0a86f36f414280a3dc232c7433776cf6676064cde3
Jul 02 14:15:41.770157 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:41.770125    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ff40ff7_e770_4d86_b327_b437a9cfa15a.slice/crio-e5cb8b9795ec6289c9f0ebd9290d900960935591a9a2b60238d292867f8dbcd7 WatchSource:0}: Error finding container e5cb8b9795ec6289c9f0ebd9290d900960935591a9a2b60238d292867f8dbcd7: Status 404 returned error can't find the container with id e5cb8b9795ec6289c9f0ebd9290d900960935591a9a2b60238d292867f8dbcd7
Jul 02 14:15:41.772571 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:41.772377    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd68b1d02_d681_429f_913b_8eedb0835382.slice/crio-e8c05832cad50ad475e1e86c44ed5631ddcc5e79b4a8055172534ca74e61c1ac WatchSource:0}: Error finding container e8c05832cad50ad475e1e86c44ed5631ddcc5e79b4a8055172534ca74e61c1ac: Status 404 returned error can't find the container with id e8c05832cad50ad475e1e86c44ed5631ddcc5e79b4a8055172534ca74e61c1ac
Jul 02 14:15:41.773413 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:41.773384    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca909057_dd63_451d_9c05_b671ca5ae671.slice/crio-bc25f02d1c21fa82987ddce7e36a2149b7d99ff96b0a940682ceba4514f9d987 WatchSource:0}: Error finding container bc25f02d1c21fa82987ddce7e36a2149b7d99ff96b0a940682ceba4514f9d987: Status 404 returned error can't find the container with id bc25f02d1c21fa82987ddce7e36a2149b7d99ff96b0a940682ceba4514f9d987
Jul 02 14:15:41.780254 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:41.780233    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cb188f8_101f_4d6f_b035_90cf16106a5e.slice/crio-9babafeda90af05ce9a4fcda78c7ebe8c4c14db648f3b6ea4cd9daa021054108 WatchSource:0}: Error finding container 9babafeda90af05ce9a4fcda78c7ebe8c4c14db648f3b6ea4cd9daa021054108: Status 404 returned error can't find the container with id 9babafeda90af05ce9a4fcda78c7ebe8c4c14db648f3b6ea4cd9daa021054108
Jul 02 14:15:41.781941 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:15:41.781915    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod929e19e8_c3ec_4f7a_810f_cd805cf058f6.slice/crio-d277864d9027cf1dc8227b42c87f785b43bc0861edb63cf7196334cecf3f7b74 WatchSource:0}: Error finding container d277864d9027cf1dc8227b42c87f785b43bc0861edb63cf7196334cecf3f7b74: Status 404 returned error can't find the container with id d277864d9027cf1dc8227b42c87f785b43bc0861edb63cf7196334cecf3f7b74
Jul 02 14:15:42.039253 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:42.039083    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ada3d319-cd6a-42ad-b8f9-c04da51705ef-metrics-certs\") pod \"network-metrics-daemon-lpdzr\" (UID: \"ada3d319-cd6a-42ad-b8f9-c04da51705ef\") " pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:15:42.039379 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:42.039232    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 14:15:42.039379 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:42.039345    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ada3d319-cd6a-42ad-b8f9-c04da51705ef-metrics-certs podName:ada3d319-cd6a-42ad-b8f9-c04da51705ef nodeName:}" failed. No retries permitted until 2026-07-02 14:15:44.03932902 +0000 UTC m=+5.990666646 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ada3d319-cd6a-42ad-b8f9-c04da51705ef-metrics-certs") pod "network-metrics-daemon-lpdzr" (UID: "ada3d319-cd6a-42ad-b8f9-c04da51705ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 14:15:42.139847 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:42.139816    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2bb3f329-4286-46e2-8a7d-450769d1aa15-original-pull-secret\") pod \"global-pull-secret-syncer-p64q5\" (UID: \"2bb3f329-4286-46e2-8a7d-450769d1aa15\") " pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:15:42.139990 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:42.139953    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 14:15:42.140052 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:42.140040    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bb3f329-4286-46e2-8a7d-450769d1aa15-original-pull-secret podName:2bb3f329-4286-46e2-8a7d-450769d1aa15 nodeName:}" failed. No retries permitted until 2026-07-02 14:15:44.140020548 +0000 UTC m=+6.091358176 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2bb3f329-4286-46e2-8a7d-450769d1aa15-original-pull-secret") pod "global-pull-secret-syncer-p64q5" (UID: "2bb3f329-4286-46e2-8a7d-450769d1aa15") : object "kube-system"/"original-pull-secret" not registered
Jul 02 14:15:42.240257 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:42.240214    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq248\" (UniqueName: \"kubernetes.io/projected/81c88401-37f6-493c-80ad-c65003c62e57-kube-api-access-bq248\") pod \"network-check-target-7gvk9\" (UID: \"81c88401-37f6-493c-80ad-c65003c62e57\") " pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:15:42.240462 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:42.240439    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 14:15:42.240521 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:42.240470    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 14:15:42.240521 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:42.240493    2582 projected.go:196] Error preparing data for projected volume kube-api-access-bq248 for pod openshift-network-diagnostics/network-check-target-7gvk9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 14:15:42.240619 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:42.240562    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/81c88401-37f6-493c-80ad-c65003c62e57-kube-api-access-bq248 podName:81c88401-37f6-493c-80ad-c65003c62e57 nodeName:}" failed. No retries permitted until 2026-07-02 14:15:44.24054248 +0000 UTC m=+6.191880113 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq248" (UniqueName: "kubernetes.io/projected/81c88401-37f6-493c-80ad-c65003c62e57-kube-api-access-bq248") pod "network-check-target-7gvk9" (UID: "81c88401-37f6-493c-80ad-c65003c62e57") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 14:15:42.470575 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:42.470546    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:15:42.471107 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:42.470658    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p64q5" podUID="2bb3f329-4286-46e2-8a7d-450769d1aa15"
Jul 02 14:15:42.471107 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:42.470962    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:15:42.471107 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:42.471054    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7gvk9" podUID="81c88401-37f6-493c-80ad-c65003c62e57"
Jul 02 14:15:42.482629 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:42.482597    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-r6m5m" event={"ID":"fd59e931-7137-4fa6-8b4a-3e759e2eebf1","Type":"ContainerStarted","Data":"f496b5432deec49852a1e0fa5eaf174e122b93a7ba32c663521f2fcf9abb2eda"}
Jul 02 14:15:42.484246 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:42.484216    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q6zgx" event={"ID":"9b8d6940-2ec0-442f-ae0f-0e7221348432","Type":"ContainerStarted","Data":"dface25a16433aabeb7c7626c2f9ee3af54a8d3ceebe3bcad53e31fbdcbf5b25"}
Jul 02 14:15:42.486567 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:42.486533    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4vgf6" event={"ID":"d68b1d02-d681-429f-913b-8eedb0835382","Type":"ContainerStarted","Data":"e8c05832cad50ad475e1e86c44ed5631ddcc5e79b4a8055172534ca74e61c1ac"}
Jul 02 14:15:42.488476 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:42.488446    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p96q8" event={"ID":"2ff40ff7-e770-4d86-b327-b437a9cfa15a","Type":"ContainerStarted","Data":"e5cb8b9795ec6289c9f0ebd9290d900960935591a9a2b60238d292867f8dbcd7"}
Jul 02 14:15:42.494391 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:42.494285    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-v5829" event={"ID":"b96a56ee-b8ff-41b1-8470-13914745bc35","Type":"ContainerStarted","Data":"b3bef2ff05ac3f1717e4ba0a86f36f414280a3dc232c7433776cf6676064cde3"}
Jul 02 14:15:42.498007 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:42.497969    2582 generic.go:356] "Generic (PLEG): container finished" podID="d870243e2cd87d194db7d4e9abd907fd" containerID="bf800f6c7def24c724bbe87413bfc934677a5eac17661c51a51a5860bc2d9096" exitCode=0
Jul 02 14:15:42.498257 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:42.498228    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal" event={"ID":"d870243e2cd87d194db7d4e9abd907fd","Type":"ContainerDied","Data":"bf800f6c7def24c724bbe87413bfc934677a5eac17661c51a51a5860bc2d9096"}
Jul 02 14:15:42.500226 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:42.500192    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-f8pbz" event={"ID":"76237012-8c88-464a-a19a-dfc41536ea5c","Type":"ContainerStarted","Data":"6dba66b642bb78cc2de9b903de7a0c3f3b161224ca785e88cb511c7a9a195374"}
Jul 02 14:15:42.503451 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:42.503425    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-r2rq9" event={"ID":"929e19e8-c3ec-4f7a-810f-cd805cf058f6","Type":"ContainerStarted","Data":"d277864d9027cf1dc8227b42c87f785b43bc0861edb63cf7196334cecf3f7b74"}
Jul 02 14:15:42.506771 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:42.506676    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5" event={"ID":"ca909057-dd63-451d-9c05-b671ca5ae671","Type":"ContainerStarted","Data":"bc25f02d1c21fa82987ddce7e36a2149b7d99ff96b0a940682ceba4514f9d987"}
Jul 02 14:15:42.508551 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:42.508510    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rtn2p" event={"ID":"4cb188f8-101f-4d6f-b035-90cf16106a5e","Type":"ContainerStarted","Data":"9babafeda90af05ce9a4fcda78c7ebe8c4c14db648f3b6ea4cd9daa021054108"}
Jul 02 14:15:42.510533 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:42.510510    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-160.ec2.internal" event={"ID":"e0538fe0bb3712feb96e32d2faab018b","Type":"ContainerStarted","Data":"e725754deb23d6e07dd1680e1c00025de600f50103ea4a70ebeff8a3a72c0d17"}
Jul 02 14:15:42.581205 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:42.581108    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 14:10:41 +0000 UTC" deadline="2027-12-06 09:43:20.75867707 +0000 UTC"
Jul 02 14:15:42.581205 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:42.581168    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12523h27m38.177513407s"
Jul 02 14:15:43.469830 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:43.469797    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:15:43.470071 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:43.469917    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpdzr" podUID="ada3d319-cd6a-42ad-b8f9-c04da51705ef"
Jul 02 14:15:43.530176 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:43.530136    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal" event={"ID":"d870243e2cd87d194db7d4e9abd907fd","Type":"ContainerStarted","Data":"ff682b6760f961a218ea7fd7e64beee1c4ca34297c1850e18841c205a7b824df"}
Jul 02 14:15:43.551460 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:43.550519    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-160.ec2.internal" podStartSLOduration=3.550496951 podStartE2EDuration="3.550496951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:15:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:15:42.550801606 +0000 UTC m=+4.502139256" watchObservedRunningTime="2026-07-02 14:15:43.550496951 +0000 UTC m=+5.501834601"
Jul 02 14:15:44.055800 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:44.055759    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ada3d319-cd6a-42ad-b8f9-c04da51705ef-metrics-certs\") pod \"network-metrics-daemon-lpdzr\" (UID: \"ada3d319-cd6a-42ad-b8f9-c04da51705ef\") " pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:15:44.056085 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:44.055934    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 14:15:44.056085 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:44.056027    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ada3d319-cd6a-42ad-b8f9-c04da51705ef-metrics-certs podName:ada3d319-cd6a-42ad-b8f9-c04da51705ef nodeName:}" failed. No retries permitted until 2026-07-02 14:15:48.055986271 +0000 UTC m=+10.007323913 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ada3d319-cd6a-42ad-b8f9-c04da51705ef-metrics-certs") pod "network-metrics-daemon-lpdzr" (UID: "ada3d319-cd6a-42ad-b8f9-c04da51705ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 14:15:44.156622 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:44.156577    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2bb3f329-4286-46e2-8a7d-450769d1aa15-original-pull-secret\") pod \"global-pull-secret-syncer-p64q5\" (UID: \"2bb3f329-4286-46e2-8a7d-450769d1aa15\") " pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:15:44.156830 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:44.156728    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 14:15:44.156830 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:44.156801    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bb3f329-4286-46e2-8a7d-450769d1aa15-original-pull-secret podName:2bb3f329-4286-46e2-8a7d-450769d1aa15 nodeName:}" failed. No retries permitted until 2026-07-02 14:15:48.156780579 +0000 UTC m=+10.108118221 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2bb3f329-4286-46e2-8a7d-450769d1aa15-original-pull-secret") pod "global-pull-secret-syncer-p64q5" (UID: "2bb3f329-4286-46e2-8a7d-450769d1aa15") : object "kube-system"/"original-pull-secret" not registered
Jul 02 14:15:44.257899 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:44.257858    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq248\" (UniqueName: \"kubernetes.io/projected/81c88401-37f6-493c-80ad-c65003c62e57-kube-api-access-bq248\") pod \"network-check-target-7gvk9\" (UID: \"81c88401-37f6-493c-80ad-c65003c62e57\") " pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:15:44.258104 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:44.258054    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 14:15:44.258104 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:44.258077    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 14:15:44.258104 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:44.258089    2582 projected.go:196] Error preparing data for projected volume kube-api-access-bq248 for pod openshift-network-diagnostics/network-check-target-7gvk9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 14:15:44.258322 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:44.258152    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/81c88401-37f6-493c-80ad-c65003c62e57-kube-api-access-bq248 podName:81c88401-37f6-493c-80ad-c65003c62e57 nodeName:}" failed. No retries permitted until 2026-07-02 14:15:48.25813349 +0000 UTC m=+10.209471138 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq248" (UniqueName: "kubernetes.io/projected/81c88401-37f6-493c-80ad-c65003c62e57-kube-api-access-bq248") pod "network-check-target-7gvk9" (UID: "81c88401-37f6-493c-80ad-c65003c62e57") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 14:15:44.470135 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:44.470101    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:15:44.470328 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:44.470153    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:15:44.470328 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:44.470252    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p64q5" podUID="2bb3f329-4286-46e2-8a7d-450769d1aa15"
Jul 02 14:15:44.470489 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:44.470349    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7gvk9" podUID="81c88401-37f6-493c-80ad-c65003c62e57"
Jul 02 14:15:45.470265 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:45.469673    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:15:45.470265 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:45.469820    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpdzr" podUID="ada3d319-cd6a-42ad-b8f9-c04da51705ef"
Jul 02 14:15:46.474066 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:46.471130    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:15:46.474066 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:46.471766    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7gvk9" podUID="81c88401-37f6-493c-80ad-c65003c62e57"
Jul 02 14:15:46.479595 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:46.474781    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:15:46.479595 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:46.475052    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p64q5" podUID="2bb3f329-4286-46e2-8a7d-450769d1aa15"
Jul 02 14:15:47.470115 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:47.469655    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:15:47.470115 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:47.469769    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpdzr" podUID="ada3d319-cd6a-42ad-b8f9-c04da51705ef"
Jul 02 14:15:48.092062 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:48.092023    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ada3d319-cd6a-42ad-b8f9-c04da51705ef-metrics-certs\") pod \"network-metrics-daemon-lpdzr\" (UID: \"ada3d319-cd6a-42ad-b8f9-c04da51705ef\") " pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:15:48.092553 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:48.092213    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 14:15:48.092553 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:48.092293    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ada3d319-cd6a-42ad-b8f9-c04da51705ef-metrics-certs podName:ada3d319-cd6a-42ad-b8f9-c04da51705ef nodeName:}" failed. No retries permitted until 2026-07-02 14:15:56.092269185 +0000 UTC m=+18.043606833 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ada3d319-cd6a-42ad-b8f9-c04da51705ef-metrics-certs") pod "network-metrics-daemon-lpdzr" (UID: "ada3d319-cd6a-42ad-b8f9-c04da51705ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 14:15:48.193300 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:48.193263    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2bb3f329-4286-46e2-8a7d-450769d1aa15-original-pull-secret\") pod \"global-pull-secret-syncer-p64q5\" (UID: \"2bb3f329-4286-46e2-8a7d-450769d1aa15\") " pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:15:48.193495 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:48.193456    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 14:15:48.193570 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:48.193519    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bb3f329-4286-46e2-8a7d-450769d1aa15-original-pull-secret podName:2bb3f329-4286-46e2-8a7d-450769d1aa15 nodeName:}" failed. No retries permitted until 2026-07-02 14:15:56.193505131 +0000 UTC m=+18.144842757 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2bb3f329-4286-46e2-8a7d-450769d1aa15-original-pull-secret") pod "global-pull-secret-syncer-p64q5" (UID: "2bb3f329-4286-46e2-8a7d-450769d1aa15") : object "kube-system"/"original-pull-secret" not registered
Jul 02 14:15:48.294253 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:48.294210    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq248\" (UniqueName: \"kubernetes.io/projected/81c88401-37f6-493c-80ad-c65003c62e57-kube-api-access-bq248\") pod \"network-check-target-7gvk9\" (UID: \"81c88401-37f6-493c-80ad-c65003c62e57\") " pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:15:48.294466 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:48.294445    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 14:15:48.294541 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:48.294476    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 14:15:48.294541 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:48.294489    2582 projected.go:196] Error preparing data for projected volume kube-api-access-bq248 for pod openshift-network-diagnostics/network-check-target-7gvk9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 14:15:48.294665 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:48.294548    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/81c88401-37f6-493c-80ad-c65003c62e57-kube-api-access-bq248 podName:81c88401-37f6-493c-80ad-c65003c62e57 nodeName:}" failed. No retries permitted until 2026-07-02 14:15:56.29452711 +0000 UTC m=+18.245864758 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq248" (UniqueName: "kubernetes.io/projected/81c88401-37f6-493c-80ad-c65003c62e57-kube-api-access-bq248") pod "network-check-target-7gvk9" (UID: "81c88401-37f6-493c-80ad-c65003c62e57") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 14:15:48.472259 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:48.472224    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:15:48.472418 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:48.472395    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p64q5" podUID="2bb3f329-4286-46e2-8a7d-450769d1aa15"
Jul 02 14:15:48.473071 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:48.472299    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:15:48.473071 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:48.472608    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7gvk9" podUID="81c88401-37f6-493c-80ad-c65003c62e57"
Jul 02 14:15:49.469711 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:49.469672    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:15:49.470248 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:49.469840    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpdzr" podUID="ada3d319-cd6a-42ad-b8f9-c04da51705ef"
Jul 02 14:15:50.472501 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:50.471918    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:15:50.472501 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:50.472065    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p64q5" podUID="2bb3f329-4286-46e2-8a7d-450769d1aa15"
Jul 02 14:15:50.472501 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:50.472422    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:15:50.477889 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:50.473485    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7gvk9" podUID="81c88401-37f6-493c-80ad-c65003c62e57"
Jul 02 14:15:51.469612 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:51.469578    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:15:51.469803 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:51.469711    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpdzr" podUID="ada3d319-cd6a-42ad-b8f9-c04da51705ef"
Jul 02 14:15:52.470405 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:52.470364    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:15:52.470405 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:52.470416    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:15:52.471121 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:52.470574    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p64q5" podUID="2bb3f329-4286-46e2-8a7d-450769d1aa15"
Jul 02 14:15:52.471121 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:52.470684    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7gvk9" podUID="81c88401-37f6-493c-80ad-c65003c62e57"
Jul 02 14:15:53.470124 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:53.470093    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:15:53.470306 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:53.470216    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpdzr" podUID="ada3d319-cd6a-42ad-b8f9-c04da51705ef"
Jul 02 14:15:54.469508 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:54.469467    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:15:54.469948 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:54.469596    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p64q5" podUID="2bb3f329-4286-46e2-8a7d-450769d1aa15"
Jul 02 14:15:54.469948 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:54.469659    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:15:54.469948 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:54.469772    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7gvk9" podUID="81c88401-37f6-493c-80ad-c65003c62e57"
Jul 02 14:15:55.470019 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:55.469985    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:15:55.470463 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:55.470105    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpdzr" podUID="ada3d319-cd6a-42ad-b8f9-c04da51705ef"
Jul 02 14:15:56.158068 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:56.158025    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ada3d319-cd6a-42ad-b8f9-c04da51705ef-metrics-certs\") pod \"network-metrics-daemon-lpdzr\" (UID: \"ada3d319-cd6a-42ad-b8f9-c04da51705ef\") " pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:15:56.158256 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:56.158175    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 14:15:56.158256 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:56.158255    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ada3d319-cd6a-42ad-b8f9-c04da51705ef-metrics-certs podName:ada3d319-cd6a-42ad-b8f9-c04da51705ef nodeName:}" failed. No retries permitted until 2026-07-02 14:16:12.15823629 +0000 UTC m=+34.109573931 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ada3d319-cd6a-42ad-b8f9-c04da51705ef-metrics-certs") pod "network-metrics-daemon-lpdzr" (UID: "ada3d319-cd6a-42ad-b8f9-c04da51705ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 14:15:56.258679 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:56.258640    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2bb3f329-4286-46e2-8a7d-450769d1aa15-original-pull-secret\") pod \"global-pull-secret-syncer-p64q5\" (UID: \"2bb3f329-4286-46e2-8a7d-450769d1aa15\") " pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:15:56.258859 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:56.258789    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 14:15:56.258859 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:56.258857    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bb3f329-4286-46e2-8a7d-450769d1aa15-original-pull-secret podName:2bb3f329-4286-46e2-8a7d-450769d1aa15 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:12.258840117 +0000 UTC m=+34.210177740 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2bb3f329-4286-46e2-8a7d-450769d1aa15-original-pull-secret") pod "global-pull-secret-syncer-p64q5" (UID: "2bb3f329-4286-46e2-8a7d-450769d1aa15") : object "kube-system"/"original-pull-secret" not registered
Jul 02 14:15:56.359598 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:56.359564    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq248\" (UniqueName: \"kubernetes.io/projected/81c88401-37f6-493c-80ad-c65003c62e57-kube-api-access-bq248\") pod \"network-check-target-7gvk9\" (UID: \"81c88401-37f6-493c-80ad-c65003c62e57\") " pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:15:56.359779 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:56.359709    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 14:15:56.359779 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:56.359730    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 14:15:56.359779 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:56.359745    2582 projected.go:196] Error preparing data for projected volume kube-api-access-bq248 for pod openshift-network-diagnostics/network-check-target-7gvk9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 14:15:56.359928 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:56.359805    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/81c88401-37f6-493c-80ad-c65003c62e57-kube-api-access-bq248 podName:81c88401-37f6-493c-80ad-c65003c62e57 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:12.359787942 +0000 UTC m=+34.311125588 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq248" (UniqueName: "kubernetes.io/projected/81c88401-37f6-493c-80ad-c65003c62e57-kube-api-access-bq248") pod "network-check-target-7gvk9" (UID: "81c88401-37f6-493c-80ad-c65003c62e57") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 14:15:56.470361 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:56.469917    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:15:56.470361 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:56.470068    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p64q5" podUID="2bb3f329-4286-46e2-8a7d-450769d1aa15"
Jul 02 14:15:56.470361 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:56.470112    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:15:56.470361 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:56.470216    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7gvk9" podUID="81c88401-37f6-493c-80ad-c65003c62e57"
Jul 02 14:15:57.470110 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:57.470075    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:15:57.470292 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:57.470189    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpdzr" podUID="ada3d319-cd6a-42ad-b8f9-c04da51705ef"
Jul 02 14:15:58.474934 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:58.474906    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:15:58.475410 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:58.475027    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7gvk9" podUID="81c88401-37f6-493c-80ad-c65003c62e57"
Jul 02 14:15:58.475410 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:58.474908    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:15:58.475410 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:58.475157    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p64q5" podUID="2bb3f329-4286-46e2-8a7d-450769d1aa15"
Jul 02 14:15:59.470062 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:15:59.470027    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:15:59.470249 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:15:59.470147    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpdzr" podUID="ada3d319-cd6a-42ad-b8f9-c04da51705ef"
Jul 02 14:16:00.470140 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:00.470100    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:16:00.470740 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:00.470240    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p64q5" podUID="2bb3f329-4286-46e2-8a7d-450769d1aa15"
Jul 02 14:16:00.470740 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:00.470295    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:16:00.470740 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:00.470386    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7gvk9" podUID="81c88401-37f6-493c-80ad-c65003c62e57"
Jul 02 14:16:01.469601 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:01.469571    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:16:01.469744 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:01.469707    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpdzr" podUID="ada3d319-cd6a-42ad-b8f9-c04da51705ef"
Jul 02 14:16:02.472559 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:02.472263    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:16:02.473160 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:02.472265    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:16:02.473160 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:02.472640    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7gvk9" podUID="81c88401-37f6-493c-80ad-c65003c62e57"
Jul 02 14:16:02.473160 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:02.472706    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p64q5" podUID="2bb3f329-4286-46e2-8a7d-450769d1aa15"
Jul 02 14:16:02.589701 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:02.589670    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-r6m5m" event={"ID":"fd59e931-7137-4fa6-8b4a-3e759e2eebf1","Type":"ContainerStarted","Data":"e02499675234328b6d8f045a2951c7e5a69c27b7acb5e7450eff5dfdac420f60"}
Jul 02 14:16:02.591013 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:02.590987    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q6zgx" event={"ID":"9b8d6940-2ec0-442f-ae0f-0e7221348432","Type":"ContainerStarted","Data":"df82bea078beaca0a91a9c752eae00459da97b509f5b3dcd37fdbf8ed0de51bc"}
Jul 02 14:16:02.592267 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:02.592247    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4vgf6" event={"ID":"d68b1d02-d681-429f-913b-8eedb0835382","Type":"ContainerStarted","Data":"798dd184653f83c9f3d5dcea283c3b3dd70c2a92bf8140239498501eccff3727"}
Jul 02 14:16:02.593499 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:02.593475    2582 generic.go:356] "Generic (PLEG): container finished" podID="2ff40ff7-e770-4d86-b327-b437a9cfa15a" containerID="7d402cedc577c641fbbbb659af258a569c62ff499d40109c878907adc50fefcb" exitCode=0
Jul 02 14:16:02.593591 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:02.593555    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p96q8" event={"ID":"2ff40ff7-e770-4d86-b327-b437a9cfa15a","Type":"ContainerDied","Data":"7d402cedc577c641fbbbb659af258a569c62ff499d40109c878907adc50fefcb"}
Jul 02 14:16:02.595266 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:02.595241    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-f8pbz" event={"ID":"76237012-8c88-464a-a19a-dfc41536ea5c","Type":"ContainerStarted","Data":"aea18b8524c7514e224405010981b233338b6a5e946625e6d940e27acdb4c5e8"}
Jul 02 14:16:02.596694 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:02.596673    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-r2rq9" event={"ID":"929e19e8-c3ec-4f7a-810f-cd805cf058f6","Type":"ContainerStarted","Data":"416586deefacbd59a0ebdc5f12d6b78e3c112070147c1588ce78b542ad43b16a"}
Jul 02 14:16:02.599700 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:02.599683    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnwb5_ca909057-dd63-451d-9c05-b671ca5ae671/ovn-acl-logging/0.log"
Jul 02 14:16:02.600375 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:02.600355    2582 generic.go:356] "Generic (PLEG): container finished" podID="ca909057-dd63-451d-9c05-b671ca5ae671" containerID="8f0b38c8b16f983dd0565abf36853f296e6c25860a44cd5f81ca9ad9df4f9e29" exitCode=1
Jul 02 14:16:02.600457 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:02.600415    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5" event={"ID":"ca909057-dd63-451d-9c05-b671ca5ae671","Type":"ContainerStarted","Data":"3a2fa653bef51b8e3a14e224f85a506db8c131d57142e8890bb7e21a9e5e9dc8"}
Jul 02 14:16:02.600498 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:02.600467    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5" event={"ID":"ca909057-dd63-451d-9c05-b671ca5ae671","Type":"ContainerStarted","Data":"1f8281589eb27b904271d1e7f7618203253048ccfb494f3164a32ef38cfcde99"}
Jul 02 14:16:02.600541 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:02.600512    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5" event={"ID":"ca909057-dd63-451d-9c05-b671ca5ae671","Type":"ContainerStarted","Data":"631a3412680758026fe69c4e39c751d48402ae69f916f1a06b3e9013735ef9c3"}
Jul 02 14:16:02.600589 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:02.600541    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5" event={"ID":"ca909057-dd63-451d-9c05-b671ca5ae671","Type":"ContainerStarted","Data":"f77daae965fae09853e031bad275c5b99a743cfa9b2962b3d00ee9187abfab51"}
Jul 02 14:16:02.600589 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:02.600571    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5" event={"ID":"ca909057-dd63-451d-9c05-b671ca5ae671","Type":"ContainerDied","Data":"8f0b38c8b16f983dd0565abf36853f296e6c25860a44cd5f81ca9ad9df4f9e29"}
Jul 02 14:16:02.600669 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:02.600603    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5" event={"ID":"ca909057-dd63-451d-9c05-b671ca5ae671","Type":"ContainerStarted","Data":"ac794470a2405b34520a541d67040834b9fb8e9ede86d62c5ffafe2e321220fa"}
Jul 02 14:16:02.601804 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:02.601786    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rtn2p" event={"ID":"4cb188f8-101f-4d6f-b035-90cf16106a5e","Type":"ContainerStarted","Data":"29d0bfc8c74d3501332832f480cc4c14da165dbbb370e81f5577f7ce99f019d7"}
Jul 02 14:16:02.606874 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:02.606839    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-160.ec2.internal" podStartSLOduration=22.606827954 podStartE2EDuration="22.606827954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:15:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:15:43.550336026 +0000 UTC m=+5.501673680" watchObservedRunningTime="2026-07-02 14:16:02.606827954 +0000 UTC m=+24.558165600"
Jul 02 14:16:02.607063 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:02.607043    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-r6m5m" podStartSLOduration=4.088098299 podStartE2EDuration="23.607037625s" totalImagesPullingTime="19.518939326s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:02.606684465 +0000 UTC m=+24.558022111" watchObservedRunningTime="2026-07-02 14:16:02.607037625 +0000 UTC m=+24.558375270"
Jul 02 14:16:02.650834 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:02.650788    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rtn2p" podStartSLOduration=4.064408411 podStartE2EDuration="23.650773335s" totalImagesPullingTime="19.586364924s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:02.650492495 +0000 UTC m=+24.601830141" watchObservedRunningTime="2026-07-02 14:16:02.650773335 +0000 UTC m=+24.602110980"
Jul 02 14:16:02.668016 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:02.667976    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-q6zgx" podStartSLOduration=4.050606697 podStartE2EDuration="23.667942704s" totalImagesPullingTime="19.617336007s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:02.667828992 +0000 UTC m=+24.619166639" watchObservedRunningTime="2026-07-02 14:16:02.667942704 +0000 UTC m=+24.619280350"
Jul 02 14:16:02.700767 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:02.700726    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-f8pbz" podStartSLOduration=4.083947258 podStartE2EDuration="23.700711985s" totalImagesPullingTime="19.616764727s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:02.700679669 +0000 UTC m=+24.652017315" watchObservedRunningTime="2026-07-02 14:16:02.700711985 +0000 UTC m=+24.652049630"
Jul 02 14:16:02.700869 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:02.700790    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-r2rq9" podStartSLOduration=4.078361152 podStartE2EDuration="23.700786878s" totalImagesPullingTime="19.622425726s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:02.684802044 +0000 UTC m=+24.636139692" watchObservedRunningTime="2026-07-02 14:16:02.700786878 +0000 UTC m=+24.652124523"
Jul 02 14:16:03.469782 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:03.469753    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:16:03.469945 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:03.469888    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpdzr" podUID="ada3d319-cd6a-42ad-b8f9-c04da51705ef"
Jul 02 14:16:03.606612 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:03.606560    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-v5829" event={"ID":"b96a56ee-b8ff-41b1-8470-13914745bc35","Type":"ContainerStarted","Data":"003488dd77cba5f62c88071ee42ab529f169a4bdeca688163f3d0cda10bd7d64"}
Jul 02 14:16:04.031684 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:04.031653    2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 02 14:16:04.470479 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:04.469977    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:16:04.470479 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:04.470080    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7gvk9" podUID="81c88401-37f6-493c-80ad-c65003c62e57"
Jul 02 14:16:04.470479 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:04.469977    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:16:04.470479 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:04.470435    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p64q5" podUID="2bb3f329-4286-46e2-8a7d-450769d1aa15"
Jul 02 14:16:04.485832 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:04.485731    2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-02T14:16:04.031683848Z","UUID":"d4a1e176-2dd3-4581-bca1-515e83708adb","Handler":null,"Name":"","Endpoint":""}
Jul 02 14:16:04.487353 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:04.487329    2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 02 14:16:04.487468 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:04.487367    2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 02 14:16:04.611787 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:04.611639    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4vgf6" event={"ID":"d68b1d02-d681-429f-913b-8eedb0835382","Type":"ContainerStarted","Data":"aa1ad8fcb6a4346377289f9b1732f37cbf420d9e0dde1131710a559d01ee130b"}
Jul 02 14:16:04.615190 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:04.615167    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnwb5_ca909057-dd63-451d-9c05-b671ca5ae671/ovn-acl-logging/0.log"
Jul 02 14:16:04.615688 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:04.615658    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5" event={"ID":"ca909057-dd63-451d-9c05-b671ca5ae671","Type":"ContainerStarted","Data":"a79f84472e21b490381d6db9e8de3cd217cf8c8b3bf98f5f84a4182bf09cb02d"}
Jul 02 14:16:05.265136 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:05.264926    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-r6m5m"
Jul 02 14:16:05.265703 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:05.265665    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-r6m5m"
Jul 02 14:16:05.284468 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:05.284421    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-v5829" podStartSLOduration=6.652060864 podStartE2EDuration="26.284405724s" totalImagesPullingTime="19.63234486s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:03.626796845 +0000 UTC m=+25.578134491" watchObservedRunningTime="2026-07-02 14:16:05.284405724 +0000 UTC m=+27.235743373"
Jul 02 14:16:05.469848 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:05.469804    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:16:05.470029 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:05.469929    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpdzr" podUID="ada3d319-cd6a-42ad-b8f9-c04da51705ef"
Jul 02 14:16:05.620040 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:05.619945    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4vgf6" event={"ID":"d68b1d02-d681-429f-913b-8eedb0835382","Type":"ContainerStarted","Data":"eabd42652a39d3e62a2a593ff6e4e29142dd44e69287cdcc6cde85b728320b33"}
Jul 02 14:16:05.620464 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:05.620158    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-r6m5m"
Jul 02 14:16:05.620696 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:05.620677    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-r6m5m"
Jul 02 14:16:05.639645 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:05.639599    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4vgf6" podStartSLOduration=4.014242323 podStartE2EDuration="26.639583926s" totalImagesPullingTime="22.625341603s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:15:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:05.63897675 +0000 UTC m=+27.590314395" watchObservedRunningTime="2026-07-02 14:16:05.639583926 +0000 UTC m=+27.590921571"
Jul 02 14:16:06.473188 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:06.473162    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:16:06.473354 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:06.473164    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:16:06.473354 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:06.473266    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7gvk9" podUID="81c88401-37f6-493c-80ad-c65003c62e57"
Jul 02 14:16:06.473354 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:06.473343    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p64q5" podUID="2bb3f329-4286-46e2-8a7d-450769d1aa15"
Jul 02 14:16:07.469486 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:07.469308    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:16:07.470178 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:07.469574    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpdzr" podUID="ada3d319-cd6a-42ad-b8f9-c04da51705ef"
Jul 02 14:16:07.626004 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:07.625947    2582 generic.go:356] "Generic (PLEG): container finished" podID="2ff40ff7-e770-4d86-b327-b437a9cfa15a" containerID="fa642ea02fc3e34ad611f71e30eeb8be75e6f8f2f80a1485d4ba5156bc73ea3e" exitCode=0
Jul 02 14:16:07.626004 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:07.625993    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p96q8" event={"ID":"2ff40ff7-e770-4d86-b327-b437a9cfa15a","Type":"ContainerDied","Data":"fa642ea02fc3e34ad611f71e30eeb8be75e6f8f2f80a1485d4ba5156bc73ea3e"}
Jul 02 14:16:07.629586 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:07.629569    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnwb5_ca909057-dd63-451d-9c05-b671ca5ae671/ovn-acl-logging/0.log"
Jul 02 14:16:07.630010 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:07.629989    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5" event={"ID":"ca909057-dd63-451d-9c05-b671ca5ae671","Type":"ContainerStarted","Data":"432e083bb07227cd6c8f11734bf92870d814f18ba86afe3c27d9be1af85b7e05"}
Jul 02 14:16:07.630391 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:07.630376    2582 scope.go:117] "RemoveContainer" containerID="8f0b38c8b16f983dd0565abf36853f296e6c25860a44cd5f81ca9ad9df4f9e29"
Jul 02 14:16:08.472837 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:08.472752    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:16:08.472837 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:08.472848    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7gvk9" podUID="81c88401-37f6-493c-80ad-c65003c62e57"
Jul 02 14:16:08.472837 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:08.472874    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:16:08.473470 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:08.472930    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p64q5" podUID="2bb3f329-4286-46e2-8a7d-450769d1aa15"
Jul 02 14:16:08.634176 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:08.634151    2582 generic.go:356] "Generic (PLEG): container finished" podID="2ff40ff7-e770-4d86-b327-b437a9cfa15a" containerID="67b009ba913953d3e6b8e27b2faf228d37b789529b2af4dd7af2d77a7a222aa7" exitCode=0
Jul 02 14:16:08.634278 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:08.634219    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p96q8" event={"ID":"2ff40ff7-e770-4d86-b327-b437a9cfa15a","Type":"ContainerDied","Data":"67b009ba913953d3e6b8e27b2faf228d37b789529b2af4dd7af2d77a7a222aa7"}
Jul 02 14:16:08.638290 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:08.638273    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnwb5_ca909057-dd63-451d-9c05-b671ca5ae671/ovn-acl-logging/0.log"
Jul 02 14:16:08.638677 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:08.638648    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5" event={"ID":"ca909057-dd63-451d-9c05-b671ca5ae671","Type":"ContainerStarted","Data":"f2ac8c376586d172a0e7d251ce36fd3e5d353fadc78f5f643195c4964089d4fd"}
Jul 02 14:16:08.638874 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:08.638859    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:16:08.638948 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:08.638935    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:16:08.639035 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:08.639014    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:16:08.654713 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:08.654695    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:16:08.654798 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:08.654788    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:16:08.689147 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:08.689111    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5" podStartSLOduration=10.01653703 podStartE2EDuration="29.689098579s" totalImagesPullingTime="19.672561549s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:08.687433409 +0000 UTC m=+30.638771057" watchObservedRunningTime="2026-07-02 14:16:08.689098579 +0000 UTC m=+30.640436225"
Jul 02 14:16:08.839313 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:08.839281    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p64q5"]
Jul 02 14:16:08.839471 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:08.839437    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:16:08.839572 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:08.839549    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p64q5" podUID="2bb3f329-4286-46e2-8a7d-450769d1aa15"
Jul 02 14:16:08.842663 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:08.842634    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lpdzr"]
Jul 02 14:16:08.842803 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:08.842762    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:16:08.842891 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:08.842877    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpdzr" podUID="ada3d319-cd6a-42ad-b8f9-c04da51705ef"
Jul 02 14:16:08.843259 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:08.843231    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7gvk9"]
Jul 02 14:16:08.843350 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:08.843316    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:16:08.843404 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:08.843387    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7gvk9" podUID="81c88401-37f6-493c-80ad-c65003c62e57"
Jul 02 14:16:10.475657 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:10.473591    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:16:10.475657 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:10.473998    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpdzr" podUID="ada3d319-cd6a-42ad-b8f9-c04da51705ef"
Jul 02 14:16:10.475657 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:10.474143    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:16:10.475657 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:10.474248    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p64q5" podUID="2bb3f329-4286-46e2-8a7d-450769d1aa15"
Jul 02 14:16:10.475657 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:10.474314    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:16:10.475657 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:10.474440    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7gvk9" podUID="81c88401-37f6-493c-80ad-c65003c62e57"
Jul 02 14:16:10.645262 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:10.645220    2582 generic.go:356] "Generic (PLEG): container finished" podID="2ff40ff7-e770-4d86-b327-b437a9cfa15a" containerID="1b703340ed21f6edc8700d1ae0c72004ebc2005816bbd61fc70d2ca08a055257" exitCode=0
Jul 02 14:16:10.645401 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:10.645296    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p96q8" event={"ID":"2ff40ff7-e770-4d86-b327-b437a9cfa15a","Type":"ContainerDied","Data":"1b703340ed21f6edc8700d1ae0c72004ebc2005816bbd61fc70d2ca08a055257"}
Jul 02 14:16:12.175328 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:12.175290    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ada3d319-cd6a-42ad-b8f9-c04da51705ef-metrics-certs\") pod \"network-metrics-daemon-lpdzr\" (UID: \"ada3d319-cd6a-42ad-b8f9-c04da51705ef\") " pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:16:12.175916 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:12.175463    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 14:16:12.175916 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:12.175540    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ada3d319-cd6a-42ad-b8f9-c04da51705ef-metrics-certs podName:ada3d319-cd6a-42ad-b8f9-c04da51705ef nodeName:}" failed. No retries permitted until 2026-07-02 14:16:44.175519889 +0000 UTC m=+66.126857528 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ada3d319-cd6a-42ad-b8f9-c04da51705ef-metrics-certs") pod "network-metrics-daemon-lpdzr" (UID: "ada3d319-cd6a-42ad-b8f9-c04da51705ef") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 14:16:12.276379 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:12.276342    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2bb3f329-4286-46e2-8a7d-450769d1aa15-original-pull-secret\") pod \"global-pull-secret-syncer-p64q5\" (UID: \"2bb3f329-4286-46e2-8a7d-450769d1aa15\") " pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:16:12.276530 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:12.276488    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 14:16:12.276577 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:12.276569    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bb3f329-4286-46e2-8a7d-450769d1aa15-original-pull-secret podName:2bb3f329-4286-46e2-8a7d-450769d1aa15 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:44.276549162 +0000 UTC m=+66.227886789 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2bb3f329-4286-46e2-8a7d-450769d1aa15-original-pull-secret") pod "global-pull-secret-syncer-p64q5" (UID: "2bb3f329-4286-46e2-8a7d-450769d1aa15") : object "kube-system"/"original-pull-secret" not registered
Jul 02 14:16:12.377461 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:12.377431    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq248\" (UniqueName: \"kubernetes.io/projected/81c88401-37f6-493c-80ad-c65003c62e57-kube-api-access-bq248\") pod \"network-check-target-7gvk9\" (UID: \"81c88401-37f6-493c-80ad-c65003c62e57\") " pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:16:12.377647 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:12.377627    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 14:16:12.377688 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:12.377654    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 14:16:12.377688 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:12.377664    2582 projected.go:196] Error preparing data for projected volume kube-api-access-bq248 for pod openshift-network-diagnostics/network-check-target-7gvk9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 14:16:12.377762 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:12.377730    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/81c88401-37f6-493c-80ad-c65003c62e57-kube-api-access-bq248 podName:81c88401-37f6-493c-80ad-c65003c62e57 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:44.377707947 +0000 UTC m=+66.329045594 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq248" (UniqueName: "kubernetes.io/projected/81c88401-37f6-493c-80ad-c65003c62e57-kube-api-access-bq248") pod "network-check-target-7gvk9" (UID: "81c88401-37f6-493c-80ad-c65003c62e57") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 14:16:12.473141 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:12.473108    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:16:12.473316 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:12.473221    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:16:12.473316 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:12.473234    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:16:12.473316 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:12.473231    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lpdzr" podUID="ada3d319-cd6a-42ad-b8f9-c04da51705ef"
Jul 02 14:16:12.473483 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:12.473359    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p64q5" podUID="2bb3f329-4286-46e2-8a7d-450769d1aa15"
Jul 02 14:16:12.473483 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:12.473441    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7gvk9" podUID="81c88401-37f6-493c-80ad-c65003c62e57"
Jul 02 14:16:14.325772 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.325739    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-160.ec2.internal" event="NodeReady"
Jul 02 14:16:14.326461 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.325897    2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 02 14:16:14.375702 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.375537    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-295qn"]
Jul 02 14:16:14.405874 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.405780    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-548f7fc796-9ts6k"]
Jul 02 14:16:14.406043 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.405904    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-295qn"
Jul 02 14:16:14.409057 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.409020    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4kf2n\""
Jul 02 14:16:14.409171 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.409056    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 02 14:16:14.409171 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.409020    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 02 14:16:14.427662 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.427637    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rlkvc"]
Jul 02 14:16:14.427798 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.427770    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:14.430735 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.430714    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 02 14:16:14.431079 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.430931    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-pcs9h\""
Jul 02 14:16:14.431079 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.430991    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 02 14:16:14.431250 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.431124    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 02 14:16:14.444135 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.444114    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 02 14:16:14.454146 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.454126    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-295qn"]
Jul 02 14:16:14.454258 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.454168    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rlkvc"]
Jul 02 14:16:14.454258 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.454189    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-548f7fc796-9ts6k"]
Jul 02 14:16:14.454258 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.454247    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rlkvc"
Jul 02 14:16:14.457288 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.457267    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 02 14:16:14.457400 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.457269    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 02 14:16:14.457629 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.457602    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 02 14:16:14.458000 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.457985    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-x2r4j\""
Jul 02 14:16:14.473639 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.473610    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:16:14.473639 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.473631    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:16:14.473876 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.473794    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:16:14.476477 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.476458    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 02 14:16:14.476694 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.476678    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-x92rb\""
Jul 02 14:16:14.476817 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.476747    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 02 14:16:14.476817 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.476747    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sn6mc\""
Jul 02 14:16:14.477864 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.477599    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 02 14:16:14.477864 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.477602    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 02 14:16:14.491068 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.491042    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/977cb808-d191-4c87-b63b-8efe4dc2d00f-config-volume\") pod \"dns-default-295qn\" (UID: \"977cb808-d191-4c87-b63b-8efe4dc2d00f\") " pod="openshift-dns/dns-default-295qn"
Jul 02 14:16:14.491187 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.491130    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/977cb808-d191-4c87-b63b-8efe4dc2d00f-metrics-tls\") pod \"dns-default-295qn\" (UID: \"977cb808-d191-4c87-b63b-8efe4dc2d00f\") " pod="openshift-dns/dns-default-295qn"
Jul 02 14:16:14.491187 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.491161    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/977cb808-d191-4c87-b63b-8efe4dc2d00f-tmp-dir\") pod \"dns-default-295qn\" (UID: \"977cb808-d191-4c87-b63b-8efe4dc2d00f\") " pod="openshift-dns/dns-default-295qn"
Jul 02 14:16:14.491290 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.491193    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22zbq\" (UniqueName: \"kubernetes.io/projected/977cb808-d191-4c87-b63b-8efe4dc2d00f-kube-api-access-22zbq\") pod \"dns-default-295qn\" (UID: \"977cb808-d191-4c87-b63b-8efe4dc2d00f\") " pod="openshift-dns/dns-default-295qn"
Jul 02 14:16:14.592113 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.592078    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-registry-tls\") pod \"image-registry-548f7fc796-9ts6k\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") " pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:14.592113 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.592116    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc14ce21-e15d-484d-b934-bb9409c53413-cert\") pod \"ingress-canary-rlkvc\" (UID: \"cc14ce21-e15d-484d-b934-bb9409c53413\") " pod="openshift-ingress-canary/ingress-canary-rlkvc"
Jul 02 14:16:14.592331 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.592136    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5fe4ae17-d881-4f0d-b306-2c3db8d36259-ca-trust-extracted\") pod \"image-registry-548f7fc796-9ts6k\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") " pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:14.592331 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.592208    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/977cb808-d191-4c87-b63b-8efe4dc2d00f-config-volume\") pod \"dns-default-295qn\" (UID: \"977cb808-d191-4c87-b63b-8efe4dc2d00f\") " pod="openshift-dns/dns-default-295qn"
Jul 02 14:16:14.592331 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.592266    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5fe4ae17-d881-4f0d-b306-2c3db8d36259-installation-pull-secrets\") pod \"image-registry-548f7fc796-9ts6k\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") " pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:14.592331 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.592307    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-bound-sa-token\") pod \"image-registry-548f7fc796-9ts6k\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") " pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:14.592486 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.592386    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5fe4ae17-d881-4f0d-b306-2c3db8d36259-image-registry-private-configuration\") pod \"image-registry-548f7fc796-9ts6k\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") " pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:14.592486 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.592430    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/977cb808-d191-4c87-b63b-8efe4dc2d00f-metrics-tls\") pod \"dns-default-295qn\" (UID: \"977cb808-d191-4c87-b63b-8efe4dc2d00f\") " pod="openshift-dns/dns-default-295qn"
Jul 02 14:16:14.592486 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.592464    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/977cb808-d191-4c87-b63b-8efe4dc2d00f-tmp-dir\") pod \"dns-default-295qn\" (UID: \"977cb808-d191-4c87-b63b-8efe4dc2d00f\") " pod="openshift-dns/dns-default-295qn"
Jul 02 14:16:14.592621 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.592485    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs4s5\" (UniqueName: \"kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-kube-api-access-qs4s5\") pod \"image-registry-548f7fc796-9ts6k\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") " pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:14.592621 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:14.592602    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 14:16:14.592621 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.592606    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqrcl\" (UniqueName: \"kubernetes.io/projected/cc14ce21-e15d-484d-b934-bb9409c53413-kube-api-access-jqrcl\") pod \"ingress-canary-rlkvc\" (UID: \"cc14ce21-e15d-484d-b934-bb9409c53413\") " pod="openshift-ingress-canary/ingress-canary-rlkvc"
Jul 02 14:16:14.592749 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:14.592681    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/977cb808-d191-4c87-b63b-8efe4dc2d00f-metrics-tls podName:977cb808-d191-4c87-b63b-8efe4dc2d00f nodeName:}" failed. No retries permitted until 2026-07-02 14:16:15.092661398 +0000 UTC m=+37.043999044 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/977cb808-d191-4c87-b63b-8efe4dc2d00f-metrics-tls") pod "dns-default-295qn" (UID: "977cb808-d191-4c87-b63b-8efe4dc2d00f") : secret "dns-default-metrics-tls" not found
Jul 02 14:16:14.592749 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.592738    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22zbq\" (UniqueName: \"kubernetes.io/projected/977cb808-d191-4c87-b63b-8efe4dc2d00f-kube-api-access-22zbq\") pod \"dns-default-295qn\" (UID: \"977cb808-d191-4c87-b63b-8efe4dc2d00f\") " pod="openshift-dns/dns-default-295qn"
Jul 02 14:16:14.592849 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.592767    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/977cb808-d191-4c87-b63b-8efe4dc2d00f-config-volume\") pod \"dns-default-295qn\" (UID: \"977cb808-d191-4c87-b63b-8efe4dc2d00f\") " pod="openshift-dns/dns-default-295qn"
Jul 02 14:16:14.592849 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.592806    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5fe4ae17-d881-4f0d-b306-2c3db8d36259-registry-certificates\") pod \"image-registry-548f7fc796-9ts6k\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") " pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:14.592849 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.592837    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5fe4ae17-d881-4f0d-b306-2c3db8d36259-trusted-ca\") pod \"image-registry-548f7fc796-9ts6k\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") " pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:14.593050 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.592807    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/977cb808-d191-4c87-b63b-8efe4dc2d00f-tmp-dir\") pod \"dns-default-295qn\" (UID: \"977cb808-d191-4c87-b63b-8efe4dc2d00f\") " pod="openshift-dns/dns-default-295qn"
Jul 02 14:16:14.603912 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.603892    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-22zbq\" (UniqueName: \"kubernetes.io/projected/977cb808-d191-4c87-b63b-8efe4dc2d00f-kube-api-access-22zbq\") pod \"dns-default-295qn\" (UID: \"977cb808-d191-4c87-b63b-8efe4dc2d00f\") " pod="openshift-dns/dns-default-295qn"
Jul 02 14:16:14.693883 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.693797    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qs4s5\" (UniqueName: \"kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-kube-api-access-qs4s5\") pod \"image-registry-548f7fc796-9ts6k\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") " pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:14.693883 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.693850    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqrcl\" (UniqueName: \"kubernetes.io/projected/cc14ce21-e15d-484d-b934-bb9409c53413-kube-api-access-jqrcl\") pod \"ingress-canary-rlkvc\" (UID: \"cc14ce21-e15d-484d-b934-bb9409c53413\") " pod="openshift-ingress-canary/ingress-canary-rlkvc"
Jul 02 14:16:14.694131 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.693901    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5fe4ae17-d881-4f0d-b306-2c3db8d36259-registry-certificates\") pod \"image-registry-548f7fc796-9ts6k\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") " pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:14.694131 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.693927    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5fe4ae17-d881-4f0d-b306-2c3db8d36259-trusted-ca\") pod \"image-registry-548f7fc796-9ts6k\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") " pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:14.694131 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.693977    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-registry-tls\") pod \"image-registry-548f7fc796-9ts6k\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") " pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:14.694131 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.694004    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc14ce21-e15d-484d-b934-bb9409c53413-cert\") pod \"ingress-canary-rlkvc\" (UID: \"cc14ce21-e15d-484d-b934-bb9409c53413\") " pod="openshift-ingress-canary/ingress-canary-rlkvc"
Jul 02 14:16:14.694131 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.694033    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5fe4ae17-d881-4f0d-b306-2c3db8d36259-ca-trust-extracted\") pod \"image-registry-548f7fc796-9ts6k\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") " pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:14.694131 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.694073    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5fe4ae17-d881-4f0d-b306-2c3db8d36259-installation-pull-secrets\") pod \"image-registry-548f7fc796-9ts6k\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") " pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:14.694131 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.694103    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-bound-sa-token\") pod \"image-registry-548f7fc796-9ts6k\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") " pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:14.694471 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.694138    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5fe4ae17-d881-4f0d-b306-2c3db8d36259-image-registry-private-configuration\") pod \"image-registry-548f7fc796-9ts6k\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") " pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:14.694471 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:14.694282    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 14:16:14.694471 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:14.694296    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-548f7fc796-9ts6k: secret "image-registry-tls" not found
Jul 02 14:16:14.694471 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:14.694356    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-registry-tls podName:5fe4ae17-d881-4f0d-b306-2c3db8d36259 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:15.194339042 +0000 UTC m=+37.145676673 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-registry-tls") pod "image-registry-548f7fc796-9ts6k" (UID: "5fe4ae17-d881-4f0d-b306-2c3db8d36259") : secret "image-registry-tls" not found
Jul 02 14:16:14.695552 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.695072    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5fe4ae17-d881-4f0d-b306-2c3db8d36259-registry-certificates\") pod \"image-registry-548f7fc796-9ts6k\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") " pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:14.695552 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.695378    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5fe4ae17-d881-4f0d-b306-2c3db8d36259-ca-trust-extracted\") pod \"image-registry-548f7fc796-9ts6k\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") " pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:14.695552 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.695514    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5fe4ae17-d881-4f0d-b306-2c3db8d36259-trusted-ca\") pod \"image-registry-548f7fc796-9ts6k\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") " pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:14.697580 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.697560    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5fe4ae17-d881-4f0d-b306-2c3db8d36259-image-registry-private-configuration\") pod \"image-registry-548f7fc796-9ts6k\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") " pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:14.697700 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.697601    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5fe4ae17-d881-4f0d-b306-2c3db8d36259-installation-pull-secrets\") pod \"image-registry-548f7fc796-9ts6k\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") " pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:14.697755 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.697729    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc14ce21-e15d-484d-b934-bb9409c53413-cert\") pod \"ingress-canary-rlkvc\" (UID: \"cc14ce21-e15d-484d-b934-bb9409c53413\") " pod="openshift-ingress-canary/ingress-canary-rlkvc"
Jul 02 14:16:14.706200 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.706181    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs4s5\" (UniqueName: \"kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-kube-api-access-qs4s5\") pod \"image-registry-548f7fc796-9ts6k\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") " pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:14.706626 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.706604    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-bound-sa-token\") pod \"image-registry-548f7fc796-9ts6k\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") " pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:14.706706 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.706697    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqrcl\" (UniqueName: \"kubernetes.io/projected/cc14ce21-e15d-484d-b934-bb9409c53413-kube-api-access-jqrcl\") pod \"ingress-canary-rlkvc\" (UID: \"cc14ce21-e15d-484d-b934-bb9409c53413\") " pod="openshift-ingress-canary/ingress-canary-rlkvc"
Jul 02 14:16:14.765050 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:14.765014    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rlkvc"
Jul 02 14:16:15.096860 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:15.096828    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/977cb808-d191-4c87-b63b-8efe4dc2d00f-metrics-tls\") pod \"dns-default-295qn\" (UID: \"977cb808-d191-4c87-b63b-8efe4dc2d00f\") " pod="openshift-dns/dns-default-295qn"
Jul 02 14:16:15.097024 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:15.096987    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 14:16:15.097076 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:15.097071    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/977cb808-d191-4c87-b63b-8efe4dc2d00f-metrics-tls podName:977cb808-d191-4c87-b63b-8efe4dc2d00f nodeName:}" failed. No retries permitted until 2026-07-02 14:16:16.097055673 +0000 UTC m=+38.048393297 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/977cb808-d191-4c87-b63b-8efe4dc2d00f-metrics-tls") pod "dns-default-295qn" (UID: "977cb808-d191-4c87-b63b-8efe4dc2d00f") : secret "dns-default-metrics-tls" not found
Jul 02 14:16:15.101891 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:15.101867    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rlkvc"]
Jul 02 14:16:15.171509 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:16:15.171481    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc14ce21_e15d_484d_b934_bb9409c53413.slice/crio-2a4dff81c841968e8ffaf8f57903a88c5e18b3121ca4ee9beb1da750ddedb80d WatchSource:0}: Error finding container 2a4dff81c841968e8ffaf8f57903a88c5e18b3121ca4ee9beb1da750ddedb80d: Status 404 returned error can't find the container with id 2a4dff81c841968e8ffaf8f57903a88c5e18b3121ca4ee9beb1da750ddedb80d
Jul 02 14:16:15.197773 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:15.197753    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-registry-tls\") pod \"image-registry-548f7fc796-9ts6k\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") " pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:15.197872 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:15.197864    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 14:16:15.197907 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:15.197875    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-548f7fc796-9ts6k: secret "image-registry-tls" not found
Jul 02 14:16:15.197940 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:15.197919    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-registry-tls podName:5fe4ae17-d881-4f0d-b306-2c3db8d36259 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:16.197905071 +0000 UTC m=+38.149242695 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-registry-tls") pod "image-registry-548f7fc796-9ts6k" (UID: "5fe4ae17-d881-4f0d-b306-2c3db8d36259") : secret "image-registry-tls" not found
Jul 02 14:16:15.660080 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:15.659977    2582 generic.go:356] "Generic (PLEG): container finished" podID="2ff40ff7-e770-4d86-b327-b437a9cfa15a" containerID="6c150021b161e1dd5c97913a089b5286739137fd776b9f62dd178e6d1a329725" exitCode=0
Jul 02 14:16:15.660080 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:15.660047    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p96q8" event={"ID":"2ff40ff7-e770-4d86-b327-b437a9cfa15a","Type":"ContainerDied","Data":"6c150021b161e1dd5c97913a089b5286739137fd776b9f62dd178e6d1a329725"}
Jul 02 14:16:15.661304 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:15.661280    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rlkvc" event={"ID":"cc14ce21-e15d-484d-b934-bb9409c53413","Type":"ContainerStarted","Data":"2a4dff81c841968e8ffaf8f57903a88c5e18b3121ca4ee9beb1da750ddedb80d"}
Jul 02 14:16:16.021564 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:16.021534    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-786f9fbc5-zmsd5"]
Jul 02 14:16:16.034342 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:16.034317    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-zmsd5"
Jul 02 14:16:16.035435 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:16.035402    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-786f9fbc5-zmsd5"]
Jul 02 14:16:16.037644 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:16.037622    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 02 14:16:16.037644 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:16.037638    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-v9gv4\""
Jul 02 14:16:16.037797 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:16.037639    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 02 14:16:16.104908 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:16.104877    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rw89\" (UniqueName: \"kubernetes.io/projected/5b2261d4-525c-425a-abd6-ee77b01d6392-kube-api-access-5rw89\") pod \"migrator-786f9fbc5-zmsd5\" (UID: \"5b2261d4-525c-425a-abd6-ee77b01d6392\") " pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-zmsd5"
Jul 02 14:16:16.105105 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:16.105006    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/977cb808-d191-4c87-b63b-8efe4dc2d00f-metrics-tls\") pod \"dns-default-295qn\" (UID: \"977cb808-d191-4c87-b63b-8efe4dc2d00f\") " pod="openshift-dns/dns-default-295qn"
Jul 02 14:16:16.105171 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:16.105125    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 14:16:16.105222 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:16.105191    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/977cb808-d191-4c87-b63b-8efe4dc2d00f-metrics-tls podName:977cb808-d191-4c87-b63b-8efe4dc2d00f nodeName:}" failed. No retries permitted until 2026-07-02 14:16:18.105172219 +0000 UTC m=+40.056509847 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/977cb808-d191-4c87-b63b-8efe4dc2d00f-metrics-tls") pod "dns-default-295qn" (UID: "977cb808-d191-4c87-b63b-8efe4dc2d00f") : secret "dns-default-metrics-tls" not found
Jul 02 14:16:16.206058 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:16.206016    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-registry-tls\") pod \"image-registry-548f7fc796-9ts6k\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") " pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:16.206058 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:16.206058    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rw89\" (UniqueName: \"kubernetes.io/projected/5b2261d4-525c-425a-abd6-ee77b01d6392-kube-api-access-5rw89\") pod \"migrator-786f9fbc5-zmsd5\" (UID: \"5b2261d4-525c-425a-abd6-ee77b01d6392\") " pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-zmsd5"
Jul 02 14:16:16.206302 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:16.206170    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 14:16:16.206302 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:16.206189    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-548f7fc796-9ts6k: secret "image-registry-tls" not found
Jul 02 14:16:16.206302 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:16.206245    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-registry-tls podName:5fe4ae17-d881-4f0d-b306-2c3db8d36259 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:18.206228746 +0000 UTC m=+40.157566373 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-registry-tls") pod "image-registry-548f7fc796-9ts6k" (UID: "5fe4ae17-d881-4f0d-b306-2c3db8d36259") : secret "image-registry-tls" not found
Jul 02 14:16:16.219031 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:16.219008    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rw89\" (UniqueName: \"kubernetes.io/projected/5b2261d4-525c-425a-abd6-ee77b01d6392-kube-api-access-5rw89\") pod \"migrator-786f9fbc5-zmsd5\" (UID: \"5b2261d4-525c-425a-abd6-ee77b01d6392\") " pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-zmsd5"
Jul 02 14:16:16.343766 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:16.343678    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-zmsd5"
Jul 02 14:16:16.507634 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:16.507604    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-786f9fbc5-zmsd5"]
Jul 02 14:16:16.510651 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:16:16.510620    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b2261d4_525c_425a_abd6_ee77b01d6392.slice/crio-8af25d6ec788ce20ff0bd29d7bf91a1cf5eab7a08aa52c6ee80b5fe2b604fd4c WatchSource:0}: Error finding container 8af25d6ec788ce20ff0bd29d7bf91a1cf5eab7a08aa52c6ee80b5fe2b604fd4c: Status 404 returned error can't find the container with id 8af25d6ec788ce20ff0bd29d7bf91a1cf5eab7a08aa52c6ee80b5fe2b604fd4c
Jul 02 14:16:16.667557 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:16.667468    2582 generic.go:356] "Generic (PLEG): container finished" podID="2ff40ff7-e770-4d86-b327-b437a9cfa15a" containerID="017c8326da96955490015b907edb6041d02c26f28e56223fd26e5904f9569ac3" exitCode=0
Jul 02 14:16:16.667990 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:16.667549    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p96q8" event={"ID":"2ff40ff7-e770-4d86-b327-b437a9cfa15a","Type":"ContainerDied","Data":"017c8326da96955490015b907edb6041d02c26f28e56223fd26e5904f9569ac3"}
Jul 02 14:16:16.668795 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:16.668768    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-zmsd5" event={"ID":"5b2261d4-525c-425a-abd6-ee77b01d6392","Type":"ContainerStarted","Data":"8af25d6ec788ce20ff0bd29d7bf91a1cf5eab7a08aa52c6ee80b5fe2b604fd4c"}
Jul 02 14:16:16.964677 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:16.964437    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-srnvs"]
Jul 02 14:16:16.981996 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:16.981939    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-srnvs"
Jul 02 14:16:16.988336 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:16.988308    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 02 14:16:16.988573 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:16.988555    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 02 14:16:16.988678 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:16.988589    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 02 14:16:16.988678 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:16.988636    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 02 14:16:16.988881 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:16.988850    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gzldj\""
Jul 02 14:16:16.998115 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:16.998087    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-srnvs"]
Jul 02 14:16:17.112217 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:17.112184    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/196e2ce2-d807-46d8-81e5-ff0b627c3b08-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-srnvs\" (UID: \"196e2ce2-d807-46d8-81e5-ff0b627c3b08\") " pod="openshift-insights/insights-runtime-extractor-srnvs"
Jul 02 14:16:17.112399 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:17.112228    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4g68w\" (UniqueName: \"kubernetes.io/projected/196e2ce2-d807-46d8-81e5-ff0b627c3b08-kube-api-access-4g68w\") pod \"insights-runtime-extractor-srnvs\" (UID: \"196e2ce2-d807-46d8-81e5-ff0b627c3b08\") " pod="openshift-insights/insights-runtime-extractor-srnvs"
Jul 02 14:16:17.112399 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:17.112279    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/196e2ce2-d807-46d8-81e5-ff0b627c3b08-crio-socket\") pod \"insights-runtime-extractor-srnvs\" (UID: \"196e2ce2-d807-46d8-81e5-ff0b627c3b08\") " pod="openshift-insights/insights-runtime-extractor-srnvs"
Jul 02 14:16:17.112399 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:17.112324    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/196e2ce2-d807-46d8-81e5-ff0b627c3b08-data-volume\") pod \"insights-runtime-extractor-srnvs\" (UID: \"196e2ce2-d807-46d8-81e5-ff0b627c3b08\") " pod="openshift-insights/insights-runtime-extractor-srnvs"
Jul 02 14:16:17.112399 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:17.112352    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/196e2ce2-d807-46d8-81e5-ff0b627c3b08-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-srnvs\" (UID: \"196e2ce2-d807-46d8-81e5-ff0b627c3b08\") " pod="openshift-insights/insights-runtime-extractor-srnvs"
Jul 02 14:16:17.213495 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:17.213405    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/196e2ce2-d807-46d8-81e5-ff0b627c3b08-crio-socket\") pod \"insights-runtime-extractor-srnvs\" (UID: \"196e2ce2-d807-46d8-81e5-ff0b627c3b08\") " pod="openshift-insights/insights-runtime-extractor-srnvs"
Jul 02 14:16:17.213495 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:17.213489    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/196e2ce2-d807-46d8-81e5-ff0b627c3b08-data-volume\") pod \"insights-runtime-extractor-srnvs\" (UID: \"196e2ce2-d807-46d8-81e5-ff0b627c3b08\") " pod="openshift-insights/insights-runtime-extractor-srnvs"
Jul 02 14:16:17.213729 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:17.213519    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/196e2ce2-d807-46d8-81e5-ff0b627c3b08-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-srnvs\" (UID: \"196e2ce2-d807-46d8-81e5-ff0b627c3b08\") " pod="openshift-insights/insights-runtime-extractor-srnvs"
Jul 02 14:16:17.213729 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:17.213566    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/196e2ce2-d807-46d8-81e5-ff0b627c3b08-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-srnvs\" (UID: \"196e2ce2-d807-46d8-81e5-ff0b627c3b08\") " pod="openshift-insights/insights-runtime-extractor-srnvs"
Jul 02 14:16:17.213729 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:17.213614    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4g68w\" (UniqueName: \"kubernetes.io/projected/196e2ce2-d807-46d8-81e5-ff0b627c3b08-kube-api-access-4g68w\") pod \"insights-runtime-extractor-srnvs\" (UID: \"196e2ce2-d807-46d8-81e5-ff0b627c3b08\") " pod="openshift-insights/insights-runtime-extractor-srnvs"
Jul 02 14:16:17.213729 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:17.213634    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/196e2ce2-d807-46d8-81e5-ff0b627c3b08-crio-socket\") pod \"insights-runtime-extractor-srnvs\" (UID: \"196e2ce2-d807-46d8-81e5-ff0b627c3b08\") " pod="openshift-insights/insights-runtime-extractor-srnvs"
Jul 02 14:16:17.213910 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:17.213755    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 14:16:17.213910 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:17.213815    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/196e2ce2-d807-46d8-81e5-ff0b627c3b08-insights-runtime-extractor-tls podName:196e2ce2-d807-46d8-81e5-ff0b627c3b08 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:17.713796242 +0000 UTC m=+39.665133867 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/196e2ce2-d807-46d8-81e5-ff0b627c3b08-insights-runtime-extractor-tls") pod "insights-runtime-extractor-srnvs" (UID: "196e2ce2-d807-46d8-81e5-ff0b627c3b08") : secret "insights-runtime-extractor-tls" not found
Jul 02 14:16:17.214133 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:17.214100    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 14:16:17.214263 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:17.214169    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/196e2ce2-d807-46d8-81e5-ff0b627c3b08-kube-rbac-proxy-cm podName:196e2ce2-d807-46d8-81e5-ff0b627c3b08 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:17.714150639 +0000 UTC m=+39.665488263 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/196e2ce2-d807-46d8-81e5-ff0b627c3b08-kube-rbac-proxy-cm") pod "insights-runtime-extractor-srnvs" (UID: "196e2ce2-d807-46d8-81e5-ff0b627c3b08") : configmap "kube-rbac-proxy" not found
Jul 02 14:16:17.223708 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:17.223673    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/196e2ce2-d807-46d8-81e5-ff0b627c3b08-data-volume\") pod \"insights-runtime-extractor-srnvs\" (UID: \"196e2ce2-d807-46d8-81e5-ff0b627c3b08\") " pod="openshift-insights/insights-runtime-extractor-srnvs"
Jul 02 14:16:17.229483 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:17.229460    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4g68w\" (UniqueName: \"kubernetes.io/projected/196e2ce2-d807-46d8-81e5-ff0b627c3b08-kube-api-access-4g68w\") pod \"insights-runtime-extractor-srnvs\" (UID: \"196e2ce2-d807-46d8-81e5-ff0b627c3b08\") " pod="openshift-insights/insights-runtime-extractor-srnvs"
Jul 02 14:16:17.675781 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:17.675696    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p96q8" event={"ID":"2ff40ff7-e770-4d86-b327-b437a9cfa15a","Type":"ContainerStarted","Data":"2d6cc991b0e3298b749f67dc92b17e06f95941cdac7ab7bab5169a67e1adffc9"}
Jul 02 14:16:17.719528 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:17.719500    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/196e2ce2-d807-46d8-81e5-ff0b627c3b08-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-srnvs\" (UID: \"196e2ce2-d807-46d8-81e5-ff0b627c3b08\") " pod="openshift-insights/insights-runtime-extractor-srnvs"
Jul 02 14:16:17.719682 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:17.719556    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/196e2ce2-d807-46d8-81e5-ff0b627c3b08-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-srnvs\" (UID: \"196e2ce2-d807-46d8-81e5-ff0b627c3b08\") " pod="openshift-insights/insights-runtime-extractor-srnvs"
Jul 02 14:16:17.719682 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:17.719678    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 14:16:17.719781 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:17.719724    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/196e2ce2-d807-46d8-81e5-ff0b627c3b08-kube-rbac-proxy-cm podName:196e2ce2-d807-46d8-81e5-ff0b627c3b08 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:18.719710247 +0000 UTC m=+40.671047872 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/196e2ce2-d807-46d8-81e5-ff0b627c3b08-kube-rbac-proxy-cm") pod "insights-runtime-extractor-srnvs" (UID: "196e2ce2-d807-46d8-81e5-ff0b627c3b08") : configmap "kube-rbac-proxy" not found
Jul 02 14:16:17.719950 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:17.719926    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 14:16:17.720030 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:17.720004    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/196e2ce2-d807-46d8-81e5-ff0b627c3b08-insights-runtime-extractor-tls podName:196e2ce2-d807-46d8-81e5-ff0b627c3b08 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:18.719990533 +0000 UTC m=+40.671328162 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/196e2ce2-d807-46d8-81e5-ff0b627c3b08-insights-runtime-extractor-tls") pod "insights-runtime-extractor-srnvs" (UID: "196e2ce2-d807-46d8-81e5-ff0b627c3b08") : secret "insights-runtime-extractor-tls" not found
Jul 02 14:16:18.123558 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:18.123526    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/977cb808-d191-4c87-b63b-8efe4dc2d00f-metrics-tls\") pod \"dns-default-295qn\" (UID: \"977cb808-d191-4c87-b63b-8efe4dc2d00f\") " pod="openshift-dns/dns-default-295qn"
Jul 02 14:16:18.123720 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:18.123647    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 14:16:18.123720 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:18.123699    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/977cb808-d191-4c87-b63b-8efe4dc2d00f-metrics-tls podName:977cb808-d191-4c87-b63b-8efe4dc2d00f nodeName:}" failed. No retries permitted until 2026-07-02 14:16:22.123687163 +0000 UTC m=+44.075024787 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/977cb808-d191-4c87-b63b-8efe4dc2d00f-metrics-tls") pod "dns-default-295qn" (UID: "977cb808-d191-4c87-b63b-8efe4dc2d00f") : secret "dns-default-metrics-tls" not found
Jul 02 14:16:18.224194 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:18.224160    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-registry-tls\") pod \"image-registry-548f7fc796-9ts6k\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") " pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:18.224361 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:18.224338    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 14:16:18.224436 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:18.224365    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-548f7fc796-9ts6k: secret "image-registry-tls" not found
Jul 02 14:16:18.224488 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:18.224438    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-registry-tls podName:5fe4ae17-d881-4f0d-b306-2c3db8d36259 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:22.224414917 +0000 UTC m=+44.175752558 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-registry-tls") pod "image-registry-548f7fc796-9ts6k" (UID: "5fe4ae17-d881-4f0d-b306-2c3db8d36259") : secret "image-registry-tls" not found
Jul 02 14:16:18.679326 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:18.679243    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rlkvc" event={"ID":"cc14ce21-e15d-484d-b934-bb9409c53413","Type":"ContainerStarted","Data":"83ea5f50da1588ed6ac0e0a1362b21de0ad634e66c836844cfc8ac01189de76e"}
Jul 02 14:16:18.697750 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:18.697702    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-p96q8" podStartSLOduration=9.875546679 podStartE2EDuration="39.697687176s" totalImagesPullingTime="29.822140497s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:15:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:17.712747663 +0000 UTC m=+39.664085310" watchObservedRunningTime="2026-07-02 14:16:18.697687176 +0000 UTC m=+40.649024823"
Jul 02 14:16:18.698428 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:18.698391    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rlkvc" podStartSLOduration=1.563544512 podStartE2EDuration="4.698380577s" totalImagesPullingTime="3.134836065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:16:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:18.69729815 +0000 UTC m=+40.648635796" watchObservedRunningTime="2026-07-02 14:16:18.698380577 +0000 UTC m=+40.649718223"
Jul 02 14:16:18.728897 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:18.728865    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/196e2ce2-d807-46d8-81e5-ff0b627c3b08-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-srnvs\" (UID: \"196e2ce2-d807-46d8-81e5-ff0b627c3b08\") " pod="openshift-insights/insights-runtime-extractor-srnvs"
Jul 02 14:16:18.729034 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:18.728918    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/196e2ce2-d807-46d8-81e5-ff0b627c3b08-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-srnvs\" (UID: \"196e2ce2-d807-46d8-81e5-ff0b627c3b08\") " pod="openshift-insights/insights-runtime-extractor-srnvs"
Jul 02 14:16:18.729079 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:18.729040    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 14:16:18.729079 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:18.729057    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 14:16:18.729165 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:18.729100    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/196e2ce2-d807-46d8-81e5-ff0b627c3b08-insights-runtime-extractor-tls podName:196e2ce2-d807-46d8-81e5-ff0b627c3b08 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:20.729085488 +0000 UTC m=+42.680423116 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/196e2ce2-d807-46d8-81e5-ff0b627c3b08-insights-runtime-extractor-tls") pod "insights-runtime-extractor-srnvs" (UID: "196e2ce2-d807-46d8-81e5-ff0b627c3b08") : secret "insights-runtime-extractor-tls" not found
Jul 02 14:16:18.729165 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:18.729125    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/196e2ce2-d807-46d8-81e5-ff0b627c3b08-kube-rbac-proxy-cm podName:196e2ce2-d807-46d8-81e5-ff0b627c3b08 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:20.729106975 +0000 UTC m=+42.680444602 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/196e2ce2-d807-46d8-81e5-ff0b627c3b08-kube-rbac-proxy-cm") pod "insights-runtime-extractor-srnvs" (UID: "196e2ce2-d807-46d8-81e5-ff0b627c3b08") : configmap "kube-rbac-proxy" not found
Jul 02 14:16:19.156798 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:19.156773    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-f8pbz_76237012-8c88-464a-a19a-dfc41536ea5c/dns-node-resolver/0.log"
Jul 02 14:16:20.357815 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:20.357787    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rtn2p_4cb188f8-101f-4d6f-b035-90cf16106a5e/node-ca/0.log"
Jul 02 14:16:20.685924 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:20.685833    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-zmsd5" event={"ID":"5b2261d4-525c-425a-abd6-ee77b01d6392","Type":"ContainerStarted","Data":"e1d3309c2d5d8a0fa222f839fa5ddce10b23d0590b28a81b2eb88dcab8c4d691"}
Jul 02 14:16:20.685924 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:20.685889    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-zmsd5" event={"ID":"5b2261d4-525c-425a-abd6-ee77b01d6392","Type":"ContainerStarted","Data":"5a9572e3f3a73312ed37d6abfbee3eedf190d221a34476fc49307dd3376138b6"}
Jul 02 14:16:20.704973 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:20.704917    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-zmsd5" podStartSLOduration=1.501878947 podStartE2EDuration="4.704902615s" totalImagesPullingTime="3.203023668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:16:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:20.704106605 +0000 UTC m=+42.655444251" watchObservedRunningTime="2026-07-02 14:16:20.704902615 +0000 UTC m=+42.656240261"
Jul 02 14:16:20.743359 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:20.743319    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/196e2ce2-d807-46d8-81e5-ff0b627c3b08-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-srnvs\" (UID: \"196e2ce2-d807-46d8-81e5-ff0b627c3b08\") " pod="openshift-insights/insights-runtime-extractor-srnvs"
Jul 02 14:16:20.743524 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:20.743378    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/196e2ce2-d807-46d8-81e5-ff0b627c3b08-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-srnvs\" (UID: \"196e2ce2-d807-46d8-81e5-ff0b627c3b08\") " pod="openshift-insights/insights-runtime-extractor-srnvs"
Jul 02 14:16:20.743524 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:20.743460    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 14:16:20.743524 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:20.743489    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 14:16:20.743524 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:20.743518    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/196e2ce2-d807-46d8-81e5-ff0b627c3b08-kube-rbac-proxy-cm podName:196e2ce2-d807-46d8-81e5-ff0b627c3b08 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:24.743504713 +0000 UTC m=+46.694842337 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/196e2ce2-d807-46d8-81e5-ff0b627c3b08-kube-rbac-proxy-cm") pod "insights-runtime-extractor-srnvs" (UID: "196e2ce2-d807-46d8-81e5-ff0b627c3b08") : configmap "kube-rbac-proxy" not found
Jul 02 14:16:20.743648 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:20.743543    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/196e2ce2-d807-46d8-81e5-ff0b627c3b08-insights-runtime-extractor-tls podName:196e2ce2-d807-46d8-81e5-ff0b627c3b08 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:24.74353023 +0000 UTC m=+46.694867853 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/196e2ce2-d807-46d8-81e5-ff0b627c3b08-insights-runtime-extractor-tls") pod "insights-runtime-extractor-srnvs" (UID: "196e2ce2-d807-46d8-81e5-ff0b627c3b08") : secret "insights-runtime-extractor-tls" not found
Jul 02 14:16:21.357012 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:21.356974    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rlkvc_cc14ce21-e15d-484d-b934-bb9409c53413/serve-healthcheck-canary/0.log"
Jul 02 14:16:21.557637 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:21.557609    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-786f9fbc5-zmsd5_5b2261d4-525c-425a-abd6-ee77b01d6392/migrator/0.log"
Jul 02 14:16:21.756719 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:21.756695    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-786f9fbc5-zmsd5_5b2261d4-525c-425a-abd6-ee77b01d6392/graceful-termination/0.log"
Jul 02 14:16:22.154929 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:22.154824    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/977cb808-d191-4c87-b63b-8efe4dc2d00f-metrics-tls\") pod \"dns-default-295qn\" (UID: \"977cb808-d191-4c87-b63b-8efe4dc2d00f\") " pod="openshift-dns/dns-default-295qn"
Jul 02 14:16:22.155115 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:22.154989    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 14:16:22.155115 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:22.155058    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/977cb808-d191-4c87-b63b-8efe4dc2d00f-metrics-tls podName:977cb808-d191-4c87-b63b-8efe4dc2d00f nodeName:}" failed. No retries permitted until 2026-07-02 14:16:30.155039254 +0000 UTC m=+52.106376878 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/977cb808-d191-4c87-b63b-8efe4dc2d00f-metrics-tls") pod "dns-default-295qn" (UID: "977cb808-d191-4c87-b63b-8efe4dc2d00f") : secret "dns-default-metrics-tls" not found
Jul 02 14:16:22.255566 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:22.255528    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-registry-tls\") pod \"image-registry-548f7fc796-9ts6k\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") " pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:22.255730 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:22.255685    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 14:16:22.255730 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:22.255705    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-548f7fc796-9ts6k: secret "image-registry-tls" not found
Jul 02 14:16:22.255795 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:22.255760    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-registry-tls podName:5fe4ae17-d881-4f0d-b306-2c3db8d36259 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:30.255745205 +0000 UTC m=+52.207082828 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-registry-tls") pod "image-registry-548f7fc796-9ts6k" (UID: "5fe4ae17-d881-4f0d-b306-2c3db8d36259") : secret "image-registry-tls" not found
Jul 02 14:16:24.775525 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:24.775487    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/196e2ce2-d807-46d8-81e5-ff0b627c3b08-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-srnvs\" (UID: \"196e2ce2-d807-46d8-81e5-ff0b627c3b08\") " pod="openshift-insights/insights-runtime-extractor-srnvs"
Jul 02 14:16:24.775899 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:24.775539    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/196e2ce2-d807-46d8-81e5-ff0b627c3b08-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-srnvs\" (UID: \"196e2ce2-d807-46d8-81e5-ff0b627c3b08\") " pod="openshift-insights/insights-runtime-extractor-srnvs"
Jul 02 14:16:24.775899 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:24.775647    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 14:16:24.775899 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:24.775721    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/196e2ce2-d807-46d8-81e5-ff0b627c3b08-insights-runtime-extractor-tls podName:196e2ce2-d807-46d8-81e5-ff0b627c3b08 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:32.775704891 +0000 UTC m=+54.727042514 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/196e2ce2-d807-46d8-81e5-ff0b627c3b08-insights-runtime-extractor-tls") pod "insights-runtime-extractor-srnvs" (UID: "196e2ce2-d807-46d8-81e5-ff0b627c3b08") : secret "insights-runtime-extractor-tls" not found
Jul 02 14:16:24.775899 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:24.775650    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 14:16:24.775899 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:24.775827    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/196e2ce2-d807-46d8-81e5-ff0b627c3b08-kube-rbac-proxy-cm podName:196e2ce2-d807-46d8-81e5-ff0b627c3b08 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:32.775795738 +0000 UTC m=+54.727133364 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/196e2ce2-d807-46d8-81e5-ff0b627c3b08-kube-rbac-proxy-cm") pod "insights-runtime-extractor-srnvs" (UID: "196e2ce2-d807-46d8-81e5-ff0b627c3b08") : configmap "kube-rbac-proxy" not found
Jul 02 14:16:30.221253 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:30.221218    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/977cb808-d191-4c87-b63b-8efe4dc2d00f-metrics-tls\") pod \"dns-default-295qn\" (UID: \"977cb808-d191-4c87-b63b-8efe4dc2d00f\") " pod="openshift-dns/dns-default-295qn"
Jul 02 14:16:30.223941 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:30.223918    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/977cb808-d191-4c87-b63b-8efe4dc2d00f-metrics-tls\") pod \"dns-default-295qn\" (UID: \"977cb808-d191-4c87-b63b-8efe4dc2d00f\") " pod="openshift-dns/dns-default-295qn"
Jul 02 14:16:30.317554 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:30.317510    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-295qn"
Jul 02 14:16:30.321814 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:30.321783    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-registry-tls\") pod \"image-registry-548f7fc796-9ts6k\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") " pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:30.324194 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:30.324172    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-registry-tls\") pod \"image-registry-548f7fc796-9ts6k\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") " pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:30.337393 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:30.337375    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:30.465135 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:30.465108    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-295qn"]
Jul 02 14:16:30.467447 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:16:30.467422    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod977cb808_d191_4c87_b63b_8efe4dc2d00f.slice/crio-169801cf589da85a64afb83e0d40b12e3abe766f7fbe79b97b1ac7cd5af76672 WatchSource:0}: Error finding container 169801cf589da85a64afb83e0d40b12e3abe766f7fbe79b97b1ac7cd5af76672: Status 404 returned error can't find the container with id 169801cf589da85a64afb83e0d40b12e3abe766f7fbe79b97b1ac7cd5af76672
Jul 02 14:16:30.490252 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:30.490227    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-548f7fc796-9ts6k"]
Jul 02 14:16:30.493351 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:16:30.493323    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fe4ae17_d881_4f0d_b306_2c3db8d36259.slice/crio-f14eeef4619f3c71d78aa031dd486eaa963d85c13d199c17b40e257a270fbd14 WatchSource:0}: Error finding container f14eeef4619f3c71d78aa031dd486eaa963d85c13d199c17b40e257a270fbd14: Status 404 returned error can't find the container with id f14eeef4619f3c71d78aa031dd486eaa963d85c13d199c17b40e257a270fbd14
Jul 02 14:16:30.713817 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:30.713776    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-548f7fc796-9ts6k" event={"ID":"5fe4ae17-d881-4f0d-b306-2c3db8d36259","Type":"ContainerStarted","Data":"6c618d9cc7e150a1485b539c46b7c3637c9e740238f8610dffba6d2254eb0be8"}
Jul 02 14:16:30.714005 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:30.713833    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-548f7fc796-9ts6k" event={"ID":"5fe4ae17-d881-4f0d-b306-2c3db8d36259","Type":"ContainerStarted","Data":"f14eeef4619f3c71d78aa031dd486eaa963d85c13d199c17b40e257a270fbd14"}
Jul 02 14:16:30.714005 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:30.713894    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:16:30.714966 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:30.714928    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-295qn" event={"ID":"977cb808-d191-4c87-b63b-8efe4dc2d00f","Type":"ContainerStarted","Data":"169801cf589da85a64afb83e0d40b12e3abe766f7fbe79b97b1ac7cd5af76672"}
Jul 02 14:16:32.840977 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:32.840921    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/196e2ce2-d807-46d8-81e5-ff0b627c3b08-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-srnvs\" (UID: \"196e2ce2-d807-46d8-81e5-ff0b627c3b08\") " pod="openshift-insights/insights-runtime-extractor-srnvs"
Jul 02 14:16:32.841415 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:32.841000    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/196e2ce2-d807-46d8-81e5-ff0b627c3b08-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-srnvs\" (UID: \"196e2ce2-d807-46d8-81e5-ff0b627c3b08\") " pod="openshift-insights/insights-runtime-extractor-srnvs"
Jul 02 14:16:32.841609 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:32.841588    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/196e2ce2-d807-46d8-81e5-ff0b627c3b08-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-srnvs\" (UID: \"196e2ce2-d807-46d8-81e5-ff0b627c3b08\") " pod="openshift-insights/insights-runtime-extractor-srnvs"
Jul 02 14:16:32.841719 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:32.841702    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 14:16:32.841778 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:16:32.841767    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/196e2ce2-d807-46d8-81e5-ff0b627c3b08-insights-runtime-extractor-tls podName:196e2ce2-d807-46d8-81e5-ff0b627c3b08 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:48.841748741 +0000 UTC m=+70.793086370 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/196e2ce2-d807-46d8-81e5-ff0b627c3b08-insights-runtime-extractor-tls") pod "insights-runtime-extractor-srnvs" (UID: "196e2ce2-d807-46d8-81e5-ff0b627c3b08") : secret "insights-runtime-extractor-tls" not found
Jul 02 14:16:33.725125 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:33.725093    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-295qn" event={"ID":"977cb808-d191-4c87-b63b-8efe4dc2d00f","Type":"ContainerStarted","Data":"2ede4ca2a60687e4715ddae82271440903bab0c5aa098e63e1ff4b7065a37514"}
Jul 02 14:16:33.725306 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:33.725176    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-295qn" event={"ID":"977cb808-d191-4c87-b63b-8efe4dc2d00f","Type":"ContainerStarted","Data":"c9bf492647ba822f3d4d1aec9fd1e941bd6b5b5a118d6ea394d37972b339e1bc"}
Jul 02 14:16:33.725306 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:33.725236    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-295qn"
Jul 02 14:16:33.749308 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:33.749267    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-548f7fc796-9ts6k" podStartSLOduration=23.749253356 podStartE2EDuration="23.749253356s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:16:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:30.736892405 +0000 UTC m=+52.688230051" watchObservedRunningTime="2026-07-02 14:16:33.749253356 +0000 UTC m=+55.700591001"
Jul 02 14:16:33.749859 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:33.749837    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-295qn" podStartSLOduration=17.506557165 podStartE2EDuration="19.749830033s" totalImagesPullingTime="2.243272868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:16:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:33.748035626 +0000 UTC m=+55.699373273" watchObservedRunningTime="2026-07-02 14:16:33.749830033 +0000 UTC m=+55.701167678"
Jul 02 14:16:40.084407 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.084371    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d9776b5-sjv6k"]
Jul 02 14:16:40.087134 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.087117    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d9776b5-sjv6k"
Jul 02 14:16:40.091378 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.091351    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 02 14:16:40.092466 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.092436    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 02 14:16:40.092614 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.092593    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 02 14:16:40.092711 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.092597    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 02 14:16:40.092711 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.092652    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 02 14:16:40.092711 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.092654    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 02 14:16:40.092860 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.092601    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 02 14:16:40.110219 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.110190    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d9776b5-sjv6k"]
Jul 02 14:16:40.139063 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.139007    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-548f7fc796-9ts6k"]
Jul 02 14:16:40.191828 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.191788    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b4a6658b-f67a-4454-b5a0-57567097c57b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-64d9776b5-sjv6k\" (UID: \"b4a6658b-f67a-4454-b5a0-57567097c57b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d9776b5-sjv6k"
Jul 02 14:16:40.191828 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.191827    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4a6658b-f67a-4454-b5a0-57567097c57b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-64d9776b5-sjv6k\" (UID: \"b4a6658b-f67a-4454-b5a0-57567097c57b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d9776b5-sjv6k"
Jul 02 14:16:40.192107 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.191852    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b4a6658b-f67a-4454-b5a0-57567097c57b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-64d9776b5-sjv6k\" (UID: \"b4a6658b-f67a-4454-b5a0-57567097c57b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d9776b5-sjv6k"
Jul 02 14:16:40.192107 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.192009    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b4a6658b-f67a-4454-b5a0-57567097c57b-ca\") pod \"cluster-proxy-proxy-agent-64d9776b5-sjv6k\" (UID: \"b4a6658b-f67a-4454-b5a0-57567097c57b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d9776b5-sjv6k"
Jul 02 14:16:40.192107 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.192034    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b4a6658b-f67a-4454-b5a0-57567097c57b-hub\") pod \"cluster-proxy-proxy-agent-64d9776b5-sjv6k\" (UID: \"b4a6658b-f67a-4454-b5a0-57567097c57b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d9776b5-sjv6k"
Jul 02 14:16:40.192107 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.192055    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jgpc\" (UniqueName: \"kubernetes.io/projected/b4a6658b-f67a-4454-b5a0-57567097c57b-kube-api-access-6jgpc\") pod \"cluster-proxy-proxy-agent-64d9776b5-sjv6k\" (UID: \"b4a6658b-f67a-4454-b5a0-57567097c57b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d9776b5-sjv6k"
Jul 02 14:16:40.292504 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.292472    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4a6658b-f67a-4454-b5a0-57567097c57b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-64d9776b5-sjv6k\" (UID: \"b4a6658b-f67a-4454-b5a0-57567097c57b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d9776b5-sjv6k"
Jul 02 14:16:40.292504 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.292507    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b4a6658b-f67a-4454-b5a0-57567097c57b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-64d9776b5-sjv6k\" (UID: \"b4a6658b-f67a-4454-b5a0-57567097c57b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d9776b5-sjv6k"
Jul 02 14:16:40.292741 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.292710    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b4a6658b-f67a-4454-b5a0-57567097c57b-ca\") pod \"cluster-proxy-proxy-agent-64d9776b5-sjv6k\" (UID: \"b4a6658b-f67a-4454-b5a0-57567097c57b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d9776b5-sjv6k"
Jul 02 14:16:40.292801 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.292745    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b4a6658b-f67a-4454-b5a0-57567097c57b-hub\") pod \"cluster-proxy-proxy-agent-64d9776b5-sjv6k\" (UID: \"b4a6658b-f67a-4454-b5a0-57567097c57b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d9776b5-sjv6k"
Jul 02 14:16:40.292801 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.292781    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jgpc\" (UniqueName: \"kubernetes.io/projected/b4a6658b-f67a-4454-b5a0-57567097c57b-kube-api-access-6jgpc\") pod \"cluster-proxy-proxy-agent-64d9776b5-sjv6k\" (UID: \"b4a6658b-f67a-4454-b5a0-57567097c57b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d9776b5-sjv6k"
Jul 02 14:16:40.292908 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.292837    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b4a6658b-f67a-4454-b5a0-57567097c57b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-64d9776b5-sjv6k\" (UID: \"b4a6658b-f67a-4454-b5a0-57567097c57b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d9776b5-sjv6k"
Jul 02 14:16:40.293350 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.293318    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4a6658b-f67a-4454-b5a0-57567097c57b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-64d9776b5-sjv6k\" (UID: \"b4a6658b-f67a-4454-b5a0-57567097c57b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d9776b5-sjv6k"
Jul 02 14:16:40.295219 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.295186    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b4a6658b-f67a-4454-b5a0-57567097c57b-ca\") pod \"cluster-proxy-proxy-agent-64d9776b5-sjv6k\" (UID: \"b4a6658b-f67a-4454-b5a0-57567097c57b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d9776b5-sjv6k"
Jul 02 14:16:40.295327 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.295222    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b4a6658b-f67a-4454-b5a0-57567097c57b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-64d9776b5-sjv6k\" (UID: \"b4a6658b-f67a-4454-b5a0-57567097c57b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d9776b5-sjv6k"
Jul 02 14:16:40.295327 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.295296    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b4a6658b-f67a-4454-b5a0-57567097c57b-hub\") pod \"cluster-proxy-proxy-agent-64d9776b5-sjv6k\" (UID: \"b4a6658b-f67a-4454-b5a0-57567097c57b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d9776b5-sjv6k"
Jul 02 14:16:40.295448 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.295432    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b4a6658b-f67a-4454-b5a0-57567097c57b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-64d9776b5-sjv6k\" (UID: \"b4a6658b-f67a-4454-b5a0-57567097c57b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d9776b5-sjv6k"
Jul 02 14:16:40.314483 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.314457    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jgpc\" (UniqueName: \"kubernetes.io/projected/b4a6658b-f67a-4454-b5a0-57567097c57b-kube-api-access-6jgpc\") pod \"cluster-proxy-proxy-agent-64d9776b5-sjv6k\" (UID: \"b4a6658b-f67a-4454-b5a0-57567097c57b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d9776b5-sjv6k"
Jul 02 14:16:40.414227 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.414140    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d9776b5-sjv6k"
Jul 02 14:16:40.546336 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.546305    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d9776b5-sjv6k"]
Jul 02 14:16:40.548931 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:16:40.548901    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4a6658b_f67a_4454_b5a0_57567097c57b.slice/crio-5ced368185cda02913a83f4e542300c5bd28e935d7ade314184976de7c9b3c11 WatchSource:0}: Error finding container 5ced368185cda02913a83f4e542300c5bd28e935d7ade314184976de7c9b3c11: Status 404 returned error can't find the container with id 5ced368185cda02913a83f4e542300c5bd28e935d7ade314184976de7c9b3c11
Jul 02 14:16:40.657184 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.657159    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cnwb5"
Jul 02 14:16:40.747179 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:40.747145    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d9776b5-sjv6k" event={"ID":"b4a6658b-f67a-4454-b5a0-57567097c57b","Type":"ContainerStarted","Data":"5ced368185cda02913a83f4e542300c5bd28e935d7ade314184976de7c9b3c11"}
Jul 02 14:16:43.731357 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:43.731328    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-295qn"
Jul 02 14:16:44.223230 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:44.223209    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ada3d319-cd6a-42ad-b8f9-c04da51705ef-metrics-certs\") pod \"network-metrics-daemon-lpdzr\" (UID: \"ada3d319-cd6a-42ad-b8f9-c04da51705ef\") " pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:16:44.226005 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:44.225939    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 02 14:16:44.235646 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:44.235623    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ada3d319-cd6a-42ad-b8f9-c04da51705ef-metrics-certs\") pod \"network-metrics-daemon-lpdzr\" (UID: \"ada3d319-cd6a-42ad-b8f9-c04da51705ef\") " pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:16:44.323756 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:44.323724    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2bb3f329-4286-46e2-8a7d-450769d1aa15-original-pull-secret\") pod \"global-pull-secret-syncer-p64q5\" (UID: \"2bb3f329-4286-46e2-8a7d-450769d1aa15\") " pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:16:44.327049 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:44.327025    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 02 14:16:44.336175 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:44.336152    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2bb3f329-4286-46e2-8a7d-450769d1aa15-original-pull-secret\") pod \"global-pull-secret-syncer-p64q5\" (UID: \"2bb3f329-4286-46e2-8a7d-450769d1aa15\") " pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:16:44.424311 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:44.424280    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq248\" (UniqueName: \"kubernetes.io/projected/81c88401-37f6-493c-80ad-c65003c62e57-kube-api-access-bq248\") pod \"network-check-target-7gvk9\" (UID: \"81c88401-37f6-493c-80ad-c65003c62e57\") " pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:16:44.427138 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:44.427121    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 02 14:16:44.437802 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:44.437784    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 02 14:16:44.447680 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:44.447657    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq248\" (UniqueName: \"kubernetes.io/projected/81c88401-37f6-493c-80ad-c65003c62e57-kube-api-access-bq248\") pod \"network-check-target-7gvk9\" (UID: \"81c88401-37f6-493c-80ad-c65003c62e57\") " pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:16:44.488764 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:44.488692    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sn6mc\""
Jul 02 14:16:44.494626 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:44.494607    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-x92rb\""
Jul 02 14:16:44.496684 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:44.496668    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lpdzr"
Jul 02 14:16:44.497938 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:44.497920    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p64q5"
Jul 02 14:16:44.502704 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:44.502687    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:16:44.648073 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:44.648042    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7gvk9"]
Jul 02 14:16:44.651422 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:16:44.651395    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81c88401_37f6_493c_80ad_c65003c62e57.slice/crio-027f9f3caa09a1d598d5bb9797993f213f82c56432b675866b2bea24a0033600 WatchSource:0}: Error finding container 027f9f3caa09a1d598d5bb9797993f213f82c56432b675866b2bea24a0033600: Status 404 returned error can't find the container with id 027f9f3caa09a1d598d5bb9797993f213f82c56432b675866b2bea24a0033600
Jul 02 14:16:44.665697 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:44.665603    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lpdzr"]
Jul 02 14:16:44.668235 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:16:44.668210    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podada3d319_cd6a_42ad_b8f9_c04da51705ef.slice/crio-e3bd6c6a8dd9b99f98e6e30ecf5b043ce3b3937528a436de60e6ff01c44f1a1e WatchSource:0}: Error finding container e3bd6c6a8dd9b99f98e6e30ecf5b043ce3b3937528a436de60e6ff01c44f1a1e: Status 404 returned error can't find the container with id e3bd6c6a8dd9b99f98e6e30ecf5b043ce3b3937528a436de60e6ff01c44f1a1e
Jul 02 14:16:44.687076 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:44.687054    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p64q5"]
Jul 02 14:16:44.690019 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:16:44.689996    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bb3f329_4286_46e2_8a7d_450769d1aa15.slice/crio-6a644fd60d189d9befb7749e350eda698cb4874954cbca229c4cabc6b3555cec WatchSource:0}: Error finding container 6a644fd60d189d9befb7749e350eda698cb4874954cbca229c4cabc6b3555cec: Status 404 returned error can't find the container with id 6a644fd60d189d9befb7749e350eda698cb4874954cbca229c4cabc6b3555cec
Jul 02 14:16:44.764669 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:44.764587    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7gvk9" event={"ID":"81c88401-37f6-493c-80ad-c65003c62e57","Type":"ContainerStarted","Data":"027f9f3caa09a1d598d5bb9797993f213f82c56432b675866b2bea24a0033600"}
Jul 02 14:16:44.765651 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:44.765630    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p64q5" event={"ID":"2bb3f329-4286-46e2-8a7d-450769d1aa15","Type":"ContainerStarted","Data":"6a644fd60d189d9befb7749e350eda698cb4874954cbca229c4cabc6b3555cec"}
Jul 02 14:16:44.767719 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:44.767694    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d9776b5-sjv6k" event={"ID":"b4a6658b-f67a-4454-b5a0-57567097c57b","Type":"ContainerStarted","Data":"03ac2b458107a492760a8cfba2a6a0bc3e5663f64f30b20cbec2faebd8473265"}
Jul 02 14:16:44.767846 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:44.767731    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d9776b5-sjv6k" event={"ID":"b4a6658b-f67a-4454-b5a0-57567097c57b","Type":"ContainerStarted","Data":"320cbf5cf061b1f94892ba144473738cd5f1904b5cfc92ea48352f65b7dc49fb"}
Jul 02 14:16:44.767846 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:44.767746    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d9776b5-sjv6k" event={"ID":"b4a6658b-f67a-4454-b5a0-57567097c57b","Type":"ContainerStarted","Data":"8aa50f65c1b1d12bb4c375ae57df1d436d86fab3d751943903c031476dc60115"}
Jul 02 14:16:44.768861 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:44.768836    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lpdzr" event={"ID":"ada3d319-cd6a-42ad-b8f9-c04da51705ef","Type":"ContainerStarted","Data":"e3bd6c6a8dd9b99f98e6e30ecf5b043ce3b3937528a436de60e6ff01c44f1a1e"}
Jul 02 14:16:44.794555 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:44.794509    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64d9776b5-sjv6k" podStartSLOduration=1.122291448 podStartE2EDuration="4.794497664s" totalImagesPullingTime="3.672206216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:16:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:44.792745143 +0000 UTC m=+66.744082788" watchObservedRunningTime="2026-07-02 14:16:44.794497664 +0000 UTC m=+66.745835307"
Jul 02 14:16:46.779809 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:46.779776    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lpdzr" event={"ID":"ada3d319-cd6a-42ad-b8f9-c04da51705ef","Type":"ContainerStarted","Data":"4d2b473d78267a0a971c422008619280d6cf353c6177789eedfadd73e4d9c146"}
Jul 02 14:16:46.780305 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:46.779839    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lpdzr" event={"ID":"ada3d319-cd6a-42ad-b8f9-c04da51705ef","Type":"ContainerStarted","Data":"6a534266f9076ca06c0caecb4eec29a1deeffb470a01d5a7469047c1433a2429"}
Jul 02 14:16:46.808283 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:46.808234    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lpdzr" podStartSLOduration=66.463102968 podStartE2EDuration="1m7.808217277s" totalImagesPullingTime="1.345114309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:46.806117094 +0000 UTC m=+68.757454741" watchObservedRunningTime="2026-07-02 14:16:46.808217277 +0000 UTC m=+68.759554939"
Jul 02 14:16:48.859381 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:48.859334    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/196e2ce2-d807-46d8-81e5-ff0b627c3b08-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-srnvs\" (UID: \"196e2ce2-d807-46d8-81e5-ff0b627c3b08\") " pod="openshift-insights/insights-runtime-extractor-srnvs"
Jul 02 14:16:48.862052 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:48.862022    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/196e2ce2-d807-46d8-81e5-ff0b627c3b08-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-srnvs\" (UID: \"196e2ce2-d807-46d8-81e5-ff0b627c3b08\") " pod="openshift-insights/insights-runtime-extractor-srnvs"
Jul 02 14:16:49.097156 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:49.097122    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gzldj\""
Jul 02 14:16:49.104714 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:49.104678    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-srnvs"
Jul 02 14:16:49.287204 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:49.287174    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-srnvs"]
Jul 02 14:16:49.287656 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:16:49.287625    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod196e2ce2_d807_46d8_81e5_ff0b627c3b08.slice/crio-9092ee01183eb2041ead06b5ff481acc21fa279a205cc05909bf4f2afc76e3f9 WatchSource:0}: Error finding container 9092ee01183eb2041ead06b5ff481acc21fa279a205cc05909bf4f2afc76e3f9: Status 404 returned error can't find the container with id 9092ee01183eb2041ead06b5ff481acc21fa279a205cc05909bf4f2afc76e3f9
Jul 02 14:16:49.793302 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:49.793268    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7gvk9" event={"ID":"81c88401-37f6-493c-80ad-c65003c62e57","Type":"ContainerStarted","Data":"855933418f5e35e59a86ebe2138f0e65c5128d9702233ea834c6f02129f6ec74"}
Jul 02 14:16:49.793506 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:49.793342    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:16:49.794713 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:49.794685    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p64q5" event={"ID":"2bb3f329-4286-46e2-8a7d-450769d1aa15","Type":"ContainerStarted","Data":"ac78d3ff44bb8bb42a023a77957119b31f68a33012d965eb971028a8ac1bf5dc"}
Jul 02 14:16:49.795774 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:49.795753    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-srnvs" event={"ID":"196e2ce2-d807-46d8-81e5-ff0b627c3b08","Type":"ContainerStarted","Data":"9092ee01183eb2041ead06b5ff481acc21fa279a205cc05909bf4f2afc76e3f9"}
Jul 02 14:16:49.819302 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:49.819250    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7gvk9" podStartSLOduration=66.322682237 podStartE2EDuration="1m10.819234607s" totalImagesPullingTime="4.49655237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:49.816586024 +0000 UTC m=+71.767923669" watchObservedRunningTime="2026-07-02 14:16:49.819234607 +0000 UTC m=+71.770572253"
Jul 02 14:16:49.840809 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:49.840754    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-p64q5" podStartSLOduration=66.379593008 podStartE2EDuration="1m10.840742374s" totalImagesPullingTime="4.461149366s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:49.838775804 +0000 UTC m=+71.790113474" watchObservedRunningTime="2026-07-02 14:16:49.840742374 +0000 UTC m=+71.792080021"
Jul 02 14:16:50.147761 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:50.147678    2582 patch_prober.go:28] interesting pod/image-registry-548f7fc796-9ts6k container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 02 14:16:50.147761 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:50.147741    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-548f7fc796-9ts6k" podUID="5fe4ae17-d881-4f0d-b306-2c3db8d36259" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:16:53.811861 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:53.811828    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-srnvs" event={"ID":"196e2ce2-d807-46d8-81e5-ff0b627c3b08","Type":"ContainerStarted","Data":"b2586274c12cbe7db19bbb7f075de86bbccbbd10edd057c7877ae5e7d64413a7"}
Jul 02 14:16:54.818321 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:54.818282    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-srnvs" event={"ID":"196e2ce2-d807-46d8-81e5-ff0b627c3b08","Type":"ContainerStarted","Data":"59ed1d2db18c50601e95b7843a6d8d721d443aaa76153b537f7e5a106cecc9e7"}
Jul 02 14:16:57.830846 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:57.830814    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-srnvs" event={"ID":"196e2ce2-d807-46d8-81e5-ff0b627c3b08","Type":"ContainerStarted","Data":"f3586de5303a124b71c0e4709271fdd03337dd663ca654787efa970b24b03bc0"}
Jul 02 14:16:57.860546 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:16:57.860470    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-srnvs" podStartSLOduration=34.466584182 podStartE2EDuration="41.860450591s" totalImagesPullingTime="7.393866409s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:16:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:57.859906153 +0000 UTC m=+79.811243803" watchObservedRunningTime="2026-07-02 14:16:57.860450591 +0000 UTC m=+79.811788237"
Jul 02 14:17:00.143755 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:00.143647    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:17:05.158535 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.158487    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-548f7fc796-9ts6k" podUID="5fe4ae17-d881-4f0d-b306-2c3db8d36259" containerName="registry" containerID="cri-o://6c618d9cc7e150a1485b539c46b7c3637c9e740238f8610dffba6d2254eb0be8" gracePeriod=30
Jul 02 14:17:05.402344 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.402320    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:17:05.478638 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.478612    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qs4s5\" (UniqueName: \"kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-kube-api-access-qs4s5\") pod \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") "
Jul 02 14:17:05.478790 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.478649    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5fe4ae17-d881-4f0d-b306-2c3db8d36259-ca-trust-extracted\") pod \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") "
Jul 02 14:17:05.478790 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.478698    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5fe4ae17-d881-4f0d-b306-2c3db8d36259-installation-pull-secrets\") pod \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") "
Jul 02 14:17:05.478790 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.478717    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5fe4ae17-d881-4f0d-b306-2c3db8d36259-image-registry-private-configuration\") pod \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") "
Jul 02 14:17:05.478790 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.478738    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5fe4ae17-d881-4f0d-b306-2c3db8d36259-registry-certificates\") pod \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") "
Jul 02 14:17:05.478790 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.478771    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-registry-tls\") pod \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") "
Jul 02 14:17:05.479069 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.478803    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-bound-sa-token\") pod \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") "
Jul 02 14:17:05.479069 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.478827    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5fe4ae17-d881-4f0d-b306-2c3db8d36259-trusted-ca\") pod \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\" (UID: \"5fe4ae17-d881-4f0d-b306-2c3db8d36259\") "
Jul 02 14:17:05.479272 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.479241    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fe4ae17-d881-4f0d-b306-2c3db8d36259-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "5fe4ae17-d881-4f0d-b306-2c3db8d36259" (UID: "5fe4ae17-d881-4f0d-b306-2c3db8d36259"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:17:05.480101 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.480071    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fe4ae17-d881-4f0d-b306-2c3db8d36259-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "5fe4ae17-d881-4f0d-b306-2c3db8d36259" (UID: "5fe4ae17-d881-4f0d-b306-2c3db8d36259"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:17:05.481435 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.481396    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-kube-api-access-qs4s5" (OuterVolumeSpecName: "kube-api-access-qs4s5") pod "5fe4ae17-d881-4f0d-b306-2c3db8d36259" (UID: "5fe4ae17-d881-4f0d-b306-2c3db8d36259"). InnerVolumeSpecName "kube-api-access-qs4s5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:17:05.481826 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.481795    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fe4ae17-d881-4f0d-b306-2c3db8d36259-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "5fe4ae17-d881-4f0d-b306-2c3db8d36259" (UID: "5fe4ae17-d881-4f0d-b306-2c3db8d36259"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 14:17:05.482025 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.481992    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fe4ae17-d881-4f0d-b306-2c3db8d36259-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "5fe4ae17-d881-4f0d-b306-2c3db8d36259" (UID: "5fe4ae17-d881-4f0d-b306-2c3db8d36259"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 14:17:05.482122 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.482018    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5fe4ae17-d881-4f0d-b306-2c3db8d36259" (UID: "5fe4ae17-d881-4f0d-b306-2c3db8d36259"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:17:05.482166 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.482146    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "5fe4ae17-d881-4f0d-b306-2c3db8d36259" (UID: "5fe4ae17-d881-4f0d-b306-2c3db8d36259"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:17:05.488061 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.488031    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fe4ae17-d881-4f0d-b306-2c3db8d36259-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "5fe4ae17-d881-4f0d-b306-2c3db8d36259" (UID: "5fe4ae17-d881-4f0d-b306-2c3db8d36259"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 14:17:05.579716 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.579679    2582 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-bound-sa-token\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:17:05.579716 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.579708    2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5fe4ae17-d881-4f0d-b306-2c3db8d36259-trusted-ca\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:17:05.579716 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.579717    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qs4s5\" (UniqueName: \"kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-kube-api-access-qs4s5\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:17:05.579949 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.579727    2582 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5fe4ae17-d881-4f0d-b306-2c3db8d36259-ca-trust-extracted\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:17:05.579949 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.579736    2582 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5fe4ae17-d881-4f0d-b306-2c3db8d36259-installation-pull-secrets\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:17:05.579949 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.579746    2582 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5fe4ae17-d881-4f0d-b306-2c3db8d36259-image-registry-private-configuration\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:17:05.579949 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.579756    2582 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5fe4ae17-d881-4f0d-b306-2c3db8d36259-registry-certificates\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:17:05.579949 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.579764    2582 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5fe4ae17-d881-4f0d-b306-2c3db8d36259-registry-tls\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:17:05.861806 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.861718    2582 generic.go:356] "Generic (PLEG): container finished" podID="5fe4ae17-d881-4f0d-b306-2c3db8d36259" containerID="6c618d9cc7e150a1485b539c46b7c3637c9e740238f8610dffba6d2254eb0be8" exitCode=0
Jul 02 14:17:05.861806 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.861773    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-548f7fc796-9ts6k"
Jul 02 14:17:05.861806 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.861795    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-548f7fc796-9ts6k" event={"ID":"5fe4ae17-d881-4f0d-b306-2c3db8d36259","Type":"ContainerDied","Data":"6c618d9cc7e150a1485b539c46b7c3637c9e740238f8610dffba6d2254eb0be8"}
Jul 02 14:17:05.862052 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.861863    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-548f7fc796-9ts6k" event={"ID":"5fe4ae17-d881-4f0d-b306-2c3db8d36259","Type":"ContainerDied","Data":"f14eeef4619f3c71d78aa031dd486eaa963d85c13d199c17b40e257a270fbd14"}
Jul 02 14:17:05.862052 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.861886    2582 scope.go:117] "RemoveContainer" containerID="6c618d9cc7e150a1485b539c46b7c3637c9e740238f8610dffba6d2254eb0be8"
Jul 02 14:17:05.871212 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.871190    2582 scope.go:117] "RemoveContainer" containerID="6c618d9cc7e150a1485b539c46b7c3637c9e740238f8610dffba6d2254eb0be8"
Jul 02 14:17:05.871468 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:17:05.871448    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c618d9cc7e150a1485b539c46b7c3637c9e740238f8610dffba6d2254eb0be8\": container with ID starting with 6c618d9cc7e150a1485b539c46b7c3637c9e740238f8610dffba6d2254eb0be8 not found: ID does not exist" containerID="6c618d9cc7e150a1485b539c46b7c3637c9e740238f8610dffba6d2254eb0be8"
Jul 02 14:17:05.871509 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.871477    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c618d9cc7e150a1485b539c46b7c3637c9e740238f8610dffba6d2254eb0be8"} err="failed to get container status \"6c618d9cc7e150a1485b539c46b7c3637c9e740238f8610dffba6d2254eb0be8\": rpc error: code = NotFound desc = could not find container \"6c618d9cc7e150a1485b539c46b7c3637c9e740238f8610dffba6d2254eb0be8\": container with ID starting with 6c618d9cc7e150a1485b539c46b7c3637c9e740238f8610dffba6d2254eb0be8 not found: ID does not exist"
Jul 02 14:17:05.886709 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.886678    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-548f7fc796-9ts6k"]
Jul 02 14:17:05.890821 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:05.890801    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-548f7fc796-9ts6k"]
Jul 02 14:17:06.473678 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:06.473642    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fe4ae17-d881-4f0d-b306-2c3db8d36259" path="/var/lib/kubelet/pods/5fe4ae17-d881-4f0d-b306-2c3db8d36259/volumes"
Jul 02 14:17:11.070074 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.070036    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-vqtb2"]
Jul 02 14:17:11.070433 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.070326    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fe4ae17-d881-4f0d-b306-2c3db8d36259" containerName="registry"
Jul 02 14:17:11.070433 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.070341    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fe4ae17-d881-4f0d-b306-2c3db8d36259" containerName="registry"
Jul 02 14:17:11.070433 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.070387    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fe4ae17-d881-4f0d-b306-2c3db8d36259" containerName="registry"
Jul 02 14:17:11.075489 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.075471    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-845b954455-vqtb2"
Jul 02 14:17:11.079349 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.079325    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 02 14:17:11.079434 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.079325    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 02 14:17:11.079671 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.079651    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 02 14:17:11.081011 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.080985    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 02 14:17:11.081113 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.081036    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 02 14:17:11.081113 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.081099    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-pbcj4\""
Jul 02 14:17:11.081226 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.081207    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 02 14:17:11.090495 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.090471    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-vqtb2"]
Jul 02 14:17:11.114066 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.114033    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32981337-e63c-486f-b119-cb9fc0fe22f8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-vqtb2\" (UID: \"32981337-e63c-486f-b119-cb9fc0fe22f8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-vqtb2"
Jul 02 14:17:11.114181 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.114075    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/32981337-e63c-486f-b119-cb9fc0fe22f8-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-vqtb2\" (UID: \"32981337-e63c-486f-b119-cb9fc0fe22f8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-vqtb2"
Jul 02 14:17:11.114181 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.114104    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/32981337-e63c-486f-b119-cb9fc0fe22f8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-vqtb2\" (UID: \"32981337-e63c-486f-b119-cb9fc0fe22f8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-vqtb2"
Jul 02 14:17:11.114258 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.114177    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/32981337-e63c-486f-b119-cb9fc0fe22f8-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-vqtb2\" (UID: \"32981337-e63c-486f-b119-cb9fc0fe22f8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-vqtb2"
Jul 02 14:17:11.114258 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.114212    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32981337-e63c-486f-b119-cb9fc0fe22f8-metrics-client-ca\") pod \"kube-state-metrics-845b954455-vqtb2\" (UID: \"32981337-e63c-486f-b119-cb9fc0fe22f8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-vqtb2"
Jul 02 14:17:11.114258 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.114237    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhc2p\" (UniqueName: \"kubernetes.io/projected/32981337-e63c-486f-b119-cb9fc0fe22f8-kube-api-access-xhc2p\") pod \"kube-state-metrics-845b954455-vqtb2\" (UID: \"32981337-e63c-486f-b119-cb9fc0fe22f8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-vqtb2"
Jul 02 14:17:11.175685 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.175651    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-d2qn4"]
Jul 02 14:17:11.178879 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.178856    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:11.183169 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.183152    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 02 14:17:11.184018 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.183997    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 02 14:17:11.184127 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.184053    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-v5lgr\""
Jul 02 14:17:11.184127 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.184065    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 02 14:17:11.215116 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.215088    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/32981337-e63c-486f-b119-cb9fc0fe22f8-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-vqtb2\" (UID: \"32981337-e63c-486f-b119-cb9fc0fe22f8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-vqtb2"
Jul 02 14:17:11.215241 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.215127    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd865561-ea6f-49e2-abd3-84f6870125ff-metrics-client-ca\") pod \"node-exporter-d2qn4\" (UID: \"dd865561-ea6f-49e2-abd3-84f6870125ff\") " pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:11.215241 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.215145    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32981337-e63c-486f-b119-cb9fc0fe22f8-metrics-client-ca\") pod \"kube-state-metrics-845b954455-vqtb2\" (UID: \"32981337-e63c-486f-b119-cb9fc0fe22f8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-vqtb2"
Jul 02 14:17:11.215241 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.215171    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dd865561-ea6f-49e2-abd3-84f6870125ff-node-exporter-accelerators-collector-config\") pod \"node-exporter-d2qn4\" (UID: \"dd865561-ea6f-49e2-abd3-84f6870125ff\") " pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:11.215241 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.215196    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dd865561-ea6f-49e2-abd3-84f6870125ff-node-exporter-tls\") pod \"node-exporter-d2qn4\" (UID: \"dd865561-ea6f-49e2-abd3-84f6870125ff\") " pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:11.215241 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.215219    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dd865561-ea6f-49e2-abd3-84f6870125ff-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d2qn4\" (UID: \"dd865561-ea6f-49e2-abd3-84f6870125ff\") " pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:11.215241 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.215237    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/32981337-e63c-486f-b119-cb9fc0fe22f8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-vqtb2\" (UID: \"32981337-e63c-486f-b119-cb9fc0fe22f8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-vqtb2"
Jul 02 14:17:11.215551 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.215279    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dd865561-ea6f-49e2-abd3-84f6870125ff-node-exporter-textfile\") pod \"node-exporter-d2qn4\" (UID: \"dd865561-ea6f-49e2-abd3-84f6870125ff\") " pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:11.215551 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.215373    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dd865561-ea6f-49e2-abd3-84f6870125ff-node-exporter-wtmp\") pod \"node-exporter-d2qn4\" (UID: \"dd865561-ea6f-49e2-abd3-84f6870125ff\") " pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:11.215551 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.215396    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dd865561-ea6f-49e2-abd3-84f6870125ff-root\") pod \"node-exporter-d2qn4\" (UID: \"dd865561-ea6f-49e2-abd3-84f6870125ff\") " pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:11.215551 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.215423    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32981337-e63c-486f-b119-cb9fc0fe22f8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-vqtb2\" (UID: \"32981337-e63c-486f-b119-cb9fc0fe22f8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-vqtb2"
Jul 02 14:17:11.215551 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.215496    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/32981337-e63c-486f-b119-cb9fc0fe22f8-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-vqtb2\" (UID: \"32981337-e63c-486f-b119-cb9fc0fe22f8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-vqtb2"
Jul 02 14:17:11.215551 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.215498    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dd865561-ea6f-49e2-abd3-84f6870125ff-sys\") pod \"node-exporter-d2qn4\" (UID: \"dd865561-ea6f-49e2-abd3-84f6870125ff\") " pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:11.215839 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.215557    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xm8z\" (UniqueName: \"kubernetes.io/projected/dd865561-ea6f-49e2-abd3-84f6870125ff-kube-api-access-5xm8z\") pod \"node-exporter-d2qn4\" (UID: \"dd865561-ea6f-49e2-abd3-84f6870125ff\") " pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:11.215839 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.215599    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/32981337-e63c-486f-b119-cb9fc0fe22f8-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-vqtb2\" (UID: \"32981337-e63c-486f-b119-cb9fc0fe22f8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-vqtb2"
Jul 02 14:17:11.215839 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.215635    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhc2p\" (UniqueName: \"kubernetes.io/projected/32981337-e63c-486f-b119-cb9fc0fe22f8-kube-api-access-xhc2p\") pod \"kube-state-metrics-845b954455-vqtb2\" (UID: \"32981337-e63c-486f-b119-cb9fc0fe22f8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-vqtb2"
Jul 02 14:17:11.215839 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.215833    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32981337-e63c-486f-b119-cb9fc0fe22f8-metrics-client-ca\") pod \"kube-state-metrics-845b954455-vqtb2\" (UID: \"32981337-e63c-486f-b119-cb9fc0fe22f8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-vqtb2"
Jul 02 14:17:11.216033 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.215896    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/32981337-e63c-486f-b119-cb9fc0fe22f8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-vqtb2\" (UID: \"32981337-e63c-486f-b119-cb9fc0fe22f8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-vqtb2"
Jul 02 14:17:11.217806 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.217789    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32981337-e63c-486f-b119-cb9fc0fe22f8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-vqtb2\" (UID: \"32981337-e63c-486f-b119-cb9fc0fe22f8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-vqtb2"
Jul 02 14:17:11.218014 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.217998    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/32981337-e63c-486f-b119-cb9fc0fe22f8-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-vqtb2\" (UID: \"32981337-e63c-486f-b119-cb9fc0fe22f8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-vqtb2"
Jul 02 14:17:11.237347 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.237320    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhc2p\" (UniqueName: \"kubernetes.io/projected/32981337-e63c-486f-b119-cb9fc0fe22f8-kube-api-access-xhc2p\") pod \"kube-state-metrics-845b954455-vqtb2\" (UID: \"32981337-e63c-486f-b119-cb9fc0fe22f8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-vqtb2"
Jul 02 14:17:11.316511 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.316478    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd865561-ea6f-49e2-abd3-84f6870125ff-metrics-client-ca\") pod \"node-exporter-d2qn4\" (UID: \"dd865561-ea6f-49e2-abd3-84f6870125ff\") " pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:11.316652 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.316537    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dd865561-ea6f-49e2-abd3-84f6870125ff-node-exporter-accelerators-collector-config\") pod \"node-exporter-d2qn4\" (UID: \"dd865561-ea6f-49e2-abd3-84f6870125ff\") " pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:11.316652 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.316560    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dd865561-ea6f-49e2-abd3-84f6870125ff-node-exporter-tls\") pod \"node-exporter-d2qn4\" (UID: \"dd865561-ea6f-49e2-abd3-84f6870125ff\") " pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:11.316652 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.316585    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dd865561-ea6f-49e2-abd3-84f6870125ff-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d2qn4\" (UID: \"dd865561-ea6f-49e2-abd3-84f6870125ff\") " pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:11.316652 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.316602    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dd865561-ea6f-49e2-abd3-84f6870125ff-node-exporter-textfile\") pod \"node-exporter-d2qn4\" (UID: \"dd865561-ea6f-49e2-abd3-84f6870125ff\") " pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:11.316652 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.316630    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dd865561-ea6f-49e2-abd3-84f6870125ff-node-exporter-wtmp\") pod \"node-exporter-d2qn4\" (UID: \"dd865561-ea6f-49e2-abd3-84f6870125ff\") " pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:11.316908 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.316654    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dd865561-ea6f-49e2-abd3-84f6870125ff-root\") pod \"node-exporter-d2qn4\" (UID: \"dd865561-ea6f-49e2-abd3-84f6870125ff\") " pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:11.316908 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.316695    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dd865561-ea6f-49e2-abd3-84f6870125ff-sys\") pod \"node-exporter-d2qn4\" (UID: \"dd865561-ea6f-49e2-abd3-84f6870125ff\") " pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:11.316908 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:17:11.316713    2582 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 02 14:17:11.316908 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:17:11.316789    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd865561-ea6f-49e2-abd3-84f6870125ff-node-exporter-tls podName:dd865561-ea6f-49e2-abd3-84f6870125ff nodeName:}" failed. No retries permitted until 2026-07-02 14:17:11.816766274 +0000 UTC m=+93.768103916 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/dd865561-ea6f-49e2-abd3-84f6870125ff-node-exporter-tls") pod "node-exporter-d2qn4" (UID: "dd865561-ea6f-49e2-abd3-84f6870125ff") : secret "node-exporter-tls" not found
Jul 02 14:17:11.316908 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.316799    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dd865561-ea6f-49e2-abd3-84f6870125ff-node-exporter-wtmp\") pod \"node-exporter-d2qn4\" (UID: \"dd865561-ea6f-49e2-abd3-84f6870125ff\") " pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:11.316908 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.316716    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xm8z\" (UniqueName: \"kubernetes.io/projected/dd865561-ea6f-49e2-abd3-84f6870125ff-kube-api-access-5xm8z\") pod \"node-exporter-d2qn4\" (UID: \"dd865561-ea6f-49e2-abd3-84f6870125ff\") " pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:11.316908 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.316869    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dd865561-ea6f-49e2-abd3-84f6870125ff-root\") pod \"node-exporter-d2qn4\" (UID: \"dd865561-ea6f-49e2-abd3-84f6870125ff\") " pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:11.317206 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.316920    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dd865561-ea6f-49e2-abd3-84f6870125ff-sys\") pod \"node-exporter-d2qn4\" (UID: \"dd865561-ea6f-49e2-abd3-84f6870125ff\") " pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:11.317206 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.317067    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dd865561-ea6f-49e2-abd3-84f6870125ff-node-exporter-textfile\") pod \"node-exporter-d2qn4\" (UID: \"dd865561-ea6f-49e2-abd3-84f6870125ff\") " pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:11.317271 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.317206    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd865561-ea6f-49e2-abd3-84f6870125ff-metrics-client-ca\") pod \"node-exporter-d2qn4\" (UID: \"dd865561-ea6f-49e2-abd3-84f6870125ff\") " pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:11.317271 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.317206    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dd865561-ea6f-49e2-abd3-84f6870125ff-node-exporter-accelerators-collector-config\") pod \"node-exporter-d2qn4\" (UID: \"dd865561-ea6f-49e2-abd3-84f6870125ff\") " pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:11.319088 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.319070    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dd865561-ea6f-49e2-abd3-84f6870125ff-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d2qn4\" (UID: \"dd865561-ea6f-49e2-abd3-84f6870125ff\") " pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:11.350862 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.350807    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xm8z\" (UniqueName: \"kubernetes.io/projected/dd865561-ea6f-49e2-abd3-84f6870125ff-kube-api-access-5xm8z\") pod \"node-exporter-d2qn4\" (UID: \"dd865561-ea6f-49e2-abd3-84f6870125ff\") " pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:11.383999 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.383946    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-845b954455-vqtb2"
Jul 02 14:17:11.533818 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.533789    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-vqtb2"]
Jul 02 14:17:11.537047 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:17:11.537022    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32981337_e63c_486f_b119_cb9fc0fe22f8.slice/crio-a2c8f3d11d3cfa13cb2ca15da559895c3c5706f65d2dc66cf0c724d856fa98ea WatchSource:0}: Error finding container a2c8f3d11d3cfa13cb2ca15da559895c3c5706f65d2dc66cf0c724d856fa98ea: Status 404 returned error can't find the container with id a2c8f3d11d3cfa13cb2ca15da559895c3c5706f65d2dc66cf0c724d856fa98ea
Jul 02 14:17:11.820975 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.820931    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dd865561-ea6f-49e2-abd3-84f6870125ff-node-exporter-tls\") pod \"node-exporter-d2qn4\" (UID: \"dd865561-ea6f-49e2-abd3-84f6870125ff\") " pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:11.823283 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.823254    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dd865561-ea6f-49e2-abd3-84f6870125ff-node-exporter-tls\") pod \"node-exporter-d2qn4\" (UID: \"dd865561-ea6f-49e2-abd3-84f6870125ff\") " pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:11.882214 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:11.882180    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-vqtb2" event={"ID":"32981337-e63c-486f-b119-cb9fc0fe22f8","Type":"ContainerStarted","Data":"a2c8f3d11d3cfa13cb2ca15da559895c3c5706f65d2dc66cf0c724d856fa98ea"}
Jul 02 14:17:12.087156 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:12.087055    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d2qn4"
Jul 02 14:17:12.098809 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:17:12.098778    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd865561_ea6f_49e2_abd3_84f6870125ff.slice/crio-39b0a7d8ef1ee763fbfaab9716f539f18d2a228ed0dde8f1a34cc9619f9883f3 WatchSource:0}: Error finding container 39b0a7d8ef1ee763fbfaab9716f539f18d2a228ed0dde8f1a34cc9619f9883f3: Status 404 returned error can't find the container with id 39b0a7d8ef1ee763fbfaab9716f539f18d2a228ed0dde8f1a34cc9619f9883f3
Jul 02 14:17:12.886307 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:12.886270    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d2qn4" event={"ID":"dd865561-ea6f-49e2-abd3-84f6870125ff","Type":"ContainerStarted","Data":"39b0a7d8ef1ee763fbfaab9716f539f18d2a228ed0dde8f1a34cc9619f9883f3"}
Jul 02 14:17:13.890596 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:13.890542    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d2qn4" event={"ID":"dd865561-ea6f-49e2-abd3-84f6870125ff","Type":"ContainerStarted","Data":"6e01127a6b6debe4171e86bc8a94a91eed1799b006e0826741671249c6f393f6"}
Jul 02 14:17:13.892686 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:13.892658    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-vqtb2" event={"ID":"32981337-e63c-486f-b119-cb9fc0fe22f8","Type":"ContainerStarted","Data":"b126e9444802c6109fd4824c1676e8db3ba900eee60fa83e7fc63ec9e0f1bf8b"}
Jul 02 14:17:13.892786 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:13.892719    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-vqtb2" event={"ID":"32981337-e63c-486f-b119-cb9fc0fe22f8","Type":"ContainerStarted","Data":"644e5316c3368a10ee07dd0388b999dcfbe776e5ac85374fdd69ebef4641cffe"}
Jul 02 14:17:13.892786 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:13.892751    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-vqtb2" event={"ID":"32981337-e63c-486f-b119-cb9fc0fe22f8","Type":"ContainerStarted","Data":"e6135d35cf54ff0eb7a7ef1cade9d512c3ff570caeb5652fb1cc0890d79f7132"}
Jul 02 14:17:13.944519 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:13.944469    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-845b954455-vqtb2" podStartSLOduration=1.220962002 podStartE2EDuration="2.944450975s" totalImagesPullingTime="1.723488973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:17:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:17:13.943513061 +0000 UTC m=+95.894850707" watchObservedRunningTime="2026-07-02 14:17:13.944450975 +0000 UTC m=+95.895788621"
Jul 02 14:17:14.898329 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:14.898291    2582 generic.go:356] "Generic (PLEG): container finished" podID="dd865561-ea6f-49e2-abd3-84f6870125ff" containerID="6e01127a6b6debe4171e86bc8a94a91eed1799b006e0826741671249c6f393f6" exitCode=0
Jul 02 14:17:14.898729 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:14.898383    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d2qn4" event={"ID":"dd865561-ea6f-49e2-abd3-84f6870125ff","Type":"ContainerDied","Data":"6e01127a6b6debe4171e86bc8a94a91eed1799b006e0826741671249c6f393f6"}
Jul 02 14:17:15.821179 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:15.821144    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-qhbds"]
Jul 02 14:17:15.833811 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:15.833784    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-cd8656b8-qhbds"
Jul 02 14:17:15.847933 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:15.847911    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-gtzm4\""
Jul 02 14:17:15.852301 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:15.852282    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 02 14:17:15.863740 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:15.863721    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-qhbds"]
Jul 02 14:17:15.904278 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:15.904250    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d2qn4" event={"ID":"dd865561-ea6f-49e2-abd3-84f6870125ff","Type":"ContainerStarted","Data":"b398b4fed392adef3843a3a439831419b274f6a1e0026f60c37ee9a2038406b8"}
Jul 02 14:17:15.904652 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:15.904297    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d2qn4" event={"ID":"dd865561-ea6f-49e2-abd3-84f6870125ff","Type":"ContainerStarted","Data":"21a3c48a63e5437072b997c256e6d1d1265610f0519498125f31264c9b8f92e9"}
Jul 02 14:17:15.951646 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:15.951604    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b8fc3257-7848-48da-9eab-8db8a9bd26c7-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-qhbds\" (UID: \"b8fc3257-7848-48da-9eab-8db8a9bd26c7\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-qhbds"
Jul 02 14:17:15.951906 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:15.951782    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s45wn\" (UniqueName: \"kubernetes.io/projected/b8fc3257-7848-48da-9eab-8db8a9bd26c7-kube-api-access-s45wn\") pod \"monitoring-plugin-cd8656b8-qhbds\" (UID: \"b8fc3257-7848-48da-9eab-8db8a9bd26c7\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-qhbds"
Jul 02 14:17:15.955563 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:15.955521    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-d2qn4" podStartSLOduration=3.1638932889999998 podStartE2EDuration="4.9555059s" totalImagesPullingTime="1.717598128s" totalInitContainerRuntime="74.014483ms" isStatefulPod=true podCreationTimestamp="2026-07-02 14:17:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:17:15.955065858 +0000 UTC m=+97.906403504" watchObservedRunningTime="2026-07-02 14:17:15.9555059 +0000 UTC m=+97.906843546"
Jul 02 14:17:16.052555 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.052514    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s45wn\" (UniqueName: \"kubernetes.io/projected/b8fc3257-7848-48da-9eab-8db8a9bd26c7-kube-api-access-s45wn\") pod \"monitoring-plugin-cd8656b8-qhbds\" (UID: \"b8fc3257-7848-48da-9eab-8db8a9bd26c7\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-qhbds"
Jul 02 14:17:16.052772 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.052573    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b8fc3257-7848-48da-9eab-8db8a9bd26c7-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-qhbds\" (UID: \"b8fc3257-7848-48da-9eab-8db8a9bd26c7\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-qhbds"
Jul 02 14:17:16.055258 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.055234    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b8fc3257-7848-48da-9eab-8db8a9bd26c7-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-qhbds\" (UID: \"b8fc3257-7848-48da-9eab-8db8a9bd26c7\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-qhbds"
Jul 02 14:17:16.072810 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.072744    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s45wn\" (UniqueName: \"kubernetes.io/projected/b8fc3257-7848-48da-9eab-8db8a9bd26c7-kube-api-access-s45wn\") pod \"monitoring-plugin-cd8656b8-qhbds\" (UID: \"b8fc3257-7848-48da-9eab-8db8a9bd26c7\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-qhbds"
Jul 02 14:17:16.094918 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.094880    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7f787b8f48-9lc74"]
Jul 02 14:17:16.110710 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.110678    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74"
Jul 02 14:17:16.115598 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.115563    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 02 14:17:16.115747 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.115602    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 02 14:17:16.115747 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.115737    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-1t7nkqrb91n4a\""
Jul 02 14:17:16.116023 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.115998    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 02 14:17:16.117410 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.117054    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 02 14:17:16.120120 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.118355    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-nr9nz\""
Jul 02 14:17:16.120589 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.120374    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7f787b8f48-9lc74"]
Jul 02 14:17:16.142765 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.142730    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-cd8656b8-qhbds"
Jul 02 14:17:16.154020 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.153939    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/07974ea0-3f57-4174-b4bf-65639138b2f6-secret-metrics-server-client-certs\") pod \"metrics-server-7f787b8f48-9lc74\" (UID: \"07974ea0-3f57-4174-b4bf-65639138b2f6\") " pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74"
Jul 02 14:17:16.154207 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.154043    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07974ea0-3f57-4174-b4bf-65639138b2f6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7f787b8f48-9lc74\" (UID: \"07974ea0-3f57-4174-b4bf-65639138b2f6\") " pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74"
Jul 02 14:17:16.154207 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.154066    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/07974ea0-3f57-4174-b4bf-65639138b2f6-audit-log\") pod \"metrics-server-7f787b8f48-9lc74\" (UID: \"07974ea0-3f57-4174-b4bf-65639138b2f6\") " pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74"
Jul 02 14:17:16.154207 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.154123    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/07974ea0-3f57-4174-b4bf-65639138b2f6-client-ca-bundle\") pod \"metrics-server-7f787b8f48-9lc74\" (UID: \"07974ea0-3f57-4174-b4bf-65639138b2f6\") " pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74"
Jul 02 14:17:16.154207 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.154172    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/07974ea0-3f57-4174-b4bf-65639138b2f6-metrics-server-audit-profiles\") pod \"metrics-server-7f787b8f48-9lc74\" (UID: \"07974ea0-3f57-4174-b4bf-65639138b2f6\") " pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74"
Jul 02 14:17:16.154346 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.154235    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zmpp\" (UniqueName: \"kubernetes.io/projected/07974ea0-3f57-4174-b4bf-65639138b2f6-kube-api-access-5zmpp\") pod \"metrics-server-7f787b8f48-9lc74\" (UID: \"07974ea0-3f57-4174-b4bf-65639138b2f6\") " pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74"
Jul 02 14:17:16.154346 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.154279    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/07974ea0-3f57-4174-b4bf-65639138b2f6-secret-metrics-server-tls\") pod \"metrics-server-7f787b8f48-9lc74\" (UID: \"07974ea0-3f57-4174-b4bf-65639138b2f6\") " pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74"
Jul 02 14:17:16.255532 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.255494    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07974ea0-3f57-4174-b4bf-65639138b2f6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7f787b8f48-9lc74\" (UID: \"07974ea0-3f57-4174-b4bf-65639138b2f6\") " pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74"
Jul 02 14:17:16.255701 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.255546    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/07974ea0-3f57-4174-b4bf-65639138b2f6-audit-log\") pod \"metrics-server-7f787b8f48-9lc74\" (UID: \"07974ea0-3f57-4174-b4bf-65639138b2f6\") " pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74"
Jul 02 14:17:16.255701 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.255587    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/07974ea0-3f57-4174-b4bf-65639138b2f6-client-ca-bundle\") pod \"metrics-server-7f787b8f48-9lc74\" (UID: \"07974ea0-3f57-4174-b4bf-65639138b2f6\") " pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74"
Jul 02 14:17:16.255701 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.255643    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/07974ea0-3f57-4174-b4bf-65639138b2f6-metrics-server-audit-profiles\") pod \"metrics-server-7f787b8f48-9lc74\" (UID: \"07974ea0-3f57-4174-b4bf-65639138b2f6\") " pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74"
Jul 02 14:17:16.255701 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.255672    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zmpp\" (UniqueName: \"kubernetes.io/projected/07974ea0-3f57-4174-b4bf-65639138b2f6-kube-api-access-5zmpp\") pod \"metrics-server-7f787b8f48-9lc74\" (UID: \"07974ea0-3f57-4174-b4bf-65639138b2f6\") " pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74"
Jul 02 14:17:16.256054 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.256014    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/07974ea0-3f57-4174-b4bf-65639138b2f6-secret-metrics-server-tls\") pod \"metrics-server-7f787b8f48-9lc74\" (UID: \"07974ea0-3f57-4174-b4bf-65639138b2f6\") " pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74"
Jul 02 14:17:16.256171 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.256053    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/07974ea0-3f57-4174-b4bf-65639138b2f6-audit-log\") pod \"metrics-server-7f787b8f48-9lc74\" (UID: \"07974ea0-3f57-4174-b4bf-65639138b2f6\") " pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74"
Jul 02 14:17:16.256171 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.256061    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/07974ea0-3f57-4174-b4bf-65639138b2f6-secret-metrics-server-client-certs\") pod \"metrics-server-7f787b8f48-9lc74\" (UID: \"07974ea0-3f57-4174-b4bf-65639138b2f6\") " pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74"
Jul 02 14:17:16.256558 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.256509    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07974ea0-3f57-4174-b4bf-65639138b2f6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7f787b8f48-9lc74\" (UID: \"07974ea0-3f57-4174-b4bf-65639138b2f6\") " pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74"
Jul 02 14:17:16.257058 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.257036    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/07974ea0-3f57-4174-b4bf-65639138b2f6-metrics-server-audit-profiles\") pod \"metrics-server-7f787b8f48-9lc74\" (UID: \"07974ea0-3f57-4174-b4bf-65639138b2f6\") " pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74"
Jul 02 14:17:16.258824 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.258803    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/07974ea0-3f57-4174-b4bf-65639138b2f6-secret-metrics-server-tls\") pod \"metrics-server-7f787b8f48-9lc74\" (UID: \"07974ea0-3f57-4174-b4bf-65639138b2f6\") " pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74"
Jul 02 14:17:16.259035 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.259017    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/07974ea0-3f57-4174-b4bf-65639138b2f6-client-ca-bundle\") pod \"metrics-server-7f787b8f48-9lc74\" (UID: \"07974ea0-3f57-4174-b4bf-65639138b2f6\") " pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74"
Jul 02 14:17:16.259084 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.259063    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/07974ea0-3f57-4174-b4bf-65639138b2f6-secret-metrics-server-client-certs\") pod \"metrics-server-7f787b8f48-9lc74\" (UID: \"07974ea0-3f57-4174-b4bf-65639138b2f6\") " pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74"
Jul 02 14:17:16.312778 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.312748    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zmpp\" (UniqueName: \"kubernetes.io/projected/07974ea0-3f57-4174-b4bf-65639138b2f6-kube-api-access-5zmpp\") pod \"metrics-server-7f787b8f48-9lc74\" (UID: \"07974ea0-3f57-4174-b4bf-65639138b2f6\") " pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74"
Jul 02 14:17:16.317750 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.317723    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-qhbds"]
Jul 02 14:17:16.320420 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:17:16.320399    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8fc3257_7848_48da_9eab_8db8a9bd26c7.slice/crio-3cff15bbf3048800edee68f29f5686420e86a99095ed602f92c426bbf3cad982 WatchSource:0}: Error finding container 3cff15bbf3048800edee68f29f5686420e86a99095ed602f92c426bbf3cad982: Status 404 returned error can't find the container with id 3cff15bbf3048800edee68f29f5686420e86a99095ed602f92c426bbf3cad982
Jul 02 14:17:16.424198 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.424155    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74"
Jul 02 14:17:16.582575 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.582423    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7f787b8f48-9lc74"]
Jul 02 14:17:16.586143 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:17:16.586116    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07974ea0_3f57_4174_b4bf_65639138b2f6.slice/crio-c788162bb2862406e75d194461d063cfd217aa31d907b19d7ea70a9f93fb00da WatchSource:0}: Error finding container c788162bb2862406e75d194461d063cfd217aa31d907b19d7ea70a9f93fb00da: Status 404 returned error can't find the container with id c788162bb2862406e75d194461d063cfd217aa31d907b19d7ea70a9f93fb00da
Jul 02 14:17:16.910337 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.910243    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-cd8656b8-qhbds" event={"ID":"b8fc3257-7848-48da-9eab-8db8a9bd26c7","Type":"ContainerStarted","Data":"3cff15bbf3048800edee68f29f5686420e86a99095ed602f92c426bbf3cad982"}
Jul 02 14:17:16.911656 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:16.911625    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74" event={"ID":"07974ea0-3f57-4174-b4bf-65639138b2f6","Type":"ContainerStarted","Data":"c788162bb2862406e75d194461d063cfd217aa31d907b19d7ea70a9f93fb00da"}
Jul 02 14:17:17.917811 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:17.917740    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-cd8656b8-qhbds" event={"ID":"b8fc3257-7848-48da-9eab-8db8a9bd26c7","Type":"ContainerStarted","Data":"b45152d40008028159a4e5a74b3d9a202cc58013637a17047b146d7a7dcf34b6"}
Jul 02 14:17:17.918244 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:17.917979    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-cd8656b8-qhbds"
Jul 02 14:17:17.923722 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:17.923683    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-cd8656b8-qhbds"
Jul 02 14:17:17.978386 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:17.978334    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-cd8656b8-qhbds" podStartSLOduration=1.607714465 podStartE2EDuration="2.978312114s" totalImagesPullingTime="1.370597649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:17:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:17:17.951731259 +0000 UTC m=+99.903068905" watchObservedRunningTime="2026-07-02 14:17:17.978312114 +0000 UTC m=+99.929649761"
Jul 02 14:17:18.922671 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:18.922579    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74" event={"ID":"07974ea0-3f57-4174-b4bf-65639138b2f6","Type":"ContainerStarted","Data":"cb1f83413700cdcf262144ef0abbb625326388064573315ee548542e26cbd9ca"}
Jul 02 14:17:18.944844 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:18.944785    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74" podStartSLOduration=1.032937071 podStartE2EDuration="2.944765958s" totalImagesPullingTime="1.911828887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:17:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:17:18.943630178 +0000 UTC m=+100.894967825" watchObservedRunningTime="2026-07-02 14:17:18.944765958 +0000 UTC m=+100.896103606"
Jul 02 14:17:20.803437 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:20.803404    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7gvk9"
Jul 02 14:17:36.424296 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:36.424258    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74"
Jul 02 14:17:36.424786 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:36.424320    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74"
Jul 02 14:17:56.430073 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:56.430036    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74"
Jul 02 14:17:56.436185 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:17:56.436156    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7f787b8f48-9lc74"
Jul 02 14:20:32.711746 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:32.711701    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6"]
Jul 02 14:20:32.714935 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:32.714902    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6"
Jul 02 14:20:32.718631 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:32.718608    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 02 14:20:32.718740 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:32.718616    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-pb4jf\""
Jul 02 14:20:32.718740 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:32.718617    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 02 14:20:32.724548 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:32.724524    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6"]
Jul 02 14:20:32.818231 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:32.818194    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f1cd402-ee36-406f-8d4f-2a1462b5519c-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6\" (UID: \"0f1cd402-ee36-406f-8d4f-2a1462b5519c\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6"
Jul 02 14:20:32.818231 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:32.818233    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f1cd402-ee36-406f-8d4f-2a1462b5519c-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6\" (UID: \"0f1cd402-ee36-406f-8d4f-2a1462b5519c\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6"
Jul 02 14:20:32.818457 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:32.818269    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv5kx\" (UniqueName: \"kubernetes.io/projected/0f1cd402-ee36-406f-8d4f-2a1462b5519c-kube-api-access-zv5kx\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6\" (UID: \"0f1cd402-ee36-406f-8d4f-2a1462b5519c\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6"
Jul 02 14:20:32.919405 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:32.919357    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f1cd402-ee36-406f-8d4f-2a1462b5519c-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6\" (UID: \"0f1cd402-ee36-406f-8d4f-2a1462b5519c\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6"
Jul 02 14:20:32.919597 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:32.919415    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f1cd402-ee36-406f-8d4f-2a1462b5519c-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6\" (UID: \"0f1cd402-ee36-406f-8d4f-2a1462b5519c\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6"
Jul 02 14:20:32.919597 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:32.919457    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv5kx\" (UniqueName: \"kubernetes.io/projected/0f1cd402-ee36-406f-8d4f-2a1462b5519c-kube-api-access-zv5kx\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6\" (UID: \"0f1cd402-ee36-406f-8d4f-2a1462b5519c\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6"
Jul 02 14:20:32.919841 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:32.919818    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f1cd402-ee36-406f-8d4f-2a1462b5519c-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6\" (UID: \"0f1cd402-ee36-406f-8d4f-2a1462b5519c\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6"
Jul 02 14:20:32.919897 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:32.919819    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f1cd402-ee36-406f-8d4f-2a1462b5519c-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6\" (UID: \"0f1cd402-ee36-406f-8d4f-2a1462b5519c\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6"
Jul 02 14:20:32.928254 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:32.928228    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv5kx\" (UniqueName: \"kubernetes.io/projected/0f1cd402-ee36-406f-8d4f-2a1462b5519c-kube-api-access-zv5kx\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6\" (UID: \"0f1cd402-ee36-406f-8d4f-2a1462b5519c\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6"
Jul 02 14:20:33.024505 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:33.024475    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6"
Jul 02 14:20:33.155183 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:33.155148    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6"]
Jul 02 14:20:33.157920 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:20:33.157899    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f1cd402_ee36_406f_8d4f_2a1462b5519c.slice/crio-daca5694a2a51097d8347805dc02f1e286121acca81e77d1e8bfad18563b8717 WatchSource:0}: Error finding container daca5694a2a51097d8347805dc02f1e286121acca81e77d1e8bfad18563b8717: Status 404 returned error can't find the container with id daca5694a2a51097d8347805dc02f1e286121acca81e77d1e8bfad18563b8717
Jul 02 14:20:33.576257 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:33.576222    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6" event={"ID":"0f1cd402-ee36-406f-8d4f-2a1462b5519c","Type":"ContainerStarted","Data":"daca5694a2a51097d8347805dc02f1e286121acca81e77d1e8bfad18563b8717"}
Jul 02 14:20:38.414305 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:38.414280    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnwb5_ca909057-dd63-451d-9c05-b671ca5ae671/ovn-acl-logging/0.log"
Jul 02 14:20:38.417584 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:38.417561    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnwb5_ca909057-dd63-451d-9c05-b671ca5ae671/ovn-acl-logging/0.log"
Jul 02 14:20:38.420067 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:38.420048    2582 kubelet.go:1648] "Image garbage collection succeeded"
Jul 02 14:20:41.605912 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:41.605827    2582 generic.go:356] "Generic (PLEG): container finished" podID="0f1cd402-ee36-406f-8d4f-2a1462b5519c" containerID="b8a043ac68ab9d73044bdbe5e71b43f1e570d2e09d703b6daca5aa670505b7a0" exitCode=0
Jul 02 14:20:41.606320 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:41.605922    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6" event={"ID":"0f1cd402-ee36-406f-8d4f-2a1462b5519c","Type":"ContainerDied","Data":"b8a043ac68ab9d73044bdbe5e71b43f1e570d2e09d703b6daca5aa670505b7a0"}
Jul 02 14:20:41.606867 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:41.606851    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 14:20:45.621031 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:45.620930    2582 generic.go:356] "Generic (PLEG): container finished" podID="0f1cd402-ee36-406f-8d4f-2a1462b5519c" containerID="285d30ac525d1bdda552e3de6d1fc433d41586d5fa3aedecc56b70d7a182c4ac" exitCode=0
Jul 02 14:20:45.621031 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:45.620998    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6" event={"ID":"0f1cd402-ee36-406f-8d4f-2a1462b5519c","Type":"ContainerDied","Data":"285d30ac525d1bdda552e3de6d1fc433d41586d5fa3aedecc56b70d7a182c4ac"}
Jul 02 14:20:55.660977 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:55.660924    2582 generic.go:356] "Generic (PLEG): container finished" podID="0f1cd402-ee36-406f-8d4f-2a1462b5519c" containerID="ee1bbfc87e18e4faeeabeb7975aa66b31bc3af57504e42b6f1b37bb521365af5" exitCode=0
Jul 02 14:20:55.661416 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:55.661005    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6" event={"ID":"0f1cd402-ee36-406f-8d4f-2a1462b5519c","Type":"ContainerDied","Data":"ee1bbfc87e18e4faeeabeb7975aa66b31bc3af57504e42b6f1b37bb521365af5"}
Jul 02 14:20:56.787088 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:56.787065    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6"
Jul 02 14:20:56.902573 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:56.902541    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f1cd402-ee36-406f-8d4f-2a1462b5519c-bundle\") pod \"0f1cd402-ee36-406f-8d4f-2a1462b5519c\" (UID: \"0f1cd402-ee36-406f-8d4f-2a1462b5519c\") "
Jul 02 14:20:56.902724 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:56.902611    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f1cd402-ee36-406f-8d4f-2a1462b5519c-util\") pod \"0f1cd402-ee36-406f-8d4f-2a1462b5519c\" (UID: \"0f1cd402-ee36-406f-8d4f-2a1462b5519c\") "
Jul 02 14:20:56.902724 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:56.902643    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zv5kx\" (UniqueName: \"kubernetes.io/projected/0f1cd402-ee36-406f-8d4f-2a1462b5519c-kube-api-access-zv5kx\") pod \"0f1cd402-ee36-406f-8d4f-2a1462b5519c\" (UID: \"0f1cd402-ee36-406f-8d4f-2a1462b5519c\") "
Jul 02 14:20:56.903256 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:56.903229    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f1cd402-ee36-406f-8d4f-2a1462b5519c-bundle" (OuterVolumeSpecName: "bundle") pod "0f1cd402-ee36-406f-8d4f-2a1462b5519c" (UID: "0f1cd402-ee36-406f-8d4f-2a1462b5519c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 14:20:56.905016 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:56.904988    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f1cd402-ee36-406f-8d4f-2a1462b5519c-kube-api-access-zv5kx" (OuterVolumeSpecName: "kube-api-access-zv5kx") pod "0f1cd402-ee36-406f-8d4f-2a1462b5519c" (UID: "0f1cd402-ee36-406f-8d4f-2a1462b5519c"). InnerVolumeSpecName "kube-api-access-zv5kx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:20:56.906794 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:56.906776    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f1cd402-ee36-406f-8d4f-2a1462b5519c-util" (OuterVolumeSpecName: "util") pod "0f1cd402-ee36-406f-8d4f-2a1462b5519c" (UID: "0f1cd402-ee36-406f-8d4f-2a1462b5519c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 14:20:57.004090 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:57.004056    2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f1cd402-ee36-406f-8d4f-2a1462b5519c-bundle\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:20:57.004090 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:57.004084    2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f1cd402-ee36-406f-8d4f-2a1462b5519c-util\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:20:57.004263 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:57.004097    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zv5kx\" (UniqueName: \"kubernetes.io/projected/0f1cd402-ee36-406f-8d4f-2a1462b5519c-kube-api-access-zv5kx\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:20:57.669920 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:57.669883    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6" event={"ID":"0f1cd402-ee36-406f-8d4f-2a1462b5519c","Type":"ContainerDied","Data":"daca5694a2a51097d8347805dc02f1e286121acca81e77d1e8bfad18563b8717"}
Jul 02 14:20:57.669920 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:57.669924    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hzqp6"
Jul 02 14:20:57.670158 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:20:57.669939    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="daca5694a2a51097d8347805dc02f1e286121acca81e77d1e8bfad18563b8717"
Jul 02 14:21:00.783064 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:00.783027    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4r5dc"]
Jul 02 14:21:00.783521 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:00.783371    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f1cd402-ee36-406f-8d4f-2a1462b5519c" containerName="util"
Jul 02 14:21:00.783521 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:00.783390    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f1cd402-ee36-406f-8d4f-2a1462b5519c" containerName="util"
Jul 02 14:21:00.783521 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:00.783401    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f1cd402-ee36-406f-8d4f-2a1462b5519c" containerName="pull"
Jul 02 14:21:00.783521 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:00.783408    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f1cd402-ee36-406f-8d4f-2a1462b5519c" containerName="pull"
Jul 02 14:21:00.783521 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:00.783423    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f1cd402-ee36-406f-8d4f-2a1462b5519c" containerName="extract"
Jul 02 14:21:00.783521 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:00.783431    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f1cd402-ee36-406f-8d4f-2a1462b5519c" containerName="extract"
Jul 02 14:21:00.783521 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:00.783491    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f1cd402-ee36-406f-8d4f-2a1462b5519c" containerName="extract"
Jul 02 14:21:00.810002 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:00.809970    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4r5dc"]
Jul 02 14:21:00.810157 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:00.810077    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4r5dc"
Jul 02 14:21:00.812937 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:00.812914    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Jul 02 14:21:00.813094 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:00.813041    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Jul 02 14:21:00.813533 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:00.813514    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-7s7fv\""
Jul 02 14:21:00.813618 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:00.813553    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Jul 02 14:21:00.933835 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:00.933805    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/b6e2b4f7-f2e6-4a13-b233-ad4027b5cee4-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-4r5dc\" (UID: \"b6e2b4f7-f2e6-4a13-b233-ad4027b5cee4\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4r5dc"
Jul 02 14:21:00.933835 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:00.933838    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clh7f\" (UniqueName: \"kubernetes.io/projected/b6e2b4f7-f2e6-4a13-b233-ad4027b5cee4-kube-api-access-clh7f\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-4r5dc\" (UID: \"b6e2b4f7-f2e6-4a13-b233-ad4027b5cee4\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4r5dc"
Jul 02 14:21:01.035178 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:01.035105    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/b6e2b4f7-f2e6-4a13-b233-ad4027b5cee4-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-4r5dc\" (UID: \"b6e2b4f7-f2e6-4a13-b233-ad4027b5cee4\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4r5dc"
Jul 02 14:21:01.035178 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:01.035143    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clh7f\" (UniqueName: \"kubernetes.io/projected/b6e2b4f7-f2e6-4a13-b233-ad4027b5cee4-kube-api-access-clh7f\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-4r5dc\" (UID: \"b6e2b4f7-f2e6-4a13-b233-ad4027b5cee4\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4r5dc"
Jul 02 14:21:01.037712 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:01.037687    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/b6e2b4f7-f2e6-4a13-b233-ad4027b5cee4-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-4r5dc\" (UID: \"b6e2b4f7-f2e6-4a13-b233-ad4027b5cee4\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4r5dc"
Jul 02 14:21:01.063392 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:01.063362    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clh7f\" (UniqueName: \"kubernetes.io/projected/b6e2b4f7-f2e6-4a13-b233-ad4027b5cee4-kube-api-access-clh7f\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-4r5dc\" (UID: \"b6e2b4f7-f2e6-4a13-b233-ad4027b5cee4\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4r5dc"
Jul 02 14:21:01.120124 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:01.120093    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4r5dc"
Jul 02 14:21:01.277843 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:01.277815    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4r5dc"]
Jul 02 14:21:01.280656 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:21:01.280623    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6e2b4f7_f2e6_4a13_b233_ad4027b5cee4.slice/crio-9908fc38c4502555d450ad078de1f5b940b6ec120bfbefe08fe2315c161366d4 WatchSource:0}: Error finding container 9908fc38c4502555d450ad078de1f5b940b6ec120bfbefe08fe2315c161366d4: Status 404 returned error can't find the container with id 9908fc38c4502555d450ad078de1f5b940b6ec120bfbefe08fe2315c161366d4
Jul 02 14:21:01.684936 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:01.684856    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4r5dc" event={"ID":"b6e2b4f7-f2e6-4a13-b233-ad4027b5cee4","Type":"ContainerStarted","Data":"9908fc38c4502555d450ad078de1f5b940b6ec120bfbefe08fe2315c161366d4"}
Jul 02 14:21:05.701540 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:05.701502    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4r5dc" event={"ID":"b6e2b4f7-f2e6-4a13-b233-ad4027b5cee4","Type":"ContainerStarted","Data":"5f6d46f8661e48a49f7b0f4f1fd2f4d9ddb8ef8b8dd477f71de9cbc1d730512d"}
Jul 02 14:21:05.702208 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:05.701647    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4r5dc"
Jul 02 14:21:05.730415 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:05.730363    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4r5dc" podStartSLOduration=2.147643587 podStartE2EDuration="5.730348327s" totalImagesPullingTime="3.58270474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:21:05.729442772 +0000 UTC m=+327.680780419" watchObservedRunningTime="2026-07-02 14:21:05.730348327 +0000 UTC m=+327.681685972"
Jul 02 14:21:05.952388 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:05.952313    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-ln9l2"]
Jul 02 14:21:05.976679 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:05.976654    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-ln9l2"]
Jul 02 14:21:05.976809 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:05.976751    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-ln9l2"
Jul 02 14:21:05.979834 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:05.979812    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Jul 02 14:21:05.980043 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:05.979819    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-mhpdj\""
Jul 02 14:21:05.980192 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:05.980177    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Jul 02 14:21:06.075215 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:06.075182    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/4aafc810-655c-4222-a105-254ca1f54cdc-certificates\") pod \"keda-metrics-apiserver-559f75f947-ln9l2\" (UID: \"4aafc810-655c-4222-a105-254ca1f54cdc\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-ln9l2"
Jul 02 14:21:06.075215 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:06.075219    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/4aafc810-655c-4222-a105-254ca1f54cdc-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-ln9l2\" (UID: \"4aafc810-655c-4222-a105-254ca1f54cdc\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-ln9l2"
Jul 02 14:21:06.075408 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:06.075238    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flqzr\" (UniqueName: \"kubernetes.io/projected/4aafc810-655c-4222-a105-254ca1f54cdc-kube-api-access-flqzr\") pod \"keda-metrics-apiserver-559f75f947-ln9l2\" (UID: \"4aafc810-655c-4222-a105-254ca1f54cdc\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-ln9l2"
Jul 02 14:21:06.176477 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:06.176440    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flqzr\" (UniqueName: \"kubernetes.io/projected/4aafc810-655c-4222-a105-254ca1f54cdc-kube-api-access-flqzr\") pod \"keda-metrics-apiserver-559f75f947-ln9l2\" (UID: \"4aafc810-655c-4222-a105-254ca1f54cdc\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-ln9l2"
Jul 02 14:21:06.176638 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:06.176533    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/4aafc810-655c-4222-a105-254ca1f54cdc-certificates\") pod \"keda-metrics-apiserver-559f75f947-ln9l2\" (UID: \"4aafc810-655c-4222-a105-254ca1f54cdc\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-ln9l2"
Jul 02 14:21:06.176638 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:06.176590    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/4aafc810-655c-4222-a105-254ca1f54cdc-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-ln9l2\" (UID: \"4aafc810-655c-4222-a105-254ca1f54cdc\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-ln9l2"
Jul 02 14:21:06.176736 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:21:06.176676    2582 secret.go:281] references non-existent secret key: tls.crt
Jul 02 14:21:06.176736 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:21:06.176700    2582 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Jul 02 14:21:06.176736 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:21:06.176720    2582 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-559f75f947-ln9l2: references non-existent secret key: tls.crt
Jul 02 14:21:06.176851 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:21:06.176780    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4aafc810-655c-4222-a105-254ca1f54cdc-certificates podName:4aafc810-655c-4222-a105-254ca1f54cdc nodeName:}" failed. No retries permitted until 2026-07-02 14:21:06.6767646 +0000 UTC m=+328.628102223 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/4aafc810-655c-4222-a105-254ca1f54cdc-certificates") pod "keda-metrics-apiserver-559f75f947-ln9l2" (UID: "4aafc810-655c-4222-a105-254ca1f54cdc") : references non-existent secret key: tls.crt
Jul 02 14:21:06.176990 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:06.176945    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/4aafc810-655c-4222-a105-254ca1f54cdc-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-ln9l2\" (UID: \"4aafc810-655c-4222-a105-254ca1f54cdc\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-ln9l2"
Jul 02 14:21:06.188203 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:06.188171    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-flqzr\" (UniqueName: \"kubernetes.io/projected/4aafc810-655c-4222-a105-254ca1f54cdc-kube-api-access-flqzr\") pod \"keda-metrics-apiserver-559f75f947-ln9l2\" (UID: \"4aafc810-655c-4222-a105-254ca1f54cdc\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-ln9l2"
Jul 02 14:21:06.539318 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:06.539286    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-gcxkk"]
Jul 02 14:21:06.563291 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:06.563264    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-gcxkk"]
Jul 02 14:21:06.563430 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:06.563376    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-gcxkk"
Jul 02 14:21:06.566318 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:06.566301    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Jul 02 14:21:06.680719 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:06.680691    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pkwt\" (UniqueName: \"kubernetes.io/projected/51019664-f511-4879-86ee-8b953e1116bf-kube-api-access-6pkwt\") pod \"keda-admission-5fbd5c4644-gcxkk\" (UID: \"51019664-f511-4879-86ee-8b953e1116bf\") " pod="openshift-keda/keda-admission-5fbd5c4644-gcxkk"
Jul 02 14:21:06.680904 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:06.680763    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/51019664-f511-4879-86ee-8b953e1116bf-certificates\") pod \"keda-admission-5fbd5c4644-gcxkk\" (UID: \"51019664-f511-4879-86ee-8b953e1116bf\") " pod="openshift-keda/keda-admission-5fbd5c4644-gcxkk"
Jul 02 14:21:06.680904 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:06.680803    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/4aafc810-655c-4222-a105-254ca1f54cdc-certificates\") pod \"keda-metrics-apiserver-559f75f947-ln9l2\" (UID: \"4aafc810-655c-4222-a105-254ca1f54cdc\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-ln9l2"
Jul 02 14:21:06.683435 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:06.683408    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/4aafc810-655c-4222-a105-254ca1f54cdc-certificates\") pod \"keda-metrics-apiserver-559f75f947-ln9l2\" (UID: \"4aafc810-655c-4222-a105-254ca1f54cdc\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-ln9l2"
Jul 02 14:21:06.781998 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:06.781941    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/51019664-f511-4879-86ee-8b953e1116bf-certificates\") pod \"keda-admission-5fbd5c4644-gcxkk\" (UID: \"51019664-f511-4879-86ee-8b953e1116bf\") " pod="openshift-keda/keda-admission-5fbd5c4644-gcxkk"
Jul 02 14:21:06.782377 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:06.782033    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pkwt\" (UniqueName: \"kubernetes.io/projected/51019664-f511-4879-86ee-8b953e1116bf-kube-api-access-6pkwt\") pod \"keda-admission-5fbd5c4644-gcxkk\" (UID: \"51019664-f511-4879-86ee-8b953e1116bf\") " pod="openshift-keda/keda-admission-5fbd5c4644-gcxkk"
Jul 02 14:21:06.784533 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:06.784507    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/51019664-f511-4879-86ee-8b953e1116bf-certificates\") pod \"keda-admission-5fbd5c4644-gcxkk\" (UID: \"51019664-f511-4879-86ee-8b953e1116bf\") " pod="openshift-keda/keda-admission-5fbd5c4644-gcxkk"
Jul 02 14:21:06.791609 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:06.791548    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pkwt\" (UniqueName: \"kubernetes.io/projected/51019664-f511-4879-86ee-8b953e1116bf-kube-api-access-6pkwt\") pod \"keda-admission-5fbd5c4644-gcxkk\" (UID: \"51019664-f511-4879-86ee-8b953e1116bf\") " pod="openshift-keda/keda-admission-5fbd5c4644-gcxkk"
Jul 02 14:21:06.873493 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:06.873460    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-gcxkk"
Jul 02 14:21:06.886742 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:06.886706    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-ln9l2"
Jul 02 14:21:07.043744 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:07.043692    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-gcxkk"]
Jul 02 14:21:07.046882 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:21:07.046854    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51019664_f511_4879_86ee_8b953e1116bf.slice/crio-108cd7e6d8c29f9916970827e940d77001737f66f1fa0f6a0c2898883b811e3b WatchSource:0}: Error finding container 108cd7e6d8c29f9916970827e940d77001737f66f1fa0f6a0c2898883b811e3b: Status 404 returned error can't find the container with id 108cd7e6d8c29f9916970827e940d77001737f66f1fa0f6a0c2898883b811e3b
Jul 02 14:21:07.065780 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:07.065755    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-ln9l2"]
Jul 02 14:21:07.067895 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:21:07.067865    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aafc810_655c_4222_a105_254ca1f54cdc.slice/crio-1a07f57bc2e42d3222716e2da924349b4b8c0a300def0b5f6bed019b254e19d4 WatchSource:0}: Error finding container 1a07f57bc2e42d3222716e2da924349b4b8c0a300def0b5f6bed019b254e19d4: Status 404 returned error can't find the container with id 1a07f57bc2e42d3222716e2da924349b4b8c0a300def0b5f6bed019b254e19d4
Jul 02 14:21:07.710980 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:07.710922    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-ln9l2" event={"ID":"4aafc810-655c-4222-a105-254ca1f54cdc","Type":"ContainerStarted","Data":"1a07f57bc2e42d3222716e2da924349b4b8c0a300def0b5f6bed019b254e19d4"}
Jul 02 14:21:07.712043 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:07.712015    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-gcxkk" event={"ID":"51019664-f511-4879-86ee-8b953e1116bf","Type":"ContainerStarted","Data":"108cd7e6d8c29f9916970827e940d77001737f66f1fa0f6a0c2898883b811e3b"}
Jul 02 14:21:09.723196 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:09.723153    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-gcxkk" event={"ID":"51019664-f511-4879-86ee-8b953e1116bf","Type":"ContainerStarted","Data":"517f2ce20a123436b623b42d3b8ecf48793d424cf14e25968c71446d18d8f785"}
Jul 02 14:21:09.723632 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:09.723249    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-5fbd5c4644-gcxkk"
Jul 02 14:21:09.743288 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:09.743231    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-5fbd5c4644-gcxkk" podStartSLOduration=1.41048927 podStartE2EDuration="3.743214979s" totalImagesPullingTime="2.332725709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:21:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:21:09.742316416 +0000 UTC m=+331.693654078" watchObservedRunningTime="2026-07-02 14:21:09.743214979 +0000 UTC m=+331.694552625"
Jul 02 14:21:10.728036 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:10.727999    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-ln9l2" event={"ID":"4aafc810-655c-4222-a105-254ca1f54cdc","Type":"ContainerStarted","Data":"395e499a2d74f80242a817e07f2d37b0955eca93c0082fd6646c0cdf588fde51"}
Jul 02 14:21:10.747816 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:10.747762    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-559f75f947-ln9l2" podStartSLOduration=2.309806033 podStartE2EDuration="5.747747339s" totalImagesPullingTime="3.437941306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:21:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:21:10.745993381 +0000 UTC m=+332.697331031" watchObservedRunningTime="2026-07-02 14:21:10.747747339 +0000 UTC m=+332.699084984"
Jul 02 14:21:11.732309 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:11.732276    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-ln9l2"
Jul 02 14:21:22.740217 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:22.740184    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-ln9l2"
Jul 02 14:21:26.708112 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:26.708081    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4r5dc"
Jul 02 14:21:30.730528 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:21:30.730434    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-5fbd5c4644-gcxkk"
Jul 02 14:22:15.083424 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:22:15.083387    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-64568bcd49-5dv4p"]
Jul 02 14:22:15.085578 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:22:15.085561    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-5dv4p"
Jul 02 14:22:15.092600 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:22:15.092573    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 02 14:22:15.093508 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:22:15.093495    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-t7wgd\""
Jul 02 14:22:15.093554 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:22:15.093522    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Jul 02 14:22:15.095435 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:22:15.095422    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 02 14:22:15.114260 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:22:15.114232    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-5dv4p"]
Jul 02 14:22:15.169122 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:22:15.169085    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ml2ws\" (UniqueName: \"kubernetes.io/projected/e0a3971e-4c32-4087-9dd6-73c316a9e1ec-kube-api-access-ml2ws\") pod \"seaweedfs-64568bcd49-5dv4p\" (UID: \"e0a3971e-4c32-4087-9dd6-73c316a9e1ec\") " pod="kserve/seaweedfs-64568bcd49-5dv4p"
Jul 02 14:22:15.169293 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:22:15.169145    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e0a3971e-4c32-4087-9dd6-73c316a9e1ec-data\") pod \"seaweedfs-64568bcd49-5dv4p\" (UID: \"e0a3971e-4c32-4087-9dd6-73c316a9e1ec\") " pod="kserve/seaweedfs-64568bcd49-5dv4p"
Jul 02 14:22:15.270231 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:22:15.270192    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e0a3971e-4c32-4087-9dd6-73c316a9e1ec-data\") pod \"seaweedfs-64568bcd49-5dv4p\" (UID: \"e0a3971e-4c32-4087-9dd6-73c316a9e1ec\") " pod="kserve/seaweedfs-64568bcd49-5dv4p"
Jul 02 14:22:15.270402 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:22:15.270261    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ml2ws\" (UniqueName: \"kubernetes.io/projected/e0a3971e-4c32-4087-9dd6-73c316a9e1ec-kube-api-access-ml2ws\") pod \"seaweedfs-64568bcd49-5dv4p\" (UID: \"e0a3971e-4c32-4087-9dd6-73c316a9e1ec\") " pod="kserve/seaweedfs-64568bcd49-5dv4p"
Jul 02 14:22:15.270594 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:22:15.270560    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e0a3971e-4c32-4087-9dd6-73c316a9e1ec-data\") pod \"seaweedfs-64568bcd49-5dv4p\" (UID: \"e0a3971e-4c32-4087-9dd6-73c316a9e1ec\") " pod="kserve/seaweedfs-64568bcd49-5dv4p"
Jul 02 14:22:15.288500 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:22:15.288476    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ml2ws\" (UniqueName: \"kubernetes.io/projected/e0a3971e-4c32-4087-9dd6-73c316a9e1ec-kube-api-access-ml2ws\") pod \"seaweedfs-64568bcd49-5dv4p\" (UID: \"e0a3971e-4c32-4087-9dd6-73c316a9e1ec\") " pod="kserve/seaweedfs-64568bcd49-5dv4p"
Jul 02 14:22:15.394470 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:22:15.394391    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-5dv4p"
Jul 02 14:22:15.531103 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:22:15.531073    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-5dv4p"]
Jul 02 14:22:15.534290 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:22:15.534265    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0a3971e_4c32_4087_9dd6_73c316a9e1ec.slice/crio-2125baee4a4eac219f6e44d0edcc1494dc5ce933ae77d91fc900eba09f23e29c WatchSource:0}: Error finding container 2125baee4a4eac219f6e44d0edcc1494dc5ce933ae77d91fc900eba09f23e29c: Status 404 returned error can't find the container with id 2125baee4a4eac219f6e44d0edcc1494dc5ce933ae77d91fc900eba09f23e29c
Jul 02 14:22:15.968177 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:22:15.968142    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-5dv4p" event={"ID":"e0a3971e-4c32-4087-9dd6-73c316a9e1ec","Type":"ContainerStarted","Data":"2125baee4a4eac219f6e44d0edcc1494dc5ce933ae77d91fc900eba09f23e29c"}
Jul 02 14:22:19.986468 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:22:19.986420    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-5dv4p" event={"ID":"e0a3971e-4c32-4087-9dd6-73c316a9e1ec","Type":"ContainerStarted","Data":"6ca4fb2796a0169f192de797c2a6a114e5651e277825420ae18a0b8005a71923"}
Jul 02 14:22:19.986931 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:22:19.986556    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-64568bcd49-5dv4p"
Jul 02 14:22:20.009706 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:22:20.009653    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-64568bcd49-5dv4p" podStartSLOduration=1.451805265 podStartE2EDuration="5.00963565s" totalImagesPullingTime="3.557830385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:22:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:22:20.008988146 +0000 UTC m=+401.960325786" watchObservedRunningTime="2026-07-02 14:22:20.00963565 +0000 UTC m=+401.960973295"
Jul 02 14:22:25.992706 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:22:25.992677    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-64568bcd49-5dv4p"
Jul 02 14:23:26.031198 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:26.031120    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-fd65d7d6-7tpvc"]
Jul 02 14:23:26.034161 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:26.034142    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-7tpvc"
Jul 02 14:23:26.037019 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:26.036990    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\""
Jul 02 14:23:26.037115 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:26.037018    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-nxxn7\""
Jul 02 14:23:26.044171 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:26.044150    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-7tpvc"]
Jul 02 14:23:26.047652 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:26.047626    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-84b54b7f97-fffqj"]
Jul 02 14:23:26.050505 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:26.050475    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-fffqj"
Jul 02 14:23:26.053530 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:26.053508    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-bqr45\""
Jul 02 14:23:26.053622 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:26.053600    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Jul 02 14:23:26.061965 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:26.061937    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-fffqj"]
Jul 02 14:23:26.100148 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:26.100119    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/52a07e67-58cb-48fe-8dfe-dce1949d9c2f-cert\") pod \"odh-model-controller-84b54b7f97-fffqj\" (UID: \"52a07e67-58cb-48fe-8dfe-dce1949d9c2f\") " pod="kserve/odh-model-controller-84b54b7f97-fffqj"
Jul 02 14:23:26.100318 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:26.100160    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41b77d29-74e2-4d81-9f5a-a8c31be03d7e-tls-certs\") pod \"model-serving-api-fd65d7d6-7tpvc\" (UID: \"41b77d29-74e2-4d81-9f5a-a8c31be03d7e\") " pod="kserve/model-serving-api-fd65d7d6-7tpvc"
Jul 02 14:23:26.100318 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:26.100195    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5grx5\" (UniqueName: \"kubernetes.io/projected/41b77d29-74e2-4d81-9f5a-a8c31be03d7e-kube-api-access-5grx5\") pod \"model-serving-api-fd65d7d6-7tpvc\" (UID: \"41b77d29-74e2-4d81-9f5a-a8c31be03d7e\") " pod="kserve/model-serving-api-fd65d7d6-7tpvc"
Jul 02 14:23:26.100318 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:26.100221    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2d4n\" (UniqueName: \"kubernetes.io/projected/52a07e67-58cb-48fe-8dfe-dce1949d9c2f-kube-api-access-t2d4n\") pod \"odh-model-controller-84b54b7f97-fffqj\" (UID: \"52a07e67-58cb-48fe-8dfe-dce1949d9c2f\") " pod="kserve/odh-model-controller-84b54b7f97-fffqj"
Jul 02 14:23:26.200869 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:26.200818    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/52a07e67-58cb-48fe-8dfe-dce1949d9c2f-cert\") pod \"odh-model-controller-84b54b7f97-fffqj\" (UID: \"52a07e67-58cb-48fe-8dfe-dce1949d9c2f\") " pod="kserve/odh-model-controller-84b54b7f97-fffqj"
Jul 02 14:23:26.200869 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:26.200875    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41b77d29-74e2-4d81-9f5a-a8c31be03d7e-tls-certs\") pod \"model-serving-api-fd65d7d6-7tpvc\" (UID: \"41b77d29-74e2-4d81-9f5a-a8c31be03d7e\") " pod="kserve/model-serving-api-fd65d7d6-7tpvc"
Jul 02 14:23:26.201097 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:26.200904    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5grx5\" (UniqueName: \"kubernetes.io/projected/41b77d29-74e2-4d81-9f5a-a8c31be03d7e-kube-api-access-5grx5\") pod \"model-serving-api-fd65d7d6-7tpvc\" (UID: \"41b77d29-74e2-4d81-9f5a-a8c31be03d7e\") " pod="kserve/model-serving-api-fd65d7d6-7tpvc"
Jul 02 14:23:26.201097 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:26.200927    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2d4n\" (UniqueName: \"kubernetes.io/projected/52a07e67-58cb-48fe-8dfe-dce1949d9c2f-kube-api-access-t2d4n\") pod \"odh-model-controller-84b54b7f97-fffqj\" (UID: \"52a07e67-58cb-48fe-8dfe-dce1949d9c2f\") " pod="kserve/odh-model-controller-84b54b7f97-fffqj"
Jul 02 14:23:26.201097 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:23:26.201001    2582 secret.go:189] Couldn't get secret kserve/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 02 14:23:26.201097 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:23:26.201069    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52a07e67-58cb-48fe-8dfe-dce1949d9c2f-cert podName:52a07e67-58cb-48fe-8dfe-dce1949d9c2f nodeName:}" failed. No retries permitted until 2026-07-02 14:23:26.701051938 +0000 UTC m=+468.652389566 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/52a07e67-58cb-48fe-8dfe-dce1949d9c2f-cert") pod "odh-model-controller-84b54b7f97-fffqj" (UID: "52a07e67-58cb-48fe-8dfe-dce1949d9c2f") : secret "odh-model-controller-webhook-cert" not found
Jul 02 14:23:26.203477 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:26.203458    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41b77d29-74e2-4d81-9f5a-a8c31be03d7e-tls-certs\") pod \"model-serving-api-fd65d7d6-7tpvc\" (UID: \"41b77d29-74e2-4d81-9f5a-a8c31be03d7e\") " pod="kserve/model-serving-api-fd65d7d6-7tpvc"
Jul 02 14:23:26.211871 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:26.211850    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5grx5\" (UniqueName: \"kubernetes.io/projected/41b77d29-74e2-4d81-9f5a-a8c31be03d7e-kube-api-access-5grx5\") pod \"model-serving-api-fd65d7d6-7tpvc\" (UID: \"41b77d29-74e2-4d81-9f5a-a8c31be03d7e\") " pod="kserve/model-serving-api-fd65d7d6-7tpvc"
Jul 02 14:23:26.212770 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:26.212746    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2d4n\" (UniqueName: \"kubernetes.io/projected/52a07e67-58cb-48fe-8dfe-dce1949d9c2f-kube-api-access-t2d4n\") pod \"odh-model-controller-84b54b7f97-fffqj\" (UID: \"52a07e67-58cb-48fe-8dfe-dce1949d9c2f\") " pod="kserve/odh-model-controller-84b54b7f97-fffqj"
Jul 02 14:23:26.344407 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:26.344306    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-7tpvc"
Jul 02 14:23:26.474904 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:23:26.474876    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41b77d29_74e2_4d81_9f5a_a8c31be03d7e.slice/crio-f9468abcd5867cb47dd9168365becdf8b0849f69c060e07da4910365432c044f WatchSource:0}: Error finding container f9468abcd5867cb47dd9168365becdf8b0849f69c060e07da4910365432c044f: Status 404 returned error can't find the container with id f9468abcd5867cb47dd9168365becdf8b0849f69c060e07da4910365432c044f
Jul 02 14:23:26.475861 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:26.475839    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-7tpvc"]
Jul 02 14:23:26.704831 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:26.704725    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/52a07e67-58cb-48fe-8dfe-dce1949d9c2f-cert\") pod \"odh-model-controller-84b54b7f97-fffqj\" (UID: \"52a07e67-58cb-48fe-8dfe-dce1949d9c2f\") " pod="kserve/odh-model-controller-84b54b7f97-fffqj"
Jul 02 14:23:26.707475 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:26.707449    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/52a07e67-58cb-48fe-8dfe-dce1949d9c2f-cert\") pod \"odh-model-controller-84b54b7f97-fffqj\" (UID: \"52a07e67-58cb-48fe-8dfe-dce1949d9c2f\") " pod="kserve/odh-model-controller-84b54b7f97-fffqj"
Jul 02 14:23:26.959604 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:26.959518    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-fffqj"
Jul 02 14:23:27.086561 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:27.086528    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-fffqj"]
Jul 02 14:23:27.089357 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:23:27.089331    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52a07e67_58cb_48fe_8dfe_dce1949d9c2f.slice/crio-1992ba0ee488f0b0898ab850ed8d64eed3b5f82996aac47ea9c44732924cc88a WatchSource:0}: Error finding container 1992ba0ee488f0b0898ab850ed8d64eed3b5f82996aac47ea9c44732924cc88a: Status 404 returned error can't find the container with id 1992ba0ee488f0b0898ab850ed8d64eed3b5f82996aac47ea9c44732924cc88a
Jul 02 14:23:27.242974 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:27.242922    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-fffqj" event={"ID":"52a07e67-58cb-48fe-8dfe-dce1949d9c2f","Type":"ContainerStarted","Data":"1992ba0ee488f0b0898ab850ed8d64eed3b5f82996aac47ea9c44732924cc88a"}
Jul 02 14:23:27.245147 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:27.245111    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-7tpvc" event={"ID":"41b77d29-74e2-4d81-9f5a-a8c31be03d7e","Type":"ContainerStarted","Data":"f9468abcd5867cb47dd9168365becdf8b0849f69c060e07da4910365432c044f"}
Jul 02 14:23:31.265789 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:31.265743    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-7tpvc" event={"ID":"41b77d29-74e2-4d81-9f5a-a8c31be03d7e","Type":"ContainerStarted","Data":"7a6205b729e88bc730b5b3f70b6fa9d7dd1ff020a88ecf00e4940cdb9a66700f"}
Jul 02 14:23:31.266254 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:31.265812    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-fd65d7d6-7tpvc"
Jul 02 14:23:31.267123 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:31.267094    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-fffqj" event={"ID":"52a07e67-58cb-48fe-8dfe-dce1949d9c2f","Type":"ContainerStarted","Data":"80876806bef07f41396a9f231a1fa160e9e6c37567dfea8f04868dc57b555fc6"}
Jul 02 14:23:31.267230 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:31.267153    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-84b54b7f97-fffqj"
Jul 02 14:23:31.300558 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:31.300505    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-84b54b7f97-fffqj" podStartSLOduration=1.974758552 podStartE2EDuration="5.300488603s" totalImagesPullingTime="3.325730051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:23:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:23:31.29997409 +0000 UTC m=+473.251311728" watchObservedRunningTime="2026-07-02 14:23:31.300488603 +0000 UTC m=+473.251826249"
Jul 02 14:23:31.301411 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:31.301372    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-fd65d7d6-7tpvc" podStartSLOduration=1.365682474 podStartE2EDuration="5.301359879s" totalImagesPullingTime="3.935677405s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:23:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:23:31.284284269 +0000 UTC m=+473.235621916" watchObservedRunningTime="2026-07-02 14:23:31.301359879 +0000 UTC m=+473.252697528"
Jul 02 14:23:42.272942 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:42.272909    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-84b54b7f97-fffqj"
Jul 02 14:23:42.274740 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:23:42.274720    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-fd65d7d6-7tpvc"
Jul 02 14:25:03.168945 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:03.168908    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-5m7dt"]
Jul 02 14:25:03.172350 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:03.172327    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-5m7dt"
Jul 02 14:25:03.175142 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:03.175119    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom-artifact\""
Jul 02 14:25:03.180642 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:03.180615    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-5m7dt"]
Jul 02 14:25:03.264084 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:03.264051    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/37756f20-90bc-41f6-b5e9-9833355df7a0-data\") pod \"seaweedfs-tls-custom-6644bf5d6c-5m7dt\" (UID: \"37756f20-90bc-41f6-b5e9-9833355df7a0\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-5m7dt"
Jul 02 14:25:03.264084 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:03.264087    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsq7p\" (UniqueName: \"kubernetes.io/projected/37756f20-90bc-41f6-b5e9-9833355df7a0-kube-api-access-dsq7p\") pod \"seaweedfs-tls-custom-6644bf5d6c-5m7dt\" (UID: \"37756f20-90bc-41f6-b5e9-9833355df7a0\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-5m7dt"
Jul 02 14:25:03.364650 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:03.364614    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/37756f20-90bc-41f6-b5e9-9833355df7a0-data\") pod \"seaweedfs-tls-custom-6644bf5d6c-5m7dt\" (UID: \"37756f20-90bc-41f6-b5e9-9833355df7a0\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-5m7dt"
Jul 02 14:25:03.364650 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:03.364651    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsq7p\" (UniqueName: \"kubernetes.io/projected/37756f20-90bc-41f6-b5e9-9833355df7a0-kube-api-access-dsq7p\") pod \"seaweedfs-tls-custom-6644bf5d6c-5m7dt\" (UID: \"37756f20-90bc-41f6-b5e9-9833355df7a0\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-5m7dt"
Jul 02 14:25:03.365038 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:03.365018    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/37756f20-90bc-41f6-b5e9-9833355df7a0-data\") pod \"seaweedfs-tls-custom-6644bf5d6c-5m7dt\" (UID: \"37756f20-90bc-41f6-b5e9-9833355df7a0\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-5m7dt"
Jul 02 14:25:03.373802 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:03.373779    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsq7p\" (UniqueName: \"kubernetes.io/projected/37756f20-90bc-41f6-b5e9-9833355df7a0-kube-api-access-dsq7p\") pod \"seaweedfs-tls-custom-6644bf5d6c-5m7dt\" (UID: \"37756f20-90bc-41f6-b5e9-9833355df7a0\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-5m7dt"
Jul 02 14:25:03.481767 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:03.481735    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-5m7dt"
Jul 02 14:25:03.609311 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:03.609286    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-5m7dt"]
Jul 02 14:25:03.611231 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:25:03.611207    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37756f20_90bc_41f6_b5e9_9833355df7a0.slice/crio-f1b3588a16a1600a87f5c5bdab85816f193b831436eaab7683dc8197153cb05f WatchSource:0}: Error finding container f1b3588a16a1600a87f5c5bdab85816f193b831436eaab7683dc8197153cb05f: Status 404 returned error can't find the container with id f1b3588a16a1600a87f5c5bdab85816f193b831436eaab7683dc8197153cb05f
Jul 02 14:25:03.633020 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:03.632985    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-5m7dt" event={"ID":"37756f20-90bc-41f6-b5e9-9833355df7a0","Type":"ContainerStarted","Data":"f1b3588a16a1600a87f5c5bdab85816f193b831436eaab7683dc8197153cb05f"}
Jul 02 14:25:04.638685 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:04.638645    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-5m7dt" event={"ID":"37756f20-90bc-41f6-b5e9-9833355df7a0","Type":"ContainerStarted","Data":"6b42bc365716b945545f7d5ec1ad8584344ccc9f63844df1415d78bed6249a2a"}
Jul 02 14:25:04.655846 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:04.655791    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-5m7dt" podStartSLOduration=1.365179571 podStartE2EDuration="1.655775987s" totalImagesPullingTime="290.596416ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:25:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:25:04.654850489 +0000 UTC m=+566.606188136" watchObservedRunningTime="2026-07-02 14:25:04.655775987 +0000 UTC m=+566.607113633"
Jul 02 14:25:06.932831 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:06.932796    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-5m7dt"]
Jul 02 14:25:06.933317 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:06.933067    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-5m7dt" podUID="37756f20-90bc-41f6-b5e9-9833355df7a0" containerName="seaweedfs-tls-custom" containerID="cri-o://6b42bc365716b945545f7d5ec1ad8584344ccc9f63844df1415d78bed6249a2a" gracePeriod=30
Jul 02 14:25:34.575900 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:34.575868    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-5m7dt"
Jul 02 14:25:34.608500 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:34.608472    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/37756f20-90bc-41f6-b5e9-9833355df7a0-data\") pod \"37756f20-90bc-41f6-b5e9-9833355df7a0\" (UID: \"37756f20-90bc-41f6-b5e9-9833355df7a0\") "
Jul 02 14:25:34.608657 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:34.608540    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsq7p\" (UniqueName: \"kubernetes.io/projected/37756f20-90bc-41f6-b5e9-9833355df7a0-kube-api-access-dsq7p\") pod \"37756f20-90bc-41f6-b5e9-9833355df7a0\" (UID: \"37756f20-90bc-41f6-b5e9-9833355df7a0\") "
Jul 02 14:25:34.609722 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:34.609688    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37756f20-90bc-41f6-b5e9-9833355df7a0-data" (OuterVolumeSpecName: "data") pod "37756f20-90bc-41f6-b5e9-9833355df7a0" (UID: "37756f20-90bc-41f6-b5e9-9833355df7a0"). InnerVolumeSpecName "data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 14:25:34.611191 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:34.611160    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37756f20-90bc-41f6-b5e9-9833355df7a0-kube-api-access-dsq7p" (OuterVolumeSpecName: "kube-api-access-dsq7p") pod "37756f20-90bc-41f6-b5e9-9833355df7a0" (UID: "37756f20-90bc-41f6-b5e9-9833355df7a0"). InnerVolumeSpecName "kube-api-access-dsq7p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:25:34.710057 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:34.709946    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsq7p\" (UniqueName: \"kubernetes.io/projected/37756f20-90bc-41f6-b5e9-9833355df7a0-kube-api-access-dsq7p\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:25:34.710057 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:34.709999    2582 reconciler_common.go:299] "Volume detached for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/37756f20-90bc-41f6-b5e9-9833355df7a0-data\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:25:34.764369 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:34.764334    2582 generic.go:356] "Generic (PLEG): container finished" podID="37756f20-90bc-41f6-b5e9-9833355df7a0" containerID="6b42bc365716b945545f7d5ec1ad8584344ccc9f63844df1415d78bed6249a2a" exitCode=0
Jul 02 14:25:34.764369 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:34.764372    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-5m7dt" event={"ID":"37756f20-90bc-41f6-b5e9-9833355df7a0","Type":"ContainerDied","Data":"6b42bc365716b945545f7d5ec1ad8584344ccc9f63844df1415d78bed6249a2a"}
Jul 02 14:25:34.764618 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:34.764383    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-5m7dt"
Jul 02 14:25:34.764618 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:34.764414    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-5m7dt" event={"ID":"37756f20-90bc-41f6-b5e9-9833355df7a0","Type":"ContainerDied","Data":"f1b3588a16a1600a87f5c5bdab85816f193b831436eaab7683dc8197153cb05f"}
Jul 02 14:25:34.764618 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:34.764436    2582 scope.go:117] "RemoveContainer" containerID="6b42bc365716b945545f7d5ec1ad8584344ccc9f63844df1415d78bed6249a2a"
Jul 02 14:25:34.774107 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:34.774089    2582 scope.go:117] "RemoveContainer" containerID="6b42bc365716b945545f7d5ec1ad8584344ccc9f63844df1415d78bed6249a2a"
Jul 02 14:25:34.774381 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:25:34.774364    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b42bc365716b945545f7d5ec1ad8584344ccc9f63844df1415d78bed6249a2a\": container with ID starting with 6b42bc365716b945545f7d5ec1ad8584344ccc9f63844df1415d78bed6249a2a not found: ID does not exist" containerID="6b42bc365716b945545f7d5ec1ad8584344ccc9f63844df1415d78bed6249a2a"
Jul 02 14:25:34.774432 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:34.774393    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b42bc365716b945545f7d5ec1ad8584344ccc9f63844df1415d78bed6249a2a"} err="failed to get container status \"6b42bc365716b945545f7d5ec1ad8584344ccc9f63844df1415d78bed6249a2a\": rpc error: code = NotFound desc = could not find container \"6b42bc365716b945545f7d5ec1ad8584344ccc9f63844df1415d78bed6249a2a\": container with ID starting with 6b42bc365716b945545f7d5ec1ad8584344ccc9f63844df1415d78bed6249a2a not found: ID does not exist"
Jul 02 14:25:34.791724 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:34.791695    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-5m7dt"]
Jul 02 14:25:34.795782 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:34.795759    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-5m7dt"]
Jul 02 14:25:34.857166 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:34.857131    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-tls-custom-8b44d767d-qcwpj"]
Jul 02 14:25:34.857419 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:34.857408    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37756f20-90bc-41f6-b5e9-9833355df7a0" containerName="seaweedfs-tls-custom"
Jul 02 14:25:34.857461 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:34.857421    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="37756f20-90bc-41f6-b5e9-9833355df7a0" containerName="seaweedfs-tls-custom"
Jul 02 14:25:34.857495 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:34.857489    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="37756f20-90bc-41f6-b5e9-9833355df7a0" containerName="seaweedfs-tls-custom"
Jul 02 14:25:34.861636 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:34.861617    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-8b44d767d-qcwpj"
Jul 02 14:25:34.864810 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:34.864794    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom\""
Jul 02 14:25:34.865060 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:34.865047    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom-artifact\""
Jul 02 14:25:34.871346 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:34.871324    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-8b44d767d-qcwpj"]
Jul 02 14:25:34.911844 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:34.911800    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85mdd\" (UniqueName: \"kubernetes.io/projected/3a5e2d1b-fe84-4da2-a2ef-c20c2b100e53-kube-api-access-85mdd\") pod \"seaweedfs-tls-custom-8b44d767d-qcwpj\" (UID: \"3a5e2d1b-fe84-4da2-a2ef-c20c2b100e53\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-qcwpj"
Jul 02 14:25:34.912069 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:34.911854    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"seaweedfs-tls-custom\" (UniqueName: \"kubernetes.io/projected/3a5e2d1b-fe84-4da2-a2ef-c20c2b100e53-seaweedfs-tls-custom\") pod \"seaweedfs-tls-custom-8b44d767d-qcwpj\" (UID: \"3a5e2d1b-fe84-4da2-a2ef-c20c2b100e53\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-qcwpj"
Jul 02 14:25:34.912069 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:34.911940    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3a5e2d1b-fe84-4da2-a2ef-c20c2b100e53-data\") pod \"seaweedfs-tls-custom-8b44d767d-qcwpj\" (UID: \"3a5e2d1b-fe84-4da2-a2ef-c20c2b100e53\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-qcwpj"
Jul 02 14:25:35.013296 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:35.013257    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3a5e2d1b-fe84-4da2-a2ef-c20c2b100e53-data\") pod \"seaweedfs-tls-custom-8b44d767d-qcwpj\" (UID: \"3a5e2d1b-fe84-4da2-a2ef-c20c2b100e53\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-qcwpj"
Jul 02 14:25:35.013477 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:35.013319    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85mdd\" (UniqueName: \"kubernetes.io/projected/3a5e2d1b-fe84-4da2-a2ef-c20c2b100e53-kube-api-access-85mdd\") pod \"seaweedfs-tls-custom-8b44d767d-qcwpj\" (UID: \"3a5e2d1b-fe84-4da2-a2ef-c20c2b100e53\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-qcwpj"
Jul 02 14:25:35.013477 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:35.013457    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"seaweedfs-tls-custom\" (UniqueName: \"kubernetes.io/projected/3a5e2d1b-fe84-4da2-a2ef-c20c2b100e53-seaweedfs-tls-custom\") pod \"seaweedfs-tls-custom-8b44d767d-qcwpj\" (UID: \"3a5e2d1b-fe84-4da2-a2ef-c20c2b100e53\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-qcwpj"
Jul 02 14:25:35.013719 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:35.013696    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/3a5e2d1b-fe84-4da2-a2ef-c20c2b100e53-data\") pod \"seaweedfs-tls-custom-8b44d767d-qcwpj\" (UID: \"3a5e2d1b-fe84-4da2-a2ef-c20c2b100e53\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-qcwpj"
Jul 02 14:25:35.016002 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:35.015985    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"seaweedfs-tls-custom\" (UniqueName: \"kubernetes.io/projected/3a5e2d1b-fe84-4da2-a2ef-c20c2b100e53-seaweedfs-tls-custom\") pod \"seaweedfs-tls-custom-8b44d767d-qcwpj\" (UID: \"3a5e2d1b-fe84-4da2-a2ef-c20c2b100e53\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-qcwpj"
Jul 02 14:25:35.024017 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:35.023991    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85mdd\" (UniqueName: \"kubernetes.io/projected/3a5e2d1b-fe84-4da2-a2ef-c20c2b100e53-kube-api-access-85mdd\") pod \"seaweedfs-tls-custom-8b44d767d-qcwpj\" (UID: \"3a5e2d1b-fe84-4da2-a2ef-c20c2b100e53\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-qcwpj"
Jul 02 14:25:35.170343 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:35.170291    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-8b44d767d-qcwpj"
Jul 02 14:25:35.304395 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:35.304371    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-8b44d767d-qcwpj"]
Jul 02 14:25:35.306703 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:25:35.306674    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a5e2d1b_fe84_4da2_a2ef_c20c2b100e53.slice/crio-f2051e37a39c5a939ff6c53992933a9a987b7755c52410ed17734d8ef7c95c95 WatchSource:0}: Error finding container f2051e37a39c5a939ff6c53992933a9a987b7755c52410ed17734d8ef7c95c95: Status 404 returned error can't find the container with id f2051e37a39c5a939ff6c53992933a9a987b7755c52410ed17734d8ef7c95c95
Jul 02 14:25:35.770290 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:35.770253    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-8b44d767d-qcwpj" event={"ID":"3a5e2d1b-fe84-4da2-a2ef-c20c2b100e53","Type":"ContainerStarted","Data":"485fe29e3a909b547c3c0677ce823d5600677b965b354b118cf34d893daf70f3"}
Jul 02 14:25:35.770756 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:35.770310    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-8b44d767d-qcwpj" event={"ID":"3a5e2d1b-fe84-4da2-a2ef-c20c2b100e53","Type":"ContainerStarted","Data":"f2051e37a39c5a939ff6c53992933a9a987b7755c52410ed17734d8ef7c95c95"}
Jul 02 14:25:35.790636 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:35.790558    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-tls-custom-8b44d767d-qcwpj" podStartSLOduration=1.505233035 podStartE2EDuration="1.790526164s" totalImagesPullingTime="285.293129ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:25:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:25:35.789529487 +0000 UTC m=+597.740867134" watchObservedRunningTime="2026-07-02 14:25:35.790526164 +0000 UTC m=+597.741863809"
Jul 02 14:25:36.475043 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:36.475009    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37756f20-90bc-41f6-b5e9-9833355df7a0" path="/var/lib/kubelet/pods/37756f20-90bc-41f6-b5e9-9833355df7a0/volumes"
Jul 02 14:25:38.439190 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:38.439160    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnwb5_ca909057-dd63-451d-9c05-b671ca5ae671/ovn-acl-logging/0.log"
Jul 02 14:25:38.442024 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:38.441998    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnwb5_ca909057-dd63-451d-9c05-b671ca5ae671/ovn-acl-logging/0.log"
Jul 02 14:25:48.270350 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:48.270312    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-tls-serving-67b7477c6d-gkts6"]
Jul 02 14:25:48.273837 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:48.273819    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-serving-67b7477c6d-gkts6"
Jul 02 14:25:48.276932 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:48.276909    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-serving\""
Jul 02 14:25:48.276932 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:48.276924    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-serving-artifact\""
Jul 02 14:25:48.286527 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:48.286505    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-serving-67b7477c6d-gkts6"]
Jul 02 14:25:48.427071 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:48.427031    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpqt4\" (UniqueName: \"kubernetes.io/projected/40676d2b-47dc-4415-8958-6498fc987001-kube-api-access-vpqt4\") pod \"seaweedfs-tls-serving-67b7477c6d-gkts6\" (UID: \"40676d2b-47dc-4415-8958-6498fc987001\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-gkts6"
Jul 02 14:25:48.427071 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:48.427074    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/40676d2b-47dc-4415-8958-6498fc987001-data\") pod \"seaweedfs-tls-serving-67b7477c6d-gkts6\" (UID: \"40676d2b-47dc-4415-8958-6498fc987001\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-gkts6"
Jul 02 14:25:48.427288 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:48.427103    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"seaweedfs-tls-serving\" (UniqueName: \"kubernetes.io/projected/40676d2b-47dc-4415-8958-6498fc987001-seaweedfs-tls-serving\") pod \"seaweedfs-tls-serving-67b7477c6d-gkts6\" (UID: \"40676d2b-47dc-4415-8958-6498fc987001\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-gkts6"
Jul 02 14:25:48.527988 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:48.527858    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpqt4\" (UniqueName: \"kubernetes.io/projected/40676d2b-47dc-4415-8958-6498fc987001-kube-api-access-vpqt4\") pod \"seaweedfs-tls-serving-67b7477c6d-gkts6\" (UID: \"40676d2b-47dc-4415-8958-6498fc987001\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-gkts6"
Jul 02 14:25:48.527988 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:48.527905    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/40676d2b-47dc-4415-8958-6498fc987001-data\") pod \"seaweedfs-tls-serving-67b7477c6d-gkts6\" (UID: \"40676d2b-47dc-4415-8958-6498fc987001\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-gkts6"
Jul 02 14:25:48.527988 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:48.527942    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"seaweedfs-tls-serving\" (UniqueName: \"kubernetes.io/projected/40676d2b-47dc-4415-8958-6498fc987001-seaweedfs-tls-serving\") pod \"seaweedfs-tls-serving-67b7477c6d-gkts6\" (UID: \"40676d2b-47dc-4415-8958-6498fc987001\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-gkts6"
Jul 02 14:25:48.528363 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:48.528340    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/40676d2b-47dc-4415-8958-6498fc987001-data\") pod \"seaweedfs-tls-serving-67b7477c6d-gkts6\" (UID: \"40676d2b-47dc-4415-8958-6498fc987001\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-gkts6"
Jul 02 14:25:48.530481 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:48.530462    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"seaweedfs-tls-serving\" (UniqueName: \"kubernetes.io/projected/40676d2b-47dc-4415-8958-6498fc987001-seaweedfs-tls-serving\") pod \"seaweedfs-tls-serving-67b7477c6d-gkts6\" (UID: \"40676d2b-47dc-4415-8958-6498fc987001\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-gkts6"
Jul 02 14:25:48.538270 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:48.538247    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpqt4\" (UniqueName: \"kubernetes.io/projected/40676d2b-47dc-4415-8958-6498fc987001-kube-api-access-vpqt4\") pod \"seaweedfs-tls-serving-67b7477c6d-gkts6\" (UID: \"40676d2b-47dc-4415-8958-6498fc987001\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-gkts6"
Jul 02 14:25:48.582876 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:48.582838    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-serving-67b7477c6d-gkts6"
Jul 02 14:25:48.719091 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:48.719066    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-serving-67b7477c6d-gkts6"]
Jul 02 14:25:48.721018 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:25:48.720994    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40676d2b_47dc_4415_8958_6498fc987001.slice/crio-b0dcf42d97a4a06e248421653cd29e1bc3eee3c66a5fe45866c338e7df50f93d WatchSource:0}: Error finding container b0dcf42d97a4a06e248421653cd29e1bc3eee3c66a5fe45866c338e7df50f93d: Status 404 returned error can't find the container with id b0dcf42d97a4a06e248421653cd29e1bc3eee3c66a5fe45866c338e7df50f93d
Jul 02 14:25:48.722308 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:48.722292    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 14:25:48.826389 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:48.826306    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-serving-67b7477c6d-gkts6" event={"ID":"40676d2b-47dc-4415-8958-6498fc987001","Type":"ContainerStarted","Data":"b0dcf42d97a4a06e248421653cd29e1bc3eee3c66a5fe45866c338e7df50f93d"}
Jul 02 14:25:49.831784 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:49.831747    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-serving-67b7477c6d-gkts6" event={"ID":"40676d2b-47dc-4415-8958-6498fc987001","Type":"ContainerStarted","Data":"3689d7525b23dcae9801eefe8e38d9df6cba2f7253298c1483fd2374dcb7fa4a"}
Jul 02 14:25:49.850512 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:25:49.850463    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-tls-serving-67b7477c6d-gkts6" podStartSLOduration=1.609850535 podStartE2EDuration="1.850447402s" totalImagesPullingTime="240.596867ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:25:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:25:49.849595468 +0000 UTC m=+611.800933113" watchObservedRunningTime="2026-07-02 14:25:49.850447402 +0000 UTC m=+611.801785048"
Jul 02 14:26:14.415091 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:14.415054    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"]
Jul 02 14:26:14.418682 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:14.418660    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"
Jul 02 14:26:14.421859 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:14.421832    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 02 14:26:14.421983 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:14.421935    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 02 14:26:14.422086 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:14.422064    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-predictor-serving-cert\""
Jul 02 14:26:14.422617 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:14.422595    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-kube-rbac-proxy-sar-config\""
Jul 02 14:26:14.422758 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:14.422732    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 02 14:26:14.422857 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:14.422772    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-7rr57\""
Jul 02 14:26:14.423250 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:14.423227    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 02 14:26:14.433491 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:14.433458    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"]
Jul 02 14:26:14.448052 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:14.448025    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9q4rs\" (UniqueName: \"kubernetes.io/projected/2e353236-2e86-4549-8aca-abea95514e22-kube-api-access-9q4rs\") pod \"isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p\" (UID: \"2e353236-2e86-4549-8aca-abea95514e22\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"
Jul 02 14:26:14.448149 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:14.448067    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2e353236-2e86-4549-8aca-abea95514e22-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p\" (UID: \"2e353236-2e86-4549-8aca-abea95514e22\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"
Jul 02 14:26:14.448149 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:14.448099    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2e353236-2e86-4549-8aca-abea95514e22-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p\" (UID: \"2e353236-2e86-4549-8aca-abea95514e22\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"
Jul 02 14:26:14.448221 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:14.448177    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2e353236-2e86-4549-8aca-abea95514e22-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p\" (UID: \"2e353236-2e86-4549-8aca-abea95514e22\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"
Jul 02 14:26:14.448254 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:14.448234    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2e353236-2e86-4549-8aca-abea95514e22-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p\" (UID: \"2e353236-2e86-4549-8aca-abea95514e22\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"
Jul 02 14:26:14.549330 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:14.549303    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2e353236-2e86-4549-8aca-abea95514e22-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p\" (UID: \"2e353236-2e86-4549-8aca-abea95514e22\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"
Jul 02 14:26:14.549330 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:14.549338    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9q4rs\" (UniqueName: \"kubernetes.io/projected/2e353236-2e86-4549-8aca-abea95514e22-kube-api-access-9q4rs\") pod \"isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p\" (UID: \"2e353236-2e86-4549-8aca-abea95514e22\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"
Jul 02 14:26:14.549520 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:14.549367    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2e353236-2e86-4549-8aca-abea95514e22-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p\" (UID: \"2e353236-2e86-4549-8aca-abea95514e22\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"
Jul 02 14:26:14.549520 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:14.549503    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2e353236-2e86-4549-8aca-abea95514e22-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p\" (UID: \"2e353236-2e86-4549-8aca-abea95514e22\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"
Jul 02 14:26:14.549622 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:14.549609    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2e353236-2e86-4549-8aca-abea95514e22-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p\" (UID: \"2e353236-2e86-4549-8aca-abea95514e22\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"
Jul 02 14:26:14.549732 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:14.549711    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2e353236-2e86-4549-8aca-abea95514e22-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p\" (UID: \"2e353236-2e86-4549-8aca-abea95514e22\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"
Jul 02 14:26:14.550168 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:14.550144    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2e353236-2e86-4549-8aca-abea95514e22-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p\" (UID: \"2e353236-2e86-4549-8aca-abea95514e22\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"
Jul 02 14:26:14.550168 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:14.550161    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2e353236-2e86-4549-8aca-abea95514e22-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p\" (UID: \"2e353236-2e86-4549-8aca-abea95514e22\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"
Jul 02 14:26:14.552046 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:14.552029    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2e353236-2e86-4549-8aca-abea95514e22-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p\" (UID: \"2e353236-2e86-4549-8aca-abea95514e22\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"
Jul 02 14:26:14.557745 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:14.557694    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9q4rs\" (UniqueName: \"kubernetes.io/projected/2e353236-2e86-4549-8aca-abea95514e22-kube-api-access-9q4rs\") pod \"isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p\" (UID: \"2e353236-2e86-4549-8aca-abea95514e22\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"
Jul 02 14:26:14.731821 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:14.731791    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"
Jul 02 14:26:14.866306 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:14.866259    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"]
Jul 02 14:26:14.868587 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:26:14.868544    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e353236_2e86_4549_8aca_abea95514e22.slice/crio-aac56d796e84b8e62a3670029899e9df818e7069239ce4add599885c7bc0d1d1 WatchSource:0}: Error finding container aac56d796e84b8e62a3670029899e9df818e7069239ce4add599885c7bc0d1d1: Status 404 returned error can't find the container with id aac56d796e84b8e62a3670029899e9df818e7069239ce4add599885c7bc0d1d1
Jul 02 14:26:14.936783 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:14.936744    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" event={"ID":"2e353236-2e86-4549-8aca-abea95514e22","Type":"ContainerStarted","Data":"aac56d796e84b8e62a3670029899e9df818e7069239ce4add599885c7bc0d1d1"}
Jul 02 14:26:19.963347 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:19.963314    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" event={"ID":"2e353236-2e86-4549-8aca-abea95514e22","Type":"ContainerStarted","Data":"1a0079939add69011eb062ac73b905817d9af86434f481e459eb70cdb2e17f0a"}
Jul 02 14:26:22.979517 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:22.979484    2582 generic.go:356] "Generic (PLEG): container finished" podID="2e353236-2e86-4549-8aca-abea95514e22" containerID="1a0079939add69011eb062ac73b905817d9af86434f481e459eb70cdb2e17f0a" exitCode=0
Jul 02 14:26:22.979906 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:22.979532    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" event={"ID":"2e353236-2e86-4549-8aca-abea95514e22","Type":"ContainerDied","Data":"1a0079939add69011eb062ac73b905817d9af86434f481e459eb70cdb2e17f0a"}
Jul 02 14:26:38.064174 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:38.064133    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" event={"ID":"2e353236-2e86-4549-8aca-abea95514e22","Type":"ContainerStarted","Data":"c84427fc1d077b52cc83faf952ffbd057c851359298b3767d4dc008d53c217e6"}
Jul 02 14:26:40.074914 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:40.074828    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" event={"ID":"2e353236-2e86-4549-8aca-abea95514e22","Type":"ContainerStarted","Data":"9c22af595c82c12b97ec5d81182540fa2c5e168ac8d3766d458aadc72b5c0e0e"}
Jul 02 14:26:43.091198 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:43.091162    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" event={"ID":"2e353236-2e86-4549-8aca-abea95514e22","Type":"ContainerStarted","Data":"0af059fc2392930de3ec5b6e12094dc0a99c03d61705ffb66b0f470092db4a65"}
Jul 02 14:26:43.091613 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:43.091408    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"
Jul 02 14:26:43.091613 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:43.091453    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"
Jul 02 14:26:43.092866 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:43.092838    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 02 14:26:43.116697 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:43.116653    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podStartSLOduration=1.5510510709999998 podStartE2EDuration="29.11663622s" totalImagesPullingTime="24.015141339s" totalInitContainerRuntime="3.55044381s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:26:14 +0000 UTC" imagePullSessionsCount=4 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:26:43.113570031 +0000 UTC m=+665.064907677" watchObservedRunningTime="2026-07-02 14:26:43.11663622 +0000 UTC m=+665.067973868"
Jul 02 14:26:44.095856 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:44.095805    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 02 14:26:44.096349 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:44.095829    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"
Jul 02 14:26:44.097018 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:44.096990    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:26:44.099786 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:44.099766    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"
Jul 02 14:26:45.100234 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:45.100191    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 02 14:26:45.100678 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:45.100527    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:26:46.104819 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:46.104782    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 02 14:26:46.105310 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:46.105016    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:26:56.105034 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:56.104987    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 02 14:26:56.105455 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:26:56.105364    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:27:06.104799 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:27:06.104757    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 02 14:27:06.105362 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:27:06.105340    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:27:16.104756 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:27:16.104705    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 02 14:27:16.105210 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:27:16.105184    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:27:26.104939 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:27:26.104878    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 02 14:27:26.105457 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:27:26.105316    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:27:36.105395 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:27:36.105296    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 02 14:27:36.105873 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:27:36.105736    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:27:46.105387 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:27:46.105341    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 02 14:27:46.105911 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:27:46.105842    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:27:56.105948 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:27:56.105915    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"
Jul 02 14:27:56.106416 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:27:56.106140    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"
Jul 02 14:28:09.524738 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:09.524706    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"]
Jul 02 14:28:09.525193 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:09.525133    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="kserve-container" containerID="cri-o://c84427fc1d077b52cc83faf952ffbd057c851359298b3767d4dc008d53c217e6" gracePeriod=30
Jul 02 14:28:09.525313 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:09.525289    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="kube-rbac-proxy" containerID="cri-o://9c22af595c82c12b97ec5d81182540fa2c5e168ac8d3766d458aadc72b5c0e0e" gracePeriod=30
Jul 02 14:28:09.525464 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:09.525296    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="agent" containerID="cri-o://0af059fc2392930de3ec5b6e12094dc0a99c03d61705ffb66b0f470092db4a65" gracePeriod=30
Jul 02 14:28:09.559620 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:09.559594    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"]
Jul 02 14:28:09.563112 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:09.563095    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"
Jul 02 14:28:09.565752 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:09.565732    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-custom-kube-rbac-proxy-sar-config\""
Jul 02 14:28:09.565849 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:09.565805    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-custom-predictor-serving-cert\""
Jul 02 14:28:09.572485 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:09.572461    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"]
Jul 02 14:28:09.663104 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:09.663068    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/613980ec-8ca3-4796-bee0-688ca252bbf4-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2\" (UID: \"613980ec-8ca3-4796-bee0-688ca252bbf4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"
Jul 02 14:28:09.663220 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:09.663118    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/613980ec-8ca3-4796-bee0-688ca252bbf4-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2\" (UID: \"613980ec-8ca3-4796-bee0-688ca252bbf4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"
Jul 02 14:28:09.663220 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:09.663173    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/613980ec-8ca3-4796-bee0-688ca252bbf4-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2\" (UID: \"613980ec-8ca3-4796-bee0-688ca252bbf4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"
Jul 02 14:28:09.663220 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:09.663193    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7f6v\" (UniqueName: \"kubernetes.io/projected/613980ec-8ca3-4796-bee0-688ca252bbf4-kube-api-access-w7f6v\") pod \"isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2\" (UID: \"613980ec-8ca3-4796-bee0-688ca252bbf4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"
Jul 02 14:28:09.663335 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:09.663260    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/613980ec-8ca3-4796-bee0-688ca252bbf4-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2\" (UID: \"613980ec-8ca3-4796-bee0-688ca252bbf4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"
Jul 02 14:28:09.764614 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:09.764568    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/613980ec-8ca3-4796-bee0-688ca252bbf4-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2\" (UID: \"613980ec-8ca3-4796-bee0-688ca252bbf4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"
Jul 02 14:28:09.764614 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:09.764612    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7f6v\" (UniqueName: \"kubernetes.io/projected/613980ec-8ca3-4796-bee0-688ca252bbf4-kube-api-access-w7f6v\") pod \"isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2\" (UID: \"613980ec-8ca3-4796-bee0-688ca252bbf4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"
Jul 02 14:28:09.764814 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:09.764654    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/613980ec-8ca3-4796-bee0-688ca252bbf4-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2\" (UID: \"613980ec-8ca3-4796-bee0-688ca252bbf4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"
Jul 02 14:28:09.764814 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:09.764686    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/613980ec-8ca3-4796-bee0-688ca252bbf4-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2\" (UID: \"613980ec-8ca3-4796-bee0-688ca252bbf4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"
Jul 02 14:28:09.764814 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:09.764759    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/613980ec-8ca3-4796-bee0-688ca252bbf4-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2\" (UID: \"613980ec-8ca3-4796-bee0-688ca252bbf4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"
Jul 02 14:28:09.764919 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:28:09.764812    2582 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-serving-cert: secret "isvc-sklearn-batcher-custom-predictor-serving-cert" not found
Jul 02 14:28:09.764919 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:28:09.764893    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/613980ec-8ca3-4796-bee0-688ca252bbf4-proxy-tls podName:613980ec-8ca3-4796-bee0-688ca252bbf4 nodeName:}" failed. No retries permitted until 2026-07-02 14:28:10.264869371 +0000 UTC m=+752.216206997 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/613980ec-8ca3-4796-bee0-688ca252bbf4-proxy-tls") pod "isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" (UID: "613980ec-8ca3-4796-bee0-688ca252bbf4") : secret "isvc-sklearn-batcher-custom-predictor-serving-cert" not found
Jul 02 14:28:09.765050 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:09.765036    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/613980ec-8ca3-4796-bee0-688ca252bbf4-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2\" (UID: \"613980ec-8ca3-4796-bee0-688ca252bbf4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"
Jul 02 14:28:09.765378 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:09.765358    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/613980ec-8ca3-4796-bee0-688ca252bbf4-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2\" (UID: \"613980ec-8ca3-4796-bee0-688ca252bbf4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"
Jul 02 14:28:09.765378 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:09.765371    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/613980ec-8ca3-4796-bee0-688ca252bbf4-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2\" (UID: \"613980ec-8ca3-4796-bee0-688ca252bbf4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"
Jul 02 14:28:09.773552 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:09.773529    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7f6v\" (UniqueName: \"kubernetes.io/projected/613980ec-8ca3-4796-bee0-688ca252bbf4-kube-api-access-w7f6v\") pod \"isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2\" (UID: \"613980ec-8ca3-4796-bee0-688ca252bbf4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"
Jul 02 14:28:10.270309 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:10.270277    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/613980ec-8ca3-4796-bee0-688ca252bbf4-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2\" (UID: \"613980ec-8ca3-4796-bee0-688ca252bbf4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"
Jul 02 14:28:10.272745 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:10.272722    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/613980ec-8ca3-4796-bee0-688ca252bbf4-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2\" (UID: \"613980ec-8ca3-4796-bee0-688ca252bbf4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"
Jul 02 14:28:10.462656 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:10.462625    2582 generic.go:356] "Generic (PLEG): container finished" podID="2e353236-2e86-4549-8aca-abea95514e22" containerID="9c22af595c82c12b97ec5d81182540fa2c5e168ac8d3766d458aadc72b5c0e0e" exitCode=2
Jul 02 14:28:10.462816 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:10.462705    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" event={"ID":"2e353236-2e86-4549-8aca-abea95514e22","Type":"ContainerDied","Data":"9c22af595c82c12b97ec5d81182540fa2c5e168ac8d3766d458aadc72b5c0e0e"}
Jul 02 14:28:10.473504 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:10.473481    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"
Jul 02 14:28:10.613857 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:10.613834    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"]
Jul 02 14:28:10.616216 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:28:10.616190    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod613980ec_8ca3_4796_bee0_688ca252bbf4.slice/crio-fc88791e1c6de9a6889011a5b91ee8d478b662b60007c0f48ce6ff815e3a91da WatchSource:0}: Error finding container fc88791e1c6de9a6889011a5b91ee8d478b662b60007c0f48ce6ff815e3a91da: Status 404 returned error can't find the container with id fc88791e1c6de9a6889011a5b91ee8d478b662b60007c0f48ce6ff815e3a91da
Jul 02 14:28:11.467936 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:11.467896    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" event={"ID":"613980ec-8ca3-4796-bee0-688ca252bbf4","Type":"ContainerStarted","Data":"a8e8f520de3a6ebc42df2d45aadaeabda7449a6e397f2fad461f9ab159da9eab"}
Jul 02 14:28:11.468140 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:11.467984    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" event={"ID":"613980ec-8ca3-4796-bee0-688ca252bbf4","Type":"ContainerStarted","Data":"fc88791e1c6de9a6889011a5b91ee8d478b662b60007c0f48ce6ff815e3a91da"}
Jul 02 14:28:14.096977 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:14.096925    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused"
Jul 02 14:28:14.481999 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:14.481966    2582 generic.go:356] "Generic (PLEG): container finished" podID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerID="a8e8f520de3a6ebc42df2d45aadaeabda7449a6e397f2fad461f9ab159da9eab" exitCode=0
Jul 02 14:28:14.482182 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:14.482026    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" event={"ID":"613980ec-8ca3-4796-bee0-688ca252bbf4","Type":"ContainerDied","Data":"a8e8f520de3a6ebc42df2d45aadaeabda7449a6e397f2fad461f9ab159da9eab"}
Jul 02 14:28:14.484678 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:14.484640    2582 generic.go:356] "Generic (PLEG): container finished" podID="2e353236-2e86-4549-8aca-abea95514e22" containerID="c84427fc1d077b52cc83faf952ffbd057c851359298b3767d4dc008d53c217e6" exitCode=0
Jul 02 14:28:14.484864 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:14.484705    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" event={"ID":"2e353236-2e86-4549-8aca-abea95514e22","Type":"ContainerDied","Data":"c84427fc1d077b52cc83faf952ffbd057c851359298b3767d4dc008d53c217e6"}
Jul 02 14:28:15.491282 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:15.491238    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" event={"ID":"613980ec-8ca3-4796-bee0-688ca252bbf4","Type":"ContainerStarted","Data":"945a517bde41be8a68ba339b1f0921844b0a5f70578a2163754c2e79c84ea1c0"}
Jul 02 14:28:15.491725 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:15.491319    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" event={"ID":"613980ec-8ca3-4796-bee0-688ca252bbf4","Type":"ContainerStarted","Data":"b8985c1853c40a32574f79d5c00d77de8d641ce03a9625876455a6886cdc207c"}
Jul 02 14:28:15.491725 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:15.491355    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" event={"ID":"613980ec-8ca3-4796-bee0-688ca252bbf4","Type":"ContainerStarted","Data":"1e54fde344ccd6dd1475e98ccac3d11b8167b6918a27a033743be99cf712dcfc"}
Jul 02 14:28:15.491725 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:15.491669    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"
Jul 02 14:28:15.491725 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:15.491716    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"
Jul 02 14:28:15.491948 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:15.491740    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"
Jul 02 14:28:15.493259 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:15.493226    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:5000: connect: connection refused"
Jul 02 14:28:15.493940 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:15.493917    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:28:15.514474 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:15.514433    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podStartSLOduration=2.880397997 podStartE2EDuration="6.514419782s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.634021785s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:28:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:28:15.512448226 +0000 UTC m=+757.463785884" watchObservedRunningTime="2026-07-02 14:28:15.514419782 +0000 UTC m=+757.465757427"
Jul 02 14:28:16.105172 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:16.105126    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 02 14:28:16.105509 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:16.105482    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:28:16.495475 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:16.495441    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:5000: connect: connection refused"
Jul 02 14:28:16.496018 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:16.495992    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:28:19.096733 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:19.096685    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused"
Jul 02 14:28:21.499921 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:21.499893    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"
Jul 02 14:28:21.500459 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:21.500429    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:5000: connect: connection refused"
Jul 02 14:28:21.500980 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:21.500932    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:28:24.097039 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:24.096990    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused"
Jul 02 14:28:24.097493 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:24.097217    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"
Jul 02 14:28:26.105499 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:26.105456    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 02 14:28:26.105993 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:26.105796    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:28:29.096033 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:29.095987    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused"
Jul 02 14:28:31.500790 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:31.500743    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:5000: connect: connection refused"
Jul 02 14:28:31.501240 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:31.501202    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:28:34.096325 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:34.096283    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused"
Jul 02 14:28:36.104974 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:36.104900    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Jul 02 14:28:36.105351 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:36.105122    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"
Jul 02 14:28:36.105351 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:36.105236    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:28:36.105351 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:36.105336    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"
Jul 02 14:28:39.096295 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:39.096259    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused"
Jul 02 14:28:39.601231 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:39.601198    2582 generic.go:356] "Generic (PLEG): container finished" podID="2e353236-2e86-4549-8aca-abea95514e22" containerID="0af059fc2392930de3ec5b6e12094dc0a99c03d61705ffb66b0f470092db4a65" exitCode=0
Jul 02 14:28:39.601356 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:39.601244    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" event={"ID":"2e353236-2e86-4549-8aca-abea95514e22","Type":"ContainerDied","Data":"0af059fc2392930de3ec5b6e12094dc0a99c03d61705ffb66b0f470092db4a65"}
Jul 02 14:28:39.677590 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:39.677566    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"
Jul 02 14:28:39.697963 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:39.697935    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2e353236-2e86-4549-8aca-abea95514e22-kserve-provision-location\") pod \"2e353236-2e86-4549-8aca-abea95514e22\" (UID: \"2e353236-2e86-4549-8aca-abea95514e22\") "
Jul 02 14:28:39.698110 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:39.697993    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9q4rs\" (UniqueName: \"kubernetes.io/projected/2e353236-2e86-4549-8aca-abea95514e22-kube-api-access-9q4rs\") pod \"2e353236-2e86-4549-8aca-abea95514e22\" (UID: \"2e353236-2e86-4549-8aca-abea95514e22\") "
Jul 02 14:28:39.698110 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:39.698033    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2e353236-2e86-4549-8aca-abea95514e22-cabundle-cert\") pod \"2e353236-2e86-4549-8aca-abea95514e22\" (UID: \"2e353236-2e86-4549-8aca-abea95514e22\") "
Jul 02 14:28:39.698110 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:39.698066    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2e353236-2e86-4549-8aca-abea95514e22-proxy-tls\") pod \"2e353236-2e86-4549-8aca-abea95514e22\" (UID: \"2e353236-2e86-4549-8aca-abea95514e22\") "
Jul 02 14:28:39.698110 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:39.698104    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2e353236-2e86-4549-8aca-abea95514e22-kube-rbac-proxy-sar-config\") pod \"2e353236-2e86-4549-8aca-abea95514e22\" (UID: \"2e353236-2e86-4549-8aca-abea95514e22\") "
Jul 02 14:28:39.698312 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:39.698185    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e353236-2e86-4549-8aca-abea95514e22-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "2e353236-2e86-4549-8aca-abea95514e22" (UID: "2e353236-2e86-4549-8aca-abea95514e22"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 14:28:39.698402 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:39.698360    2582 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2e353236-2e86-4549-8aca-abea95514e22-kserve-provision-location\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:28:39.698464 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:39.698425    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e353236-2e86-4549-8aca-abea95514e22-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "2e353236-2e86-4549-8aca-abea95514e22" (UID: "2e353236-2e86-4549-8aca-abea95514e22"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:28:39.698526 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:39.698492    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e353236-2e86-4549-8aca-abea95514e22-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "2e353236-2e86-4549-8aca-abea95514e22" (UID: "2e353236-2e86-4549-8aca-abea95514e22"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:28:39.700375 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:39.700355    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e353236-2e86-4549-8aca-abea95514e22-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "2e353236-2e86-4549-8aca-abea95514e22" (UID: "2e353236-2e86-4549-8aca-abea95514e22"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 14:28:39.700456 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:39.700439    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e353236-2e86-4549-8aca-abea95514e22-kube-api-access-9q4rs" (OuterVolumeSpecName: "kube-api-access-9q4rs") pod "2e353236-2e86-4549-8aca-abea95514e22" (UID: "2e353236-2e86-4549-8aca-abea95514e22"). InnerVolumeSpecName "kube-api-access-9q4rs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:28:39.799100 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:39.799012    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9q4rs\" (UniqueName: \"kubernetes.io/projected/2e353236-2e86-4549-8aca-abea95514e22-kube-api-access-9q4rs\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:28:39.799100 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:39.799038    2582 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2e353236-2e86-4549-8aca-abea95514e22-cabundle-cert\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:28:39.799100 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:39.799050    2582 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2e353236-2e86-4549-8aca-abea95514e22-proxy-tls\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:28:39.799100 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:39.799062    2582 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2e353236-2e86-4549-8aca-abea95514e22-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:28:40.607840 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:40.607807    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p" event={"ID":"2e353236-2e86-4549-8aca-abea95514e22","Type":"ContainerDied","Data":"aac56d796e84b8e62a3670029899e9df818e7069239ce4add599885c7bc0d1d1"}
Jul 02 14:28:40.608308 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:40.607863    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"
Jul 02 14:28:40.608308 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:40.607897    2582 scope.go:117] "RemoveContainer" containerID="0af059fc2392930de3ec5b6e12094dc0a99c03d61705ffb66b0f470092db4a65"
Jul 02 14:28:40.616196 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:40.616174    2582 scope.go:117] "RemoveContainer" containerID="9c22af595c82c12b97ec5d81182540fa2c5e168ac8d3766d458aadc72b5c0e0e"
Jul 02 14:28:40.623454 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:40.623436    2582 scope.go:117] "RemoveContainer" containerID="c84427fc1d077b52cc83faf952ffbd057c851359298b3767d4dc008d53c217e6"
Jul 02 14:28:40.628994 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:40.628968    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"]
Jul 02 14:28:40.631264 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:40.631243    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-645bdbffc9-q6l5p"]
Jul 02 14:28:40.631680 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:40.631660    2582 scope.go:117] "RemoveContainer" containerID="1a0079939add69011eb062ac73b905817d9af86434f481e459eb70cdb2e17f0a"
Jul 02 14:28:41.500568 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:41.500534    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:5000: connect: connection refused"
Jul 02 14:28:41.501095 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:41.501072    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:28:42.475353 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:42.475321    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e353236-2e86-4549-8aca-abea95514e22" path="/var/lib/kubelet/pods/2e353236-2e86-4549-8aca-abea95514e22/volumes"
Jul 02 14:28:51.500459 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:51.500417    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:5000: connect: connection refused"
Jul 02 14:28:51.501000 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:28:51.500913    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:29:01.500980 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:01.500860    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:5000: connect: connection refused"
Jul 02 14:29:01.501339 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:01.501246    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:29:11.500789 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:11.500747    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:5000: connect: connection refused"
Jul 02 14:29:11.501281 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:11.501229    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:29:21.500945 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:21.500900    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:5000: connect: connection refused"
Jul 02 14:29:21.501373 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:21.501351    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:29:31.501439 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:31.501404    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"
Jul 02 14:29:31.501993 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:31.501668    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"
Jul 02 14:29:44.750651 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:44.750620    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs"]
Jul 02 14:29:44.751170 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:44.751149    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="agent"
Jul 02 14:29:44.751251 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:44.751173    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="agent"
Jul 02 14:29:44.751251 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:44.751227    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="kube-rbac-proxy"
Jul 02 14:29:44.751251 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:44.751237    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="kube-rbac-proxy"
Jul 02 14:29:44.751251 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:44.751249    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="storage-initializer"
Jul 02 14:29:44.751445 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:44.751259    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="storage-initializer"
Jul 02 14:29:44.751445 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:44.751273    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="kserve-container"
Jul 02 14:29:44.751445 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:44.751282    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="kserve-container"
Jul 02 14:29:44.751445 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:44.751398    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="kserve-container"
Jul 02 14:29:44.751445 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:44.751414    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="agent"
Jul 02 14:29:44.751445 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:44.751428    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e353236-2e86-4549-8aca-abea95514e22" containerName="kube-rbac-proxy"
Jul 02 14:29:44.755783 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:44.755760    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs"
Jul 02 14:29:44.758785 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:44.758763    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-predictor-serving-cert\""
Jul 02 14:29:44.758909 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:44.758767    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-kube-rbac-proxy-sar-config\""
Jul 02 14:29:44.764594 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:44.764571    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs"]
Jul 02 14:29:44.793421 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:44.793393    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"]
Jul 02 14:29:44.793664 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:44.793642    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="kserve-container" containerID="cri-o://1e54fde344ccd6dd1475e98ccac3d11b8167b6918a27a033743be99cf712dcfc" gracePeriod=30
Jul 02 14:29:44.793741 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:44.793683    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="agent" containerID="cri-o://945a517bde41be8a68ba339b1f0921844b0a5f70578a2163754c2e79c84ea1c0" gracePeriod=30
Jul 02 14:29:44.793799 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:44.793721    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="kube-rbac-proxy" containerID="cri-o://b8985c1853c40a32574f79d5c00d77de8d641ce03a9625876455a6886cdc207c" gracePeriod=30
Jul 02 14:29:44.832017 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:44.831990    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/512aab7b-05bb-44db-b212-915d9a67de43-proxy-tls\") pod \"message-dumper-predictor-6575bcfdfd-8hkxs\" (UID: \"512aab7b-05bb-44db-b212-915d9a67de43\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs"
Jul 02 14:29:44.832154 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:44.832050    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-852pn\" (UniqueName: \"kubernetes.io/projected/512aab7b-05bb-44db-b212-915d9a67de43-kube-api-access-852pn\") pod \"message-dumper-predictor-6575bcfdfd-8hkxs\" (UID: \"512aab7b-05bb-44db-b212-915d9a67de43\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs"
Jul 02 14:29:44.832154 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:44.832133    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/512aab7b-05bb-44db-b212-915d9a67de43-kube-rbac-proxy-sar-config\") pod \"message-dumper-predictor-6575bcfdfd-8hkxs\" (UID: \"512aab7b-05bb-44db-b212-915d9a67de43\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs"
Jul 02 14:29:44.932509 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:44.932476    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-852pn\" (UniqueName: \"kubernetes.io/projected/512aab7b-05bb-44db-b212-915d9a67de43-kube-api-access-852pn\") pod \"message-dumper-predictor-6575bcfdfd-8hkxs\" (UID: \"512aab7b-05bb-44db-b212-915d9a67de43\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs"
Jul 02 14:29:44.932681 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:44.932529    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/512aab7b-05bb-44db-b212-915d9a67de43-kube-rbac-proxy-sar-config\") pod \"message-dumper-predictor-6575bcfdfd-8hkxs\" (UID: \"512aab7b-05bb-44db-b212-915d9a67de43\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs"
Jul 02 14:29:44.932681 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:44.932565    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/512aab7b-05bb-44db-b212-915d9a67de43-proxy-tls\") pod \"message-dumper-predictor-6575bcfdfd-8hkxs\" (UID: \"512aab7b-05bb-44db-b212-915d9a67de43\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs"
Jul 02 14:29:44.932793 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:29:44.932682    2582 secret.go:189] Couldn't get secret kserve-ci-e2e-test/message-dumper-predictor-serving-cert: secret "message-dumper-predictor-serving-cert" not found
Jul 02 14:29:44.932793 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:29:44.932746    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/512aab7b-05bb-44db-b212-915d9a67de43-proxy-tls podName:512aab7b-05bb-44db-b212-915d9a67de43 nodeName:}" failed. No retries permitted until 2026-07-02 14:29:45.432724621 +0000 UTC m=+847.384062269 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/512aab7b-05bb-44db-b212-915d9a67de43-proxy-tls") pod "message-dumper-predictor-6575bcfdfd-8hkxs" (UID: "512aab7b-05bb-44db-b212-915d9a67de43") : secret "message-dumper-predictor-serving-cert" not found
Jul 02 14:29:44.933242 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:44.933221    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/512aab7b-05bb-44db-b212-915d9a67de43-kube-rbac-proxy-sar-config\") pod \"message-dumper-predictor-6575bcfdfd-8hkxs\" (UID: \"512aab7b-05bb-44db-b212-915d9a67de43\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs"
Jul 02 14:29:44.941600 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:44.941577    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-852pn\" (UniqueName: \"kubernetes.io/projected/512aab7b-05bb-44db-b212-915d9a67de43-kube-api-access-852pn\") pod \"message-dumper-predictor-6575bcfdfd-8hkxs\" (UID: \"512aab7b-05bb-44db-b212-915d9a67de43\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs"
Jul 02 14:29:45.435697 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:45.435661    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/512aab7b-05bb-44db-b212-915d9a67de43-proxy-tls\") pod \"message-dumper-predictor-6575bcfdfd-8hkxs\" (UID: \"512aab7b-05bb-44db-b212-915d9a67de43\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs"
Jul 02 14:29:45.438368 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:45.438343    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/512aab7b-05bb-44db-b212-915d9a67de43-proxy-tls\") pod \"message-dumper-predictor-6575bcfdfd-8hkxs\" (UID: \"512aab7b-05bb-44db-b212-915d9a67de43\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs"
Jul 02 14:29:45.667170 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:45.667134    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs"
Jul 02 14:29:45.801723 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:45.801690    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs"]
Jul 02 14:29:45.804731 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:29:45.804702    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod512aab7b_05bb_44db_b212_915d9a67de43.slice/crio-0353ffda245d36ae0539b2fcec839b81df1a43b03ae53b17435d225c3fba02fc WatchSource:0}: Error finding container 0353ffda245d36ae0539b2fcec839b81df1a43b03ae53b17435d225c3fba02fc: Status 404 returned error can't find the container with id 0353ffda245d36ae0539b2fcec839b81df1a43b03ae53b17435d225c3fba02fc
Jul 02 14:29:45.882115 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:45.882080    2582 generic.go:356] "Generic (PLEG): container finished" podID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerID="b8985c1853c40a32574f79d5c00d77de8d641ce03a9625876455a6886cdc207c" exitCode=2
Jul 02 14:29:45.882298 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:45.882153    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" event={"ID":"613980ec-8ca3-4796-bee0-688ca252bbf4","Type":"ContainerDied","Data":"b8985c1853c40a32574f79d5c00d77de8d641ce03a9625876455a6886cdc207c"}
Jul 02 14:29:45.883251 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:45.883222    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs" event={"ID":"512aab7b-05bb-44db-b212-915d9a67de43","Type":"ContainerStarted","Data":"0353ffda245d36ae0539b2fcec839b81df1a43b03ae53b17435d225c3fba02fc"}
Jul 02 14:29:46.496350 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:46.496313    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.26:8643/healthz\": dial tcp 10.133.0.26:8643: connect: connection refused"
Jul 02 14:29:47.894514 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:47.894468    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs" event={"ID":"512aab7b-05bb-44db-b212-915d9a67de43","Type":"ContainerStarted","Data":"7adc1a55c856f5766f7d935794930e847af19d1d88fb1242398c59be30d80f99"}
Jul 02 14:29:47.894971 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:47.894545    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs" event={"ID":"512aab7b-05bb-44db-b212-915d9a67de43","Type":"ContainerStarted","Data":"a4ec224801b1b26de11f656ef821155f39cdbb77b5533d050289627167680923"}
Jul 02 14:29:47.894971 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:47.894574    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs"
Jul 02 14:29:47.894971 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:47.894593    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs"
Jul 02 14:29:47.896362 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:47.896341    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs"
Jul 02 14:29:47.913724 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:47.913673    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs" podStartSLOduration=2.414869029 podStartE2EDuration="3.913658173s" totalImagesPullingTime="1.498789144s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:29:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:29:47.912462359 +0000 UTC m=+849.863800007" watchObservedRunningTime="2026-07-02 14:29:47.913658173 +0000 UTC m=+849.864995797"
Jul 02 14:29:49.513402 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:29:49.513378    2582 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod613980ec_8ca3_4796_bee0_688ca252bbf4.slice/crio-1e54fde344ccd6dd1475e98ccac3d11b8167b6918a27a033743be99cf712dcfc.scope\": RecentStats: unable to find data in memory cache]"
Jul 02 14:29:49.904628 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:49.904547    2582 generic.go:356] "Generic (PLEG): container finished" podID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerID="1e54fde344ccd6dd1475e98ccac3d11b8167b6918a27a033743be99cf712dcfc" exitCode=0
Jul 02 14:29:49.904793 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:49.904624    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" event={"ID":"613980ec-8ca3-4796-bee0-688ca252bbf4","Type":"ContainerDied","Data":"1e54fde344ccd6dd1475e98ccac3d11b8167b6918a27a033743be99cf712dcfc"}
Jul 02 14:29:51.496412 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:51.496373    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.26:8643/healthz\": dial tcp 10.133.0.26:8643: connect: connection refused"
Jul 02 14:29:51.500701 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:51.500670    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:5000: connect: connection refused"
Jul 02 14:29:51.501171 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:51.501146    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:29:54.909692 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:54.909662    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs"
Jul 02 14:29:56.496574 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:56.496524    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.26:8643/healthz\": dial tcp 10.133.0.26:8643: connect: connection refused"
Jul 02 14:29:56.497044 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:29:56.496672    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"
Jul 02 14:30:01.496176 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:01.496128    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.26:8643/healthz\": dial tcp 10.133.0.26:8643: connect: connection refused"
Jul 02 14:30:01.500565 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:01.500537    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:5000: connect: connection refused"
Jul 02 14:30:01.500920 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:01.500898    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:30:04.810324 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:04.810289    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"]
Jul 02 14:30:04.814035 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:04.814013    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"
Jul 02 14:30:04.816775 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:04.816754    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-logger-predictor-serving-cert\""
Jul 02 14:30:04.816856 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:04.816803    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-logger-kube-rbac-proxy-sar-config\""
Jul 02 14:30:04.825171 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:04.825150    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"]
Jul 02 14:30:04.910563 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:04.910531    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/78697d7d-fbc6-4f1b-8754-f028b990e717-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-756bf5f9fb-28ssz\" (UID: \"78697d7d-fbc6-4f1b-8754-f028b990e717\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"
Jul 02 14:30:04.910749 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:04.910621    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/78697d7d-fbc6-4f1b-8754-f028b990e717-kserve-provision-location\") pod \"isvc-logger-predictor-756bf5f9fb-28ssz\" (UID: \"78697d7d-fbc6-4f1b-8754-f028b990e717\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"
Jul 02 14:30:04.910749 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:04.910650    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/78697d7d-fbc6-4f1b-8754-f028b990e717-proxy-tls\") pod \"isvc-logger-predictor-756bf5f9fb-28ssz\" (UID: \"78697d7d-fbc6-4f1b-8754-f028b990e717\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"
Jul 02 14:30:04.910749 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:04.910675    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpbg9\" (UniqueName: \"kubernetes.io/projected/78697d7d-fbc6-4f1b-8754-f028b990e717-kube-api-access-fpbg9\") pod \"isvc-logger-predictor-756bf5f9fb-28ssz\" (UID: \"78697d7d-fbc6-4f1b-8754-f028b990e717\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"
Jul 02 14:30:04.910877 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:04.910748    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/78697d7d-fbc6-4f1b-8754-f028b990e717-cabundle-cert\") pod \"isvc-logger-predictor-756bf5f9fb-28ssz\" (UID: \"78697d7d-fbc6-4f1b-8754-f028b990e717\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"
Jul 02 14:30:05.011601 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:05.011559    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/78697d7d-fbc6-4f1b-8754-f028b990e717-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-756bf5f9fb-28ssz\" (UID: \"78697d7d-fbc6-4f1b-8754-f028b990e717\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"
Jul 02 14:30:05.011814 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:05.011650    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/78697d7d-fbc6-4f1b-8754-f028b990e717-kserve-provision-location\") pod \"isvc-logger-predictor-756bf5f9fb-28ssz\" (UID: \"78697d7d-fbc6-4f1b-8754-f028b990e717\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"
Jul 02 14:30:05.011814 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:05.011676    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/78697d7d-fbc6-4f1b-8754-f028b990e717-proxy-tls\") pod \"isvc-logger-predictor-756bf5f9fb-28ssz\" (UID: \"78697d7d-fbc6-4f1b-8754-f028b990e717\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"
Jul 02 14:30:05.011814 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:05.011702    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpbg9\" (UniqueName: \"kubernetes.io/projected/78697d7d-fbc6-4f1b-8754-f028b990e717-kube-api-access-fpbg9\") pod \"isvc-logger-predictor-756bf5f9fb-28ssz\" (UID: \"78697d7d-fbc6-4f1b-8754-f028b990e717\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"
Jul 02 14:30:05.011814 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:05.011738    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/78697d7d-fbc6-4f1b-8754-f028b990e717-cabundle-cert\") pod \"isvc-logger-predictor-756bf5f9fb-28ssz\" (UID: \"78697d7d-fbc6-4f1b-8754-f028b990e717\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"
Jul 02 14:30:05.011814 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:30:05.011807    2582 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-logger-predictor-serving-cert: secret "isvc-logger-predictor-serving-cert" not found
Jul 02 14:30:05.012174 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:30:05.011881    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78697d7d-fbc6-4f1b-8754-f028b990e717-proxy-tls podName:78697d7d-fbc6-4f1b-8754-f028b990e717 nodeName:}" failed. No retries permitted until 2026-07-02 14:30:05.511856902 +0000 UTC m=+867.463194541 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/78697d7d-fbc6-4f1b-8754-f028b990e717-proxy-tls") pod "isvc-logger-predictor-756bf5f9fb-28ssz" (UID: "78697d7d-fbc6-4f1b-8754-f028b990e717") : secret "isvc-logger-predictor-serving-cert" not found
Jul 02 14:30:05.012174 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:05.012105    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/78697d7d-fbc6-4f1b-8754-f028b990e717-kserve-provision-location\") pod \"isvc-logger-predictor-756bf5f9fb-28ssz\" (UID: \"78697d7d-fbc6-4f1b-8754-f028b990e717\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"
Jul 02 14:30:05.012344 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:05.012325    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/78697d7d-fbc6-4f1b-8754-f028b990e717-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-756bf5f9fb-28ssz\" (UID: \"78697d7d-fbc6-4f1b-8754-f028b990e717\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"
Jul 02 14:30:05.012465 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:05.012445    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/78697d7d-fbc6-4f1b-8754-f028b990e717-cabundle-cert\") pod \"isvc-logger-predictor-756bf5f9fb-28ssz\" (UID: \"78697d7d-fbc6-4f1b-8754-f028b990e717\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"
Jul 02 14:30:05.020825 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:05.020798    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpbg9\" (UniqueName: \"kubernetes.io/projected/78697d7d-fbc6-4f1b-8754-f028b990e717-kube-api-access-fpbg9\") pod \"isvc-logger-predictor-756bf5f9fb-28ssz\" (UID: \"78697d7d-fbc6-4f1b-8754-f028b990e717\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"
Jul 02 14:30:05.515639 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:05.515601    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/78697d7d-fbc6-4f1b-8754-f028b990e717-proxy-tls\") pod \"isvc-logger-predictor-756bf5f9fb-28ssz\" (UID: \"78697d7d-fbc6-4f1b-8754-f028b990e717\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"
Jul 02 14:30:05.518154 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:05.518135    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/78697d7d-fbc6-4f1b-8754-f028b990e717-proxy-tls\") pod \"isvc-logger-predictor-756bf5f9fb-28ssz\" (UID: \"78697d7d-fbc6-4f1b-8754-f028b990e717\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"
Jul 02 14:30:05.725064 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:05.725027    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"
Jul 02 14:30:05.872067 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:05.871844    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"]
Jul 02 14:30:05.975816 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:05.975782    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" event={"ID":"78697d7d-fbc6-4f1b-8754-f028b990e717","Type":"ContainerStarted","Data":"bcfce565aa9f1aeaa6c74a10ef4d641f62cc52ad6237357d017f34d2232e9f90"}
Jul 02 14:30:05.976043 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:05.975852    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" event={"ID":"78697d7d-fbc6-4f1b-8754-f028b990e717","Type":"ContainerStarted","Data":"b25ca376fd465b21a575a2679473d851d70982e3b9061a60f8258e49841a38bd"}
Jul 02 14:30:06.495869 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:06.495828    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.26:8643/healthz\": dial tcp 10.133.0.26:8643: connect: connection refused"
Jul 02 14:30:09.993329 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:09.993285    2582 generic.go:356] "Generic (PLEG): container finished" podID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerID="bcfce565aa9f1aeaa6c74a10ef4d641f62cc52ad6237357d017f34d2232e9f90" exitCode=0
Jul 02 14:30:09.993700 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:09.993347    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" event={"ID":"78697d7d-fbc6-4f1b-8754-f028b990e717","Type":"ContainerDied","Data":"bcfce565aa9f1aeaa6c74a10ef4d641f62cc52ad6237357d017f34d2232e9f90"}
Jul 02 14:30:10.999918 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:10.999886    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" event={"ID":"78697d7d-fbc6-4f1b-8754-f028b990e717","Type":"ContainerStarted","Data":"dbf5451a7d73c8c9c208cd73aa299389e23bc13253a1736bd98761b5d88aa2b9"}
Jul 02 14:30:11.000361 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:10.999972    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" event={"ID":"78697d7d-fbc6-4f1b-8754-f028b990e717","Type":"ContainerStarted","Data":"5c54dca13af5034b1a6c6e7fa3d7b02d2e6084da428ac77047a99589d6daddfd"}
Jul 02 14:30:11.000361 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:11.000001    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" event={"ID":"78697d7d-fbc6-4f1b-8754-f028b990e717","Type":"ContainerStarted","Data":"eab3a544988799f956d4c618044de09fefdd9ae26d2e107517841b980795a2f9"}
Jul 02 14:30:11.000361 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:11.000243    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"
Jul 02 14:30:11.000361 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:11.000308    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"
Jul 02 14:30:11.001737 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:11.001713    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 02 14:30:11.026142 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:11.026097    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podStartSLOduration=3.146886009 podStartE2EDuration="7.026083359s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.87919735s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:30:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:30:11.025109471 +0000 UTC m=+872.976447117" watchObservedRunningTime="2026-07-02 14:30:11.026083359 +0000 UTC m=+872.977421004"
Jul 02 14:30:11.495938 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:11.495897    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.26:8643/healthz\": dial tcp 10.133.0.26:8643: connect: connection refused"
Jul 02 14:30:11.501302 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:11.501269    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:5000: connect: connection refused"
Jul 02 14:30:11.501454 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:11.501405    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"
Jul 02 14:30:11.501706 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:11.501673    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:30:11.501857 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:11.501769    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"
Jul 02 14:30:12.004667 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:12.004634    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"
Jul 02 14:30:12.005072 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:12.004716    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 02 14:30:12.005920 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:12.005895    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:30:13.009327 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:13.009289    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 02 14:30:13.009705 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:13.009677    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:30:14.955335 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:14.955312    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"
Jul 02 14:30:15.020438 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.020406    2582 generic.go:356] "Generic (PLEG): container finished" podID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerID="945a517bde41be8a68ba339b1f0921844b0a5f70578a2163754c2e79c84ea1c0" exitCode=0
Jul 02 14:30:15.020599 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.020478    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"
Jul 02 14:30:15.020599 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.020493    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" event={"ID":"613980ec-8ca3-4796-bee0-688ca252bbf4","Type":"ContainerDied","Data":"945a517bde41be8a68ba339b1f0921844b0a5f70578a2163754c2e79c84ea1c0"}
Jul 02 14:30:15.020599 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.020565    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2" event={"ID":"613980ec-8ca3-4796-bee0-688ca252bbf4","Type":"ContainerDied","Data":"fc88791e1c6de9a6889011a5b91ee8d478b662b60007c0f48ce6ff815e3a91da"}
Jul 02 14:30:15.020599 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.020598    2582 scope.go:117] "RemoveContainer" containerID="945a517bde41be8a68ba339b1f0921844b0a5f70578a2163754c2e79c84ea1c0"
Jul 02 14:30:15.028710 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.028692    2582 scope.go:117] "RemoveContainer" containerID="b8985c1853c40a32574f79d5c00d77de8d641ce03a9625876455a6886cdc207c"
Jul 02 14:30:15.036570 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.036548    2582 scope.go:117] "RemoveContainer" containerID="1e54fde344ccd6dd1475e98ccac3d11b8167b6918a27a033743be99cf712dcfc"
Jul 02 14:30:15.044010 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.043993    2582 scope.go:117] "RemoveContainer" containerID="a8e8f520de3a6ebc42df2d45aadaeabda7449a6e397f2fad461f9ab159da9eab"
Jul 02 14:30:15.051245 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.051229    2582 scope.go:117] "RemoveContainer" containerID="945a517bde41be8a68ba339b1f0921844b0a5f70578a2163754c2e79c84ea1c0"
Jul 02 14:30:15.051484 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:30:15.051465    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"945a517bde41be8a68ba339b1f0921844b0a5f70578a2163754c2e79c84ea1c0\": container with ID starting with 945a517bde41be8a68ba339b1f0921844b0a5f70578a2163754c2e79c84ea1c0 not found: ID does not exist" containerID="945a517bde41be8a68ba339b1f0921844b0a5f70578a2163754c2e79c84ea1c0"
Jul 02 14:30:15.051529 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.051492    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"945a517bde41be8a68ba339b1f0921844b0a5f70578a2163754c2e79c84ea1c0"} err="failed to get container status \"945a517bde41be8a68ba339b1f0921844b0a5f70578a2163754c2e79c84ea1c0\": rpc error: code = NotFound desc = could not find container \"945a517bde41be8a68ba339b1f0921844b0a5f70578a2163754c2e79c84ea1c0\": container with ID starting with 945a517bde41be8a68ba339b1f0921844b0a5f70578a2163754c2e79c84ea1c0 not found: ID does not exist"
Jul 02 14:30:15.051529 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.051512    2582 scope.go:117] "RemoveContainer" containerID="b8985c1853c40a32574f79d5c00d77de8d641ce03a9625876455a6886cdc207c"
Jul 02 14:30:15.051713 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:30:15.051699    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8985c1853c40a32574f79d5c00d77de8d641ce03a9625876455a6886cdc207c\": container with ID starting with b8985c1853c40a32574f79d5c00d77de8d641ce03a9625876455a6886cdc207c not found: ID does not exist" containerID="b8985c1853c40a32574f79d5c00d77de8d641ce03a9625876455a6886cdc207c"
Jul 02 14:30:15.051757 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.051717    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8985c1853c40a32574f79d5c00d77de8d641ce03a9625876455a6886cdc207c"} err="failed to get container status \"b8985c1853c40a32574f79d5c00d77de8d641ce03a9625876455a6886cdc207c\": rpc error: code = NotFound desc = could not find container \"b8985c1853c40a32574f79d5c00d77de8d641ce03a9625876455a6886cdc207c\": container with ID starting with b8985c1853c40a32574f79d5c00d77de8d641ce03a9625876455a6886cdc207c not found: ID does not exist"
Jul 02 14:30:15.051757 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.051729    2582 scope.go:117] "RemoveContainer" containerID="1e54fde344ccd6dd1475e98ccac3d11b8167b6918a27a033743be99cf712dcfc"
Jul 02 14:30:15.051990 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:30:15.051950    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e54fde344ccd6dd1475e98ccac3d11b8167b6918a27a033743be99cf712dcfc\": container with ID starting with 1e54fde344ccd6dd1475e98ccac3d11b8167b6918a27a033743be99cf712dcfc not found: ID does not exist" containerID="1e54fde344ccd6dd1475e98ccac3d11b8167b6918a27a033743be99cf712dcfc"
Jul 02 14:30:15.052032 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.051996    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e54fde344ccd6dd1475e98ccac3d11b8167b6918a27a033743be99cf712dcfc"} err="failed to get container status \"1e54fde344ccd6dd1475e98ccac3d11b8167b6918a27a033743be99cf712dcfc\": rpc error: code = NotFound desc = could not find container \"1e54fde344ccd6dd1475e98ccac3d11b8167b6918a27a033743be99cf712dcfc\": container with ID starting with 1e54fde344ccd6dd1475e98ccac3d11b8167b6918a27a033743be99cf712dcfc not found: ID does not exist"
Jul 02 14:30:15.052032 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.052011    2582 scope.go:117] "RemoveContainer" containerID="a8e8f520de3a6ebc42df2d45aadaeabda7449a6e397f2fad461f9ab159da9eab"
Jul 02 14:30:15.052230 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:30:15.052213    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8e8f520de3a6ebc42df2d45aadaeabda7449a6e397f2fad461f9ab159da9eab\": container with ID starting with a8e8f520de3a6ebc42df2d45aadaeabda7449a6e397f2fad461f9ab159da9eab not found: ID does not exist" containerID="a8e8f520de3a6ebc42df2d45aadaeabda7449a6e397f2fad461f9ab159da9eab"
Jul 02 14:30:15.052287 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.052239    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8e8f520de3a6ebc42df2d45aadaeabda7449a6e397f2fad461f9ab159da9eab"} err="failed to get container status \"a8e8f520de3a6ebc42df2d45aadaeabda7449a6e397f2fad461f9ab159da9eab\": rpc error: code = NotFound desc = could not find container \"a8e8f520de3a6ebc42df2d45aadaeabda7449a6e397f2fad461f9ab159da9eab\": container with ID starting with a8e8f520de3a6ebc42df2d45aadaeabda7449a6e397f2fad461f9ab159da9eab not found: ID does not exist"
Jul 02 14:30:15.099658 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.099620    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/613980ec-8ca3-4796-bee0-688ca252bbf4-kube-rbac-proxy-sar-config\") pod \"613980ec-8ca3-4796-bee0-688ca252bbf4\" (UID: \"613980ec-8ca3-4796-bee0-688ca252bbf4\") "
Jul 02 14:30:15.099827 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.099671    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/613980ec-8ca3-4796-bee0-688ca252bbf4-cabundle-cert\") pod \"613980ec-8ca3-4796-bee0-688ca252bbf4\" (UID: \"613980ec-8ca3-4796-bee0-688ca252bbf4\") "
Jul 02 14:30:15.099827 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.099696    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7f6v\" (UniqueName: \"kubernetes.io/projected/613980ec-8ca3-4796-bee0-688ca252bbf4-kube-api-access-w7f6v\") pod \"613980ec-8ca3-4796-bee0-688ca252bbf4\" (UID: \"613980ec-8ca3-4796-bee0-688ca252bbf4\") "
Jul 02 14:30:15.099827 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.099746    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/613980ec-8ca3-4796-bee0-688ca252bbf4-kserve-provision-location\") pod \"613980ec-8ca3-4796-bee0-688ca252bbf4\" (UID: \"613980ec-8ca3-4796-bee0-688ca252bbf4\") "
Jul 02 14:30:15.099827 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.099802    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/613980ec-8ca3-4796-bee0-688ca252bbf4-proxy-tls\") pod \"613980ec-8ca3-4796-bee0-688ca252bbf4\" (UID: \"613980ec-8ca3-4796-bee0-688ca252bbf4\") "
Jul 02 14:30:15.100138 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.100098    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/613980ec-8ca3-4796-bee0-688ca252bbf4-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "613980ec-8ca3-4796-bee0-688ca252bbf4" (UID: "613980ec-8ca3-4796-bee0-688ca252bbf4"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:30:15.100138 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.100108    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/613980ec-8ca3-4796-bee0-688ca252bbf4-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "613980ec-8ca3-4796-bee0-688ca252bbf4" (UID: "613980ec-8ca3-4796-bee0-688ca252bbf4"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 14:30:15.100138 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.100116    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/613980ec-8ca3-4796-bee0-688ca252bbf4-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "613980ec-8ca3-4796-bee0-688ca252bbf4" (UID: "613980ec-8ca3-4796-bee0-688ca252bbf4"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:30:15.101998 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.101946    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/613980ec-8ca3-4796-bee0-688ca252bbf4-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "613980ec-8ca3-4796-bee0-688ca252bbf4" (UID: "613980ec-8ca3-4796-bee0-688ca252bbf4"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 14:30:15.101998 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.101986    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/613980ec-8ca3-4796-bee0-688ca252bbf4-kube-api-access-w7f6v" (OuterVolumeSpecName: "kube-api-access-w7f6v") pod "613980ec-8ca3-4796-bee0-688ca252bbf4" (UID: "613980ec-8ca3-4796-bee0-688ca252bbf4"). InnerVolumeSpecName "kube-api-access-w7f6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:30:15.200975 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.200933    2582 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/613980ec-8ca3-4796-bee0-688ca252bbf4-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:30:15.200975 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.200981    2582 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/613980ec-8ca3-4796-bee0-688ca252bbf4-cabundle-cert\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:30:15.201165 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.200992    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7f6v\" (UniqueName: \"kubernetes.io/projected/613980ec-8ca3-4796-bee0-688ca252bbf4-kube-api-access-w7f6v\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:30:15.201165 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.201002    2582 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/613980ec-8ca3-4796-bee0-688ca252bbf4-kserve-provision-location\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:30:15.201165 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.201010    2582 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/613980ec-8ca3-4796-bee0-688ca252bbf4-proxy-tls\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:30:15.344309 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.344280    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"]
Jul 02 14:30:15.348402 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:15.348381    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7dbc5fbddb-h9vz2"]
Jul 02 14:30:16.476556 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:16.476512    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" path="/var/lib/kubelet/pods/613980ec-8ca3-4796-bee0-688ca252bbf4/volumes"
Jul 02 14:30:18.013755 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:18.013727    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"
Jul 02 14:30:18.014447 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:18.014421    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 02 14:30:18.014761 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:18.014733    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:30:28.015111 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:28.015064    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 02 14:30:28.015609 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:28.015462    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:30:38.015321 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:38.015276    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 02 14:30:38.015802 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:38.015695    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:30:38.466651 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:38.466623    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnwb5_ca909057-dd63-451d-9c05-b671ca5ae671/ovn-acl-logging/0.log"
Jul 02 14:30:38.468146 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:38.468122    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnwb5_ca909057-dd63-451d-9c05-b671ca5ae671/ovn-acl-logging/0.log"
Jul 02 14:30:48.014334 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:48.014290    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 02 14:30:48.014821 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:48.014796    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:30:58.014357 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:58.014315    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 02 14:30:58.016626 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:30:58.014839    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:31:08.014868 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:08.014824    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 02 14:31:08.015306 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:08.015224    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:31:18.014745 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:18.014697    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 02 14:31:18.015271 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:18.015246    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:31:21.470041 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:21.470002    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 02 14:31:21.470458 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:21.470386    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:31:31.471222 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:31.471187    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"
Jul 02 14:31:31.471715 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:31.471692    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"
Jul 02 14:31:39.840543 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:39.840510    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_message-dumper-predictor-6575bcfdfd-8hkxs_512aab7b-05bb-44db-b212-915d9a67de43/kserve-container/0.log"
Jul 02 14:31:40.344375 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.344342    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw"]
Jul 02 14:31:40.344677 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.344665    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="kube-rbac-proxy"
Jul 02 14:31:40.344754 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.344678    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="kube-rbac-proxy"
Jul 02 14:31:40.344754 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.344693    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="kserve-container"
Jul 02 14:31:40.344754 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.344698    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="kserve-container"
Jul 02 14:31:40.344754 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.344707    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="agent"
Jul 02 14:31:40.344754 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.344712    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="agent"
Jul 02 14:31:40.344754 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.344728    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="storage-initializer"
Jul 02 14:31:40.344754 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.344735    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="storage-initializer"
Jul 02 14:31:40.345076 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.344812    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="agent"
Jul 02 14:31:40.345076 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.344826    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="kube-rbac-proxy"
Jul 02 14:31:40.345076 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.344833    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="613980ec-8ca3-4796-bee0-688ca252bbf4" containerName="kserve-container"
Jul 02 14:31:40.347931 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.347914    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw"
Jul 02 14:31:40.353209 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.353188    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v1-predictor-serving-cert\""
Jul 02 14:31:40.353629 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.353605    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v1-kube-rbac-proxy-sar-config\""
Jul 02 14:31:40.372538 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.372506    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw"]
Jul 02 14:31:40.381483 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.381454    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/51d42ab1-c9d6-4b22-9654-9c973dcc4692-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw\" (UID: \"51d42ab1-c9d6-4b22-9654-9c973dcc4692\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw"
Jul 02 14:31:40.381483 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.381484    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/51d42ab1-c9d6-4b22-9654-9c973dcc4692-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw\" (UID: \"51d42ab1-c9d6-4b22-9654-9c973dcc4692\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw"
Jul 02 14:31:40.381731 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.381503    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwg9v\" (UniqueName: \"kubernetes.io/projected/51d42ab1-c9d6-4b22-9654-9c973dcc4692-kube-api-access-pwg9v\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw\" (UID: \"51d42ab1-c9d6-4b22-9654-9c973dcc4692\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw"
Jul 02 14:31:40.381731 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.381579    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/51d42ab1-c9d6-4b22-9654-9c973dcc4692-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw\" (UID: \"51d42ab1-c9d6-4b22-9654-9c973dcc4692\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw"
Jul 02 14:31:40.381731 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.381618    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/51d42ab1-c9d6-4b22-9654-9c973dcc4692-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw\" (UID: \"51d42ab1-c9d6-4b22-9654-9c973dcc4692\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw"
Jul 02 14:31:40.444751 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.444712    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"]
Jul 02 14:31:40.445028 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.444990    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="kserve-container" containerID="cri-o://eab3a544988799f956d4c618044de09fefdd9ae26d2e107517841b980795a2f9" gracePeriod=30
Jul 02 14:31:40.445173 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.445034    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="kube-rbac-proxy" containerID="cri-o://5c54dca13af5034b1a6c6e7fa3d7b02d2e6084da428ac77047a99589d6daddfd" gracePeriod=30
Jul 02 14:31:40.445173 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.445033    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="agent" containerID="cri-o://dbf5451a7d73c8c9c208cd73aa299389e23bc13253a1736bd98761b5d88aa2b9" gracePeriod=30
Jul 02 14:31:40.481126 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.481090    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs"]
Jul 02 14:31:40.481367 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.481323    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs" podUID="512aab7b-05bb-44db-b212-915d9a67de43" containerName="kserve-container" containerID="cri-o://a4ec224801b1b26de11f656ef821155f39cdbb77b5533d050289627167680923" gracePeriod=30
Jul 02 14:31:40.481484 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.481357    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs" podUID="512aab7b-05bb-44db-b212-915d9a67de43" containerName="kube-rbac-proxy" containerID="cri-o://7adc1a55c856f5766f7d935794930e847af19d1d88fb1242398c59be30d80f99" gracePeriod=30
Jul 02 14:31:40.482087 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.482059    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/51d42ab1-c9d6-4b22-9654-9c973dcc4692-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw\" (UID: \"51d42ab1-c9d6-4b22-9654-9c973dcc4692\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw"
Jul 02 14:31:40.482176 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.482139    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/51d42ab1-c9d6-4b22-9654-9c973dcc4692-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw\" (UID: \"51d42ab1-c9d6-4b22-9654-9c973dcc4692\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw"
Jul 02 14:31:40.482176 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.482168    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/51d42ab1-c9d6-4b22-9654-9c973dcc4692-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw\" (UID: \"51d42ab1-c9d6-4b22-9654-9c973dcc4692\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw"
Jul 02 14:31:40.482397 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.482193    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwg9v\" (UniqueName: \"kubernetes.io/projected/51d42ab1-c9d6-4b22-9654-9c973dcc4692-kube-api-access-pwg9v\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw\" (UID: \"51d42ab1-c9d6-4b22-9654-9c973dcc4692\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw"
Jul 02 14:31:40.482397 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.482255    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/51d42ab1-c9d6-4b22-9654-9c973dcc4692-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw\" (UID: \"51d42ab1-c9d6-4b22-9654-9c973dcc4692\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw"
Jul 02 14:31:40.482704 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.482678    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/51d42ab1-c9d6-4b22-9654-9c973dcc4692-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw\" (UID: \"51d42ab1-c9d6-4b22-9654-9c973dcc4692\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw"
Jul 02 14:31:40.483246 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.483219    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/51d42ab1-c9d6-4b22-9654-9c973dcc4692-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw\" (UID: \"51d42ab1-c9d6-4b22-9654-9c973dcc4692\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw"
Jul 02 14:31:40.483351 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.483271    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/51d42ab1-c9d6-4b22-9654-9c973dcc4692-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw\" (UID: \"51d42ab1-c9d6-4b22-9654-9c973dcc4692\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw"
Jul 02 14:31:40.485425 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.485245    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/51d42ab1-c9d6-4b22-9654-9c973dcc4692-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw\" (UID: \"51d42ab1-c9d6-4b22-9654-9c973dcc4692\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw"
Jul 02 14:31:40.498225 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.498205    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwg9v\" (UniqueName: \"kubernetes.io/projected/51d42ab1-c9d6-4b22-9654-9c973dcc4692-kube-api-access-pwg9v\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw\" (UID: \"51d42ab1-c9d6-4b22-9654-9c973dcc4692\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw"
Jul 02 14:31:40.660513 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.660468    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw"
Jul 02 14:31:40.731800 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.731699    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs"
Jul 02 14:31:40.784151 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.784070    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/512aab7b-05bb-44db-b212-915d9a67de43-proxy-tls\") pod \"512aab7b-05bb-44db-b212-915d9a67de43\" (UID: \"512aab7b-05bb-44db-b212-915d9a67de43\") "
Jul 02 14:31:40.784151 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.784130    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-852pn\" (UniqueName: \"kubernetes.io/projected/512aab7b-05bb-44db-b212-915d9a67de43-kube-api-access-852pn\") pod \"512aab7b-05bb-44db-b212-915d9a67de43\" (UID: \"512aab7b-05bb-44db-b212-915d9a67de43\") "
Jul 02 14:31:40.784442 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.784194    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/512aab7b-05bb-44db-b212-915d9a67de43-kube-rbac-proxy-sar-config\") pod \"512aab7b-05bb-44db-b212-915d9a67de43\" (UID: \"512aab7b-05bb-44db-b212-915d9a67de43\") "
Jul 02 14:31:40.784650 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.784621    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/512aab7b-05bb-44db-b212-915d9a67de43-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "512aab7b-05bb-44db-b212-915d9a67de43" (UID: "512aab7b-05bb-44db-b212-915d9a67de43"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:31:40.786675 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.786645    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/512aab7b-05bb-44db-b212-915d9a67de43-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "512aab7b-05bb-44db-b212-915d9a67de43" (UID: "512aab7b-05bb-44db-b212-915d9a67de43"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 14:31:40.786675 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.786661    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/512aab7b-05bb-44db-b212-915d9a67de43-kube-api-access-852pn" (OuterVolumeSpecName: "kube-api-access-852pn") pod "512aab7b-05bb-44db-b212-915d9a67de43" (UID: "512aab7b-05bb-44db-b212-915d9a67de43"). InnerVolumeSpecName "kube-api-access-852pn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:31:40.807014 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.806991    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw"]
Jul 02 14:31:40.809587 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:31:40.809557    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51d42ab1_c9d6_4b22_9654_9c973dcc4692.slice/crio-bcb3c7bc7de1d424a6f1501486c62ed886498c1547f499c71205acb04193a8d5 WatchSource:0}: Error finding container bcb3c7bc7de1d424a6f1501486c62ed886498c1547f499c71205acb04193a8d5: Status 404 returned error can't find the container with id bcb3c7bc7de1d424a6f1501486c62ed886498c1547f499c71205acb04193a8d5
Jul 02 14:31:40.887091 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.887050    2582 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/512aab7b-05bb-44db-b212-915d9a67de43-proxy-tls\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:31:40.887091 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.887088    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-852pn\" (UniqueName: \"kubernetes.io/projected/512aab7b-05bb-44db-b212-915d9a67de43-kube-api-access-852pn\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:31:40.887566 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:40.887109    2582 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/512aab7b-05bb-44db-b212-915d9a67de43-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:31:41.394483 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:41.394442    2582 generic.go:356] "Generic (PLEG): container finished" podID="512aab7b-05bb-44db-b212-915d9a67de43" containerID="7adc1a55c856f5766f7d935794930e847af19d1d88fb1242398c59be30d80f99" exitCode=2
Jul 02 14:31:41.394483 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:41.394473    2582 generic.go:356] "Generic (PLEG): container finished" podID="512aab7b-05bb-44db-b212-915d9a67de43" containerID="a4ec224801b1b26de11f656ef821155f39cdbb77b5533d050289627167680923" exitCode=2
Jul 02 14:31:41.394761 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:41.394529    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs"
Jul 02 14:31:41.394761 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:41.394529    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs" event={"ID":"512aab7b-05bb-44db-b212-915d9a67de43","Type":"ContainerDied","Data":"7adc1a55c856f5766f7d935794930e847af19d1d88fb1242398c59be30d80f99"}
Jul 02 14:31:41.394761 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:41.394631    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs" event={"ID":"512aab7b-05bb-44db-b212-915d9a67de43","Type":"ContainerDied","Data":"a4ec224801b1b26de11f656ef821155f39cdbb77b5533d050289627167680923"}
Jul 02 14:31:41.394761 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:41.394659    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs" event={"ID":"512aab7b-05bb-44db-b212-915d9a67de43","Type":"ContainerDied","Data":"0353ffda245d36ae0539b2fcec839b81df1a43b03ae53b17435d225c3fba02fc"}
Jul 02 14:31:41.394761 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:41.394697    2582 scope.go:117] "RemoveContainer" containerID="7adc1a55c856f5766f7d935794930e847af19d1d88fb1242398c59be30d80f99"
Jul 02 14:31:41.396322 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:41.396255    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw" event={"ID":"51d42ab1-c9d6-4b22-9654-9c973dcc4692","Type":"ContainerStarted","Data":"253eff4945778e9c4c11ac37b4c8039bf8afb9eb5bbd07b38776ee8b3205ac92"}
Jul 02 14:31:41.396322 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:41.396318    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw" event={"ID":"51d42ab1-c9d6-4b22-9654-9c973dcc4692","Type":"ContainerStarted","Data":"bcb3c7bc7de1d424a6f1501486c62ed886498c1547f499c71205acb04193a8d5"}
Jul 02 14:31:41.398912 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:41.398887    2582 generic.go:356] "Generic (PLEG): container finished" podID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerID="5c54dca13af5034b1a6c6e7fa3d7b02d2e6084da428ac77047a99589d6daddfd" exitCode=2
Jul 02 14:31:41.399026 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:41.398933    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" event={"ID":"78697d7d-fbc6-4f1b-8754-f028b990e717","Type":"ContainerDied","Data":"5c54dca13af5034b1a6c6e7fa3d7b02d2e6084da428ac77047a99589d6daddfd"}
Jul 02 14:31:41.404920 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:41.404883    2582 scope.go:117] "RemoveContainer" containerID="a4ec224801b1b26de11f656ef821155f39cdbb77b5533d050289627167680923"
Jul 02 14:31:41.414095 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:41.414077    2582 scope.go:117] "RemoveContainer" containerID="7adc1a55c856f5766f7d935794930e847af19d1d88fb1242398c59be30d80f99"
Jul 02 14:31:41.414333 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:31:41.414312    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7adc1a55c856f5766f7d935794930e847af19d1d88fb1242398c59be30d80f99\": container with ID starting with 7adc1a55c856f5766f7d935794930e847af19d1d88fb1242398c59be30d80f99 not found: ID does not exist" containerID="7adc1a55c856f5766f7d935794930e847af19d1d88fb1242398c59be30d80f99"
Jul 02 14:31:41.414396 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:41.414341    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7adc1a55c856f5766f7d935794930e847af19d1d88fb1242398c59be30d80f99"} err="failed to get container status \"7adc1a55c856f5766f7d935794930e847af19d1d88fb1242398c59be30d80f99\": rpc error: code = NotFound desc = could not find container \"7adc1a55c856f5766f7d935794930e847af19d1d88fb1242398c59be30d80f99\": container with ID starting with 7adc1a55c856f5766f7d935794930e847af19d1d88fb1242398c59be30d80f99 not found: ID does not exist"
Jul 02 14:31:41.414396 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:41.414358    2582 scope.go:117] "RemoveContainer" containerID="a4ec224801b1b26de11f656ef821155f39cdbb77b5533d050289627167680923"
Jul 02 14:31:41.414582 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:31:41.414565    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4ec224801b1b26de11f656ef821155f39cdbb77b5533d050289627167680923\": container with ID starting with a4ec224801b1b26de11f656ef821155f39cdbb77b5533d050289627167680923 not found: ID does not exist" containerID="a4ec224801b1b26de11f656ef821155f39cdbb77b5533d050289627167680923"
Jul 02 14:31:41.414632 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:41.414596    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4ec224801b1b26de11f656ef821155f39cdbb77b5533d050289627167680923"} err="failed to get container status \"a4ec224801b1b26de11f656ef821155f39cdbb77b5533d050289627167680923\": rpc error: code = NotFound desc = could not find container \"a4ec224801b1b26de11f656ef821155f39cdbb77b5533d050289627167680923\": container with ID starting with a4ec224801b1b26de11f656ef821155f39cdbb77b5533d050289627167680923 not found: ID does not exist"
Jul 02 14:31:41.414632 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:41.414613    2582 scope.go:117] "RemoveContainer" containerID="7adc1a55c856f5766f7d935794930e847af19d1d88fb1242398c59be30d80f99"
Jul 02 14:31:41.414834 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:41.414816    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7adc1a55c856f5766f7d935794930e847af19d1d88fb1242398c59be30d80f99"} err="failed to get container status \"7adc1a55c856f5766f7d935794930e847af19d1d88fb1242398c59be30d80f99\": rpc error: code = NotFound desc = could not find container \"7adc1a55c856f5766f7d935794930e847af19d1d88fb1242398c59be30d80f99\": container with ID starting with 7adc1a55c856f5766f7d935794930e847af19d1d88fb1242398c59be30d80f99 not found: ID does not exist"
Jul 02 14:31:41.414881 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:41.414834    2582 scope.go:117] "RemoveContainer" containerID="a4ec224801b1b26de11f656ef821155f39cdbb77b5533d050289627167680923"
Jul 02 14:31:41.415042 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:41.415024    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4ec224801b1b26de11f656ef821155f39cdbb77b5533d050289627167680923"} err="failed to get container status \"a4ec224801b1b26de11f656ef821155f39cdbb77b5533d050289627167680923\": rpc error: code = NotFound desc = could not find container \"a4ec224801b1b26de11f656ef821155f39cdbb77b5533d050289627167680923\": container with ID starting with a4ec224801b1b26de11f656ef821155f39cdbb77b5533d050289627167680923 not found: ID does not exist"
Jul 02 14:31:41.432917 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:41.432894    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs"]
Jul 02 14:31:41.436585 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:41.436555    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-8hkxs"]
Jul 02 14:31:41.470182 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:41.470130    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 02 14:31:41.472887 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:41.472856    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:31:42.475275 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:42.475244    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="512aab7b-05bb-44db-b212-915d9a67de43" path="/var/lib/kubelet/pods/512aab7b-05bb-44db-b212-915d9a67de43/volumes"
Jul 02 14:31:43.009819 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:43.009776    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.28:8643/healthz\": dial tcp 10.133.0.28:8643: connect: connection refused"
Jul 02 14:31:45.419173 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:45.419142    2582 generic.go:356] "Generic (PLEG): container finished" podID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerID="eab3a544988799f956d4c618044de09fefdd9ae26d2e107517841b980795a2f9" exitCode=0
Jul 02 14:31:45.419173 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:45.419184    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" event={"ID":"78697d7d-fbc6-4f1b-8754-f028b990e717","Type":"ContainerDied","Data":"eab3a544988799f956d4c618044de09fefdd9ae26d2e107517841b980795a2f9"}
Jul 02 14:31:46.425014 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:46.424943    2582 generic.go:356] "Generic (PLEG): container finished" podID="51d42ab1-c9d6-4b22-9654-9c973dcc4692" containerID="253eff4945778e9c4c11ac37b4c8039bf8afb9eb5bbd07b38776ee8b3205ac92" exitCode=0
Jul 02 14:31:46.425410 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:46.425026    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw" event={"ID":"51d42ab1-c9d6-4b22-9654-9c973dcc4692","Type":"ContainerDied","Data":"253eff4945778e9c4c11ac37b4c8039bf8afb9eb5bbd07b38776ee8b3205ac92"}
Jul 02 14:31:46.426417 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:46.426402    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 14:31:48.009768 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:48.009717    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.28:8643/healthz\": dial tcp 10.133.0.28:8643: connect: connection refused"
Jul 02 14:31:51.470021 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:51.469983    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 02 14:31:51.472752 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:51.472720    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:31:53.009852 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:53.009808    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.28:8643/healthz\": dial tcp 10.133.0.28:8643: connect: connection refused"
Jul 02 14:31:53.010349 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:53.009940    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"
Jul 02 14:31:58.009926 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:31:58.009879    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.28:8643/healthz\": dial tcp 10.133.0.28:8643: connect: connection refused"
Jul 02 14:32:01.470320 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:01.470226    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Jul 02 14:32:01.470719 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:01.470413    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"
Jul 02 14:32:01.473005 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:01.472948    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 14:32:01.473163 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:01.473104    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"
Jul 02 14:32:03.009779 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:03.009734    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.28:8643/healthz\": dial tcp 10.133.0.28:8643: connect: connection refused"
Jul 02 14:32:08.010013 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:08.009946    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.28:8643/healthz\": dial tcp 10.133.0.28:8643: connect: connection refused"
Jul 02 14:32:10.544472 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:10.544441    2582 generic.go:356] "Generic (PLEG): container finished" podID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerID="dbf5451a7d73c8c9c208cd73aa299389e23bc13253a1736bd98761b5d88aa2b9" exitCode=0
Jul 02 14:32:10.544831 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:10.544498    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" event={"ID":"78697d7d-fbc6-4f1b-8754-f028b990e717","Type":"ContainerDied","Data":"dbf5451a7d73c8c9c208cd73aa299389e23bc13253a1736bd98761b5d88aa2b9"}
Jul 02 14:32:10.613593 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:10.613567    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"
Jul 02 14:32:10.745158 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:10.745121    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fpbg9\" (UniqueName: \"kubernetes.io/projected/78697d7d-fbc6-4f1b-8754-f028b990e717-kube-api-access-fpbg9\") pod \"78697d7d-fbc6-4f1b-8754-f028b990e717\" (UID: \"78697d7d-fbc6-4f1b-8754-f028b990e717\") "
Jul 02 14:32:10.745368 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:10.745177    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/78697d7d-fbc6-4f1b-8754-f028b990e717-cabundle-cert\") pod \"78697d7d-fbc6-4f1b-8754-f028b990e717\" (UID: \"78697d7d-fbc6-4f1b-8754-f028b990e717\") "
Jul 02 14:32:10.745368 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:10.745259    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/78697d7d-fbc6-4f1b-8754-f028b990e717-kserve-provision-location\") pod \"78697d7d-fbc6-4f1b-8754-f028b990e717\" (UID: \"78697d7d-fbc6-4f1b-8754-f028b990e717\") "
Jul 02 14:32:10.745368 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:10.745285    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/78697d7d-fbc6-4f1b-8754-f028b990e717-proxy-tls\") pod \"78697d7d-fbc6-4f1b-8754-f028b990e717\" (UID: \"78697d7d-fbc6-4f1b-8754-f028b990e717\") "
Jul 02 14:32:10.745368 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:10.745368    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/78697d7d-fbc6-4f1b-8754-f028b990e717-kube-rbac-proxy-sar-config\") pod \"78697d7d-fbc6-4f1b-8754-f028b990e717\" (UID: \"78697d7d-fbc6-4f1b-8754-f028b990e717\") "
Jul 02 14:32:10.745778 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:10.745703    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78697d7d-fbc6-4f1b-8754-f028b990e717-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "78697d7d-fbc6-4f1b-8754-f028b990e717" (UID: "78697d7d-fbc6-4f1b-8754-f028b990e717"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 14:32:10.745778 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:10.745740    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78697d7d-fbc6-4f1b-8754-f028b990e717-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "78697d7d-fbc6-4f1b-8754-f028b990e717" (UID: "78697d7d-fbc6-4f1b-8754-f028b990e717"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:32:10.745916 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:10.745831    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78697d7d-fbc6-4f1b-8754-f028b990e717-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "78697d7d-fbc6-4f1b-8754-f028b990e717" (UID: "78697d7d-fbc6-4f1b-8754-f028b990e717"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:32:10.747789 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:10.747757    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78697d7d-fbc6-4f1b-8754-f028b990e717-kube-api-access-fpbg9" (OuterVolumeSpecName: "kube-api-access-fpbg9") pod "78697d7d-fbc6-4f1b-8754-f028b990e717" (UID: "78697d7d-fbc6-4f1b-8754-f028b990e717"). InnerVolumeSpecName "kube-api-access-fpbg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:32:10.747944 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:10.747921    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78697d7d-fbc6-4f1b-8754-f028b990e717-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "78697d7d-fbc6-4f1b-8754-f028b990e717" (UID: "78697d7d-fbc6-4f1b-8754-f028b990e717"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 14:32:10.846367 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:10.846321    2582 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/78697d7d-fbc6-4f1b-8754-f028b990e717-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:32:10.846367 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:10.846361    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpbg9\" (UniqueName: \"kubernetes.io/projected/78697d7d-fbc6-4f1b-8754-f028b990e717-kube-api-access-fpbg9\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:32:10.846367 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:10.846372    2582 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/78697d7d-fbc6-4f1b-8754-f028b990e717-cabundle-cert\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:32:10.846662 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:10.846381    2582 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/78697d7d-fbc6-4f1b-8754-f028b990e717-kserve-provision-location\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:32:10.846662 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:10.846393    2582 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/78697d7d-fbc6-4f1b-8754-f028b990e717-proxy-tls\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:32:11.552773 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:11.552746    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"
Jul 02 14:32:11.552773 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:11.552761    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz" event={"ID":"78697d7d-fbc6-4f1b-8754-f028b990e717","Type":"ContainerDied","Data":"b25ca376fd465b21a575a2679473d851d70982e3b9061a60f8258e49841a38bd"}
Jul 02 14:32:11.553325 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:11.552834    2582 scope.go:117] "RemoveContainer" containerID="dbf5451a7d73c8c9c208cd73aa299389e23bc13253a1736bd98761b5d88aa2b9"
Jul 02 14:32:11.564257 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:11.564238    2582 scope.go:117] "RemoveContainer" containerID="5c54dca13af5034b1a6c6e7fa3d7b02d2e6084da428ac77047a99589d6daddfd"
Jul 02 14:32:11.573793 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:11.573766    2582 scope.go:117] "RemoveContainer" containerID="eab3a544988799f956d4c618044de09fefdd9ae26d2e107517841b980795a2f9"
Jul 02 14:32:11.581152 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:11.581108    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"]
Jul 02 14:32:11.584159 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:11.584134    2582 scope.go:117] "RemoveContainer" containerID="bcfce565aa9f1aeaa6c74a10ef4d641f62cc52ad6237357d017f34d2232e9f90"
Jul 02 14:32:11.587410 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:11.587383    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-756bf5f9fb-28ssz"]
Jul 02 14:32:12.476367 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:32:12.476334    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" path="/var/lib/kubelet/pods/78697d7d-fbc6-4f1b-8754-f028b990e717/volumes"
Jul 02 14:34:07.095782 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:07.095692    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw" event={"ID":"51d42ab1-c9d6-4b22-9654-9c973dcc4692","Type":"ContainerStarted","Data":"6dd97d2828c25c670258208d0cd785bb370113cded00b60347008163ed9247ed"}
Jul 02 14:34:07.095782 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:07.095763    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw"
Jul 02 14:34:07.096326 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:07.095787    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw" event={"ID":"51d42ab1-c9d6-4b22-9654-9c973dcc4692","Type":"ContainerStarted","Data":"5d48f1c7daffa538f4719de3ed88653b1db866cd077accd830cc61d28c67c0dc"}
Jul 02 14:34:07.096326 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:07.095809    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw"
Jul 02 14:34:07.097468 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:07.097441    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw" podUID="51d42ab1-c9d6-4b22-9654-9c973dcc4692" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 02 14:34:07.127846 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:07.127801    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw" podStartSLOduration=2.153738868 podStartE2EDuration="2m27.127774693s" totalImagesPullingTime="2m20.378457374s" totalInitContainerRuntime="4.595578451s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:31:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:34:07.127692277 +0000 UTC m=+1109.079029923" watchObservedRunningTime="2026-07-02 14:34:07.127774693 +0000 UTC m=+1109.079112321"
Jul 02 14:34:08.099794 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:08.099750    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw" podUID="51d42ab1-c9d6-4b22-9654-9c973dcc4692" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 02 14:34:13.104778 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:13.104744    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw"
Jul 02 14:34:13.105378 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:13.105348    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw" podUID="51d42ab1-c9d6-4b22-9654-9c973dcc4692" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Jul 02 14:34:23.105896 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:23.105865    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw"
Jul 02 14:34:30.338029 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:30.337998    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw"]
Jul 02 14:34:30.338493 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:30.338222    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw" podUID="51d42ab1-c9d6-4b22-9654-9c973dcc4692" containerName="kserve-container" containerID="cri-o://5d48f1c7daffa538f4719de3ed88653b1db866cd077accd830cc61d28c67c0dc" gracePeriod=30
Jul 02 14:34:30.338493 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:30.338252    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw" podUID="51d42ab1-c9d6-4b22-9654-9c973dcc4692" containerName="kube-rbac-proxy" containerID="cri-o://6dd97d2828c25c670258208d0cd785bb370113cded00b60347008163ed9247ed" gracePeriod=30
Jul 02 14:34:31.196304 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:31.196278    2582 generic.go:356] "Generic (PLEG): container finished" podID="51d42ab1-c9d6-4b22-9654-9c973dcc4692" containerID="6dd97d2828c25c670258208d0cd785bb370113cded00b60347008163ed9247ed" exitCode=2
Jul 02 14:34:31.196304 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:31.196301    2582 generic.go:356] "Generic (PLEG): container finished" podID="51d42ab1-c9d6-4b22-9654-9c973dcc4692" containerID="5d48f1c7daffa538f4719de3ed88653b1db866cd077accd830cc61d28c67c0dc" exitCode=0
Jul 02 14:34:31.196462 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:31.196348    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw" event={"ID":"51d42ab1-c9d6-4b22-9654-9c973dcc4692","Type":"ContainerDied","Data":"6dd97d2828c25c670258208d0cd785bb370113cded00b60347008163ed9247ed"}
Jul 02 14:34:31.196462 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:31.196402    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw" event={"ID":"51d42ab1-c9d6-4b22-9654-9c973dcc4692","Type":"ContainerDied","Data":"5d48f1c7daffa538f4719de3ed88653b1db866cd077accd830cc61d28c67c0dc"}
Jul 02 14:34:31.196462 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:31.196422    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw" event={"ID":"51d42ab1-c9d6-4b22-9654-9c973dcc4692","Type":"ContainerDied","Data":"bcb3c7bc7de1d424a6f1501486c62ed886498c1547f499c71205acb04193a8d5"}
Jul 02 14:34:31.196462 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:31.196440    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bcb3c7bc7de1d424a6f1501486c62ed886498c1547f499c71205acb04193a8d5"
Jul 02 14:34:31.218639 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:31.218587    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw"
Jul 02 14:34:31.319380 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:31.319329    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/51d42ab1-c9d6-4b22-9654-9c973dcc4692-cabundle-cert\") pod \"51d42ab1-c9d6-4b22-9654-9c973dcc4692\" (UID: \"51d42ab1-c9d6-4b22-9654-9c973dcc4692\") "
Jul 02 14:34:31.319570 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:31.319394    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/51d42ab1-c9d6-4b22-9654-9c973dcc4692-proxy-tls\") pod \"51d42ab1-c9d6-4b22-9654-9c973dcc4692\" (UID: \"51d42ab1-c9d6-4b22-9654-9c973dcc4692\") "
Jul 02 14:34:31.319570 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:31.319417    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/51d42ab1-c9d6-4b22-9654-9c973dcc4692-kube-rbac-proxy-sar-config\") pod \"51d42ab1-c9d6-4b22-9654-9c973dcc4692\" (UID: \"51d42ab1-c9d6-4b22-9654-9c973dcc4692\") "
Jul 02 14:34:31.319570 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:31.319471    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/51d42ab1-c9d6-4b22-9654-9c973dcc4692-kserve-provision-location\") pod \"51d42ab1-c9d6-4b22-9654-9c973dcc4692\" (UID: \"51d42ab1-c9d6-4b22-9654-9c973dcc4692\") "
Jul 02 14:34:31.319570 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:31.319569    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwg9v\" (UniqueName: \"kubernetes.io/projected/51d42ab1-c9d6-4b22-9654-9c973dcc4692-kube-api-access-pwg9v\") pod \"51d42ab1-c9d6-4b22-9654-9c973dcc4692\" (UID: \"51d42ab1-c9d6-4b22-9654-9c973dcc4692\") "
Jul 02 14:34:31.319929 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:31.319894    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51d42ab1-c9d6-4b22-9654-9c973dcc4692-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "51d42ab1-c9d6-4b22-9654-9c973dcc4692" (UID: "51d42ab1-c9d6-4b22-9654-9c973dcc4692"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:34:31.320028 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:31.319901    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51d42ab1-c9d6-4b22-9654-9c973dcc4692-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "51d42ab1-c9d6-4b22-9654-9c973dcc4692" (UID: "51d42ab1-c9d6-4b22-9654-9c973dcc4692"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:34:31.321798 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:31.321769    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51d42ab1-c9d6-4b22-9654-9c973dcc4692-kube-api-access-pwg9v" (OuterVolumeSpecName: "kube-api-access-pwg9v") pod "51d42ab1-c9d6-4b22-9654-9c973dcc4692" (UID: "51d42ab1-c9d6-4b22-9654-9c973dcc4692"). InnerVolumeSpecName "kube-api-access-pwg9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:34:31.321798 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:31.321782    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51d42ab1-c9d6-4b22-9654-9c973dcc4692-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "51d42ab1-c9d6-4b22-9654-9c973dcc4692" (UID: "51d42ab1-c9d6-4b22-9654-9c973dcc4692"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 14:34:31.323846 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:31.323818    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51d42ab1-c9d6-4b22-9654-9c973dcc4692-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "51d42ab1-c9d6-4b22-9654-9c973dcc4692" (UID: "51d42ab1-c9d6-4b22-9654-9c973dcc4692"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 14:34:31.420978 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:31.420920    2582 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/51d42ab1-c9d6-4b22-9654-9c973dcc4692-cabundle-cert\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:34:31.420978 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:31.420977    2582 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/51d42ab1-c9d6-4b22-9654-9c973dcc4692-proxy-tls\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:34:31.421383 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:31.420990    2582 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/51d42ab1-c9d6-4b22-9654-9c973dcc4692-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:34:31.421383 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:31.421003    2582 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/51d42ab1-c9d6-4b22-9654-9c973dcc4692-kserve-provision-location\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:34:31.421383 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:31.421016    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwg9v\" (UniqueName: \"kubernetes.io/projected/51d42ab1-c9d6-4b22-9654-9c973dcc4692-kube-api-access-pwg9v\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:34:32.201413 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:32.201381    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw"
Jul 02 14:34:32.224181 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:32.224144    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw"]
Jul 02 14:34:32.229740 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:32.229714    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-d2dgw"]
Jul 02 14:34:32.475789 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:32.475760    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51d42ab1-c9d6-4b22-9654-9c973dcc4692" path="/var/lib/kubelet/pods/51d42ab1-c9d6-4b22-9654-9c973dcc4692/volumes"
Jul 02 14:34:35.620465 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.620430    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f"]
Jul 02 14:34:35.620846 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.620829    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="kserve-container"
Jul 02 14:34:35.620903 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.620851    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="kserve-container"
Jul 02 14:34:35.620903 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.620871    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="storage-initializer"
Jul 02 14:34:35.620903 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.620877    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="storage-initializer"
Jul 02 14:34:35.620903 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.620888    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="kube-rbac-proxy"
Jul 02 14:34:35.620903 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.620896    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="kube-rbac-proxy"
Jul 02 14:34:35.620903 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.620906    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="agent"
Jul 02 14:34:35.621222 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.620912    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="agent"
Jul 02 14:34:35.621222 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.620921    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="512aab7b-05bb-44db-b212-915d9a67de43" containerName="kserve-container"
Jul 02 14:34:35.621222 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.620927    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="512aab7b-05bb-44db-b212-915d9a67de43" containerName="kserve-container"
Jul 02 14:34:35.621222 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.620937    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51d42ab1-c9d6-4b22-9654-9c973dcc4692" containerName="kserve-container"
Jul 02 14:34:35.621222 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.620946    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="51d42ab1-c9d6-4b22-9654-9c973dcc4692" containerName="kserve-container"
Jul 02 14:34:35.621222 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.620972    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="512aab7b-05bb-44db-b212-915d9a67de43" containerName="kube-rbac-proxy"
Jul 02 14:34:35.621222 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.620980    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="512aab7b-05bb-44db-b212-915d9a67de43" containerName="kube-rbac-proxy"
Jul 02 14:34:35.621222 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.620992    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51d42ab1-c9d6-4b22-9654-9c973dcc4692" containerName="kube-rbac-proxy"
Jul 02 14:34:35.621222 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.621000    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="51d42ab1-c9d6-4b22-9654-9c973dcc4692" containerName="kube-rbac-proxy"
Jul 02 14:34:35.621222 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.621007    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51d42ab1-c9d6-4b22-9654-9c973dcc4692" containerName="storage-initializer"
Jul 02 14:34:35.621222 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.621012    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="51d42ab1-c9d6-4b22-9654-9c973dcc4692" containerName="storage-initializer"
Jul 02 14:34:35.621222 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.621071    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="kserve-container"
Jul 02 14:34:35.621222 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.621081    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="512aab7b-05bb-44db-b212-915d9a67de43" containerName="kube-rbac-proxy"
Jul 02 14:34:35.621222 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.621087    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="agent"
Jul 02 14:34:35.621222 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.621093    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="512aab7b-05bb-44db-b212-915d9a67de43" containerName="kserve-container"
Jul 02 14:34:35.621222 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.621102    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="51d42ab1-c9d6-4b22-9654-9c973dcc4692" containerName="kserve-container"
Jul 02 14:34:35.621222 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.621111    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="78697d7d-fbc6-4f1b-8754-f028b990e717" containerName="kube-rbac-proxy"
Jul 02 14:34:35.621222 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.621119    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="51d42ab1-c9d6-4b22-9654-9c973dcc4692" containerName="kube-rbac-proxy"
Jul 02 14:34:35.629496 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.629479    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f"
Jul 02 14:34:35.632417 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.632383    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-kube-rbac-proxy-sar-config\""
Jul 02 14:34:35.632537 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.632488    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 02 14:34:35.633329 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.633310    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 02 14:34:35.633329 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.633324    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 02 14:34:35.633552 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.633368    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 02 14:34:35.633552 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.633418    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-predictor-serving-cert\""
Jul 02 14:34:35.633552 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.633510    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-7rr57\""
Jul 02 14:34:35.634119 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.634100    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f"]
Jul 02 14:34:35.753967 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.753933    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a9eaf80e-5300-4198-956e-0f00b1984d04-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-mgc9f\" (UID: \"a9eaf80e-5300-4198-956e-0f00b1984d04\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f"
Jul 02 14:34:35.754144 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.753999    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a9eaf80e-5300-4198-956e-0f00b1984d04-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-mgc9f\" (UID: \"a9eaf80e-5300-4198-956e-0f00b1984d04\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f"
Jul 02 14:34:35.754144 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.754064    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a9eaf80e-5300-4198-956e-0f00b1984d04-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-mgc9f\" (UID: \"a9eaf80e-5300-4198-956e-0f00b1984d04\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f"
Jul 02 14:34:35.754144 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.754098    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg75v\" (UniqueName: \"kubernetes.io/projected/a9eaf80e-5300-4198-956e-0f00b1984d04-kube-api-access-fg75v\") pod \"isvc-autogluon-v2-predictor-67b849d456-mgc9f\" (UID: \"a9eaf80e-5300-4198-956e-0f00b1984d04\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f"
Jul 02 14:34:35.754144 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.754133    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a9eaf80e-5300-4198-956e-0f00b1984d04-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-mgc9f\" (UID: \"a9eaf80e-5300-4198-956e-0f00b1984d04\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f"
Jul 02 14:34:35.854874 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.854837    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a9eaf80e-5300-4198-956e-0f00b1984d04-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-mgc9f\" (UID: \"a9eaf80e-5300-4198-956e-0f00b1984d04\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f"
Jul 02 14:34:35.855125 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.854903    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a9eaf80e-5300-4198-956e-0f00b1984d04-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-mgc9f\" (UID: \"a9eaf80e-5300-4198-956e-0f00b1984d04\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f"
Jul 02 14:34:35.855125 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.854947    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a9eaf80e-5300-4198-956e-0f00b1984d04-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-mgc9f\" (UID: \"a9eaf80e-5300-4198-956e-0f00b1984d04\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f"
Jul 02 14:34:35.855125 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.855006    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a9eaf80e-5300-4198-956e-0f00b1984d04-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-mgc9f\" (UID: \"a9eaf80e-5300-4198-956e-0f00b1984d04\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f"
Jul 02 14:34:35.855125 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.855035    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg75v\" (UniqueName: \"kubernetes.io/projected/a9eaf80e-5300-4198-956e-0f00b1984d04-kube-api-access-fg75v\") pod \"isvc-autogluon-v2-predictor-67b849d456-mgc9f\" (UID: \"a9eaf80e-5300-4198-956e-0f00b1984d04\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f"
Jul 02 14:34:35.855333 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.855293    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a9eaf80e-5300-4198-956e-0f00b1984d04-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-mgc9f\" (UID: \"a9eaf80e-5300-4198-956e-0f00b1984d04\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f"
Jul 02 14:34:35.855665 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.855628    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a9eaf80e-5300-4198-956e-0f00b1984d04-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-mgc9f\" (UID: \"a9eaf80e-5300-4198-956e-0f00b1984d04\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f"
Jul 02 14:34:35.855944 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.855670    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a9eaf80e-5300-4198-956e-0f00b1984d04-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-mgc9f\" (UID: \"a9eaf80e-5300-4198-956e-0f00b1984d04\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f"
Jul 02 14:34:35.857971 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.857936    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a9eaf80e-5300-4198-956e-0f00b1984d04-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-mgc9f\" (UID: \"a9eaf80e-5300-4198-956e-0f00b1984d04\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f"
Jul 02 14:34:35.864057 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.864034    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg75v\" (UniqueName: \"kubernetes.io/projected/a9eaf80e-5300-4198-956e-0f00b1984d04-kube-api-access-fg75v\") pod \"isvc-autogluon-v2-predictor-67b849d456-mgc9f\" (UID: \"a9eaf80e-5300-4198-956e-0f00b1984d04\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f"
Jul 02 14:34:35.941029 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:35.940946    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f"
Jul 02 14:34:36.087266 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:36.083805    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f"]
Jul 02 14:34:36.091393 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:34:36.091365    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9eaf80e_5300_4198_956e_0f00b1984d04.slice/crio-623fabc114664bbcb9246944bdce9ef5e4dbb72d789fd9c99637b93b0d62cb0a WatchSource:0}: Error finding container 623fabc114664bbcb9246944bdce9ef5e4dbb72d789fd9c99637b93b0d62cb0a: Status 404 returned error can't find the container with id 623fabc114664bbcb9246944bdce9ef5e4dbb72d789fd9c99637b93b0d62cb0a
Jul 02 14:34:36.219970 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:36.219926    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f" event={"ID":"a9eaf80e-5300-4198-956e-0f00b1984d04","Type":"ContainerStarted","Data":"4ede94999651f3a2c939cca5e0e4311ac06be75e4d50a3d27744e0f088153faa"}
Jul 02 14:34:36.220122 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:36.220022    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f" event={"ID":"a9eaf80e-5300-4198-956e-0f00b1984d04","Type":"ContainerStarted","Data":"623fabc114664bbcb9246944bdce9ef5e4dbb72d789fd9c99637b93b0d62cb0a"}
Jul 02 14:34:41.240523 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:41.240493    2582 generic.go:356] "Generic (PLEG): container finished" podID="a9eaf80e-5300-4198-956e-0f00b1984d04" containerID="4ede94999651f3a2c939cca5e0e4311ac06be75e4d50a3d27744e0f088153faa" exitCode=0
Jul 02 14:34:41.240999 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:41.240577    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f" event={"ID":"a9eaf80e-5300-4198-956e-0f00b1984d04","Type":"ContainerDied","Data":"4ede94999651f3a2c939cca5e0e4311ac06be75e4d50a3d27744e0f088153faa"}
Jul 02 14:34:42.247084 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:42.247044    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f" event={"ID":"a9eaf80e-5300-4198-956e-0f00b1984d04","Type":"ContainerStarted","Data":"a005237320cdb48649b96065998153eb514a1884fc9161b4049f4b3cb95cb99b"}
Jul 02 14:34:42.247484 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:42.247134    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f" event={"ID":"a9eaf80e-5300-4198-956e-0f00b1984d04","Type":"ContainerStarted","Data":"41086eee39271c9b9ac349422e281435bce9a411dfc75da6c1de4ddee23fa4f6"}
Jul 02 14:34:42.247484 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:42.247348    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f"
Jul 02 14:34:42.247484 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:42.247394    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f"
Jul 02 14:34:42.266768 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:42.266703    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f" podStartSLOduration=2.355516644 podStartE2EDuration="7.266683318s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.911166674s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:34:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:34:42.266279914 +0000 UTC m=+1144.217617585" watchObservedRunningTime="2026-07-02 14:34:42.266683318 +0000 UTC m=+1144.218020962"
Jul 02 14:34:48.257653 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:34:48.257618    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f"
Jul 02 14:35:38.493070 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:35:38.493044    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnwb5_ca909057-dd63-451d-9c05-b671ca5ae671/ovn-acl-logging/0.log"
Jul 02 14:35:38.493549 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:35:38.493139    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnwb5_ca909057-dd63-451d-9c05-b671ca5ae671/ovn-acl-logging/0.log"
Jul 02 14:36:12.475205 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:12.475178    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f"
Jul 02 14:36:15.756139 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:15.756097    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f"]
Jul 02 14:36:15.756599 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:15.756430    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f" podUID="a9eaf80e-5300-4198-956e-0f00b1984d04" containerName="kserve-container" containerID="cri-o://41086eee39271c9b9ac349422e281435bce9a411dfc75da6c1de4ddee23fa4f6" gracePeriod=30
Jul 02 14:36:15.756672 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:15.756575    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f" podUID="a9eaf80e-5300-4198-956e-0f00b1984d04" containerName="kube-rbac-proxy" containerID="cri-o://a005237320cdb48649b96065998153eb514a1884fc9161b4049f4b3cb95cb99b" gracePeriod=30
Jul 02 14:36:16.602652 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.602621    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f"
Jul 02 14:36:16.644205 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.644127    2582 generic.go:356] "Generic (PLEG): container finished" podID="a9eaf80e-5300-4198-956e-0f00b1984d04" containerID="a005237320cdb48649b96065998153eb514a1884fc9161b4049f4b3cb95cb99b" exitCode=2
Jul 02 14:36:16.644205 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.644152    2582 generic.go:356] "Generic (PLEG): container finished" podID="a9eaf80e-5300-4198-956e-0f00b1984d04" containerID="41086eee39271c9b9ac349422e281435bce9a411dfc75da6c1de4ddee23fa4f6" exitCode=0
Jul 02 14:36:16.644205 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.644194    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f" event={"ID":"a9eaf80e-5300-4198-956e-0f00b1984d04","Type":"ContainerDied","Data":"a005237320cdb48649b96065998153eb514a1884fc9161b4049f4b3cb95cb99b"}
Jul 02 14:36:16.644459 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.644238    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f" event={"ID":"a9eaf80e-5300-4198-956e-0f00b1984d04","Type":"ContainerDied","Data":"41086eee39271c9b9ac349422e281435bce9a411dfc75da6c1de4ddee23fa4f6"}
Jul 02 14:36:16.644459 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.644258    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f" event={"ID":"a9eaf80e-5300-4198-956e-0f00b1984d04","Type":"ContainerDied","Data":"623fabc114664bbcb9246944bdce9ef5e4dbb72d789fd9c99637b93b0d62cb0a"}
Jul 02 14:36:16.644459 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.644195    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f"
Jul 02 14:36:16.644459 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.644285    2582 scope.go:117] "RemoveContainer" containerID="a005237320cdb48649b96065998153eb514a1884fc9161b4049f4b3cb95cb99b"
Jul 02 14:36:16.650339 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.650319    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a9eaf80e-5300-4198-956e-0f00b1984d04-proxy-tls\") pod \"a9eaf80e-5300-4198-956e-0f00b1984d04\" (UID: \"a9eaf80e-5300-4198-956e-0f00b1984d04\") "
Jul 02 14:36:16.650452 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.650387    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a9eaf80e-5300-4198-956e-0f00b1984d04-cabundle-cert\") pod \"a9eaf80e-5300-4198-956e-0f00b1984d04\" (UID: \"a9eaf80e-5300-4198-956e-0f00b1984d04\") "
Jul 02 14:36:16.650452 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.650440    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a9eaf80e-5300-4198-956e-0f00b1984d04-kserve-provision-location\") pod \"a9eaf80e-5300-4198-956e-0f00b1984d04\" (UID: \"a9eaf80e-5300-4198-956e-0f00b1984d04\") "
Jul 02 14:36:16.650578 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.650466    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a9eaf80e-5300-4198-956e-0f00b1984d04-kube-rbac-proxy-sar-config\") pod \"a9eaf80e-5300-4198-956e-0f00b1984d04\" (UID: \"a9eaf80e-5300-4198-956e-0f00b1984d04\") "
Jul 02 14:36:16.650578 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.650483    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fg75v\" (UniqueName: \"kubernetes.io/projected/a9eaf80e-5300-4198-956e-0f00b1984d04-kube-api-access-fg75v\") pod \"a9eaf80e-5300-4198-956e-0f00b1984d04\" (UID: \"a9eaf80e-5300-4198-956e-0f00b1984d04\") "
Jul 02 14:36:16.650884 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.650842    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9eaf80e-5300-4198-956e-0f00b1984d04-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "a9eaf80e-5300-4198-956e-0f00b1984d04" (UID: "a9eaf80e-5300-4198-956e-0f00b1984d04"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:36:16.650884 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.650861    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9eaf80e-5300-4198-956e-0f00b1984d04-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "a9eaf80e-5300-4198-956e-0f00b1984d04" (UID: "a9eaf80e-5300-4198-956e-0f00b1984d04"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:36:16.654303 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.654280    2582 scope.go:117] "RemoveContainer" containerID="41086eee39271c9b9ac349422e281435bce9a411dfc75da6c1de4ddee23fa4f6"
Jul 02 14:36:16.655105 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.655076    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9eaf80e-5300-4198-956e-0f00b1984d04-kube-api-access-fg75v" (OuterVolumeSpecName: "kube-api-access-fg75v") pod "a9eaf80e-5300-4198-956e-0f00b1984d04" (UID: "a9eaf80e-5300-4198-956e-0f00b1984d04"). InnerVolumeSpecName "kube-api-access-fg75v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:36:16.655297 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.655276    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9eaf80e-5300-4198-956e-0f00b1984d04-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "a9eaf80e-5300-4198-956e-0f00b1984d04" (UID: "a9eaf80e-5300-4198-956e-0f00b1984d04"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 14:36:16.656851 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.656827    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9eaf80e-5300-4198-956e-0f00b1984d04-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "a9eaf80e-5300-4198-956e-0f00b1984d04" (UID: "a9eaf80e-5300-4198-956e-0f00b1984d04"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 14:36:16.663026 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.663005    2582 scope.go:117] "RemoveContainer" containerID="4ede94999651f3a2c939cca5e0e4311ac06be75e4d50a3d27744e0f088153faa"
Jul 02 14:36:16.670450 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.670430    2582 scope.go:117] "RemoveContainer" containerID="a005237320cdb48649b96065998153eb514a1884fc9161b4049f4b3cb95cb99b"
Jul 02 14:36:16.670714 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:36:16.670692    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a005237320cdb48649b96065998153eb514a1884fc9161b4049f4b3cb95cb99b\": container with ID starting with a005237320cdb48649b96065998153eb514a1884fc9161b4049f4b3cb95cb99b not found: ID does not exist" containerID="a005237320cdb48649b96065998153eb514a1884fc9161b4049f4b3cb95cb99b"
Jul 02 14:36:16.670763 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.670727    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a005237320cdb48649b96065998153eb514a1884fc9161b4049f4b3cb95cb99b"} err="failed to get container status \"a005237320cdb48649b96065998153eb514a1884fc9161b4049f4b3cb95cb99b\": rpc error: code = NotFound desc = could not find container \"a005237320cdb48649b96065998153eb514a1884fc9161b4049f4b3cb95cb99b\": container with ID starting with a005237320cdb48649b96065998153eb514a1884fc9161b4049f4b3cb95cb99b not found: ID does not exist"
Jul 02 14:36:16.670763 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.670744    2582 scope.go:117] "RemoveContainer" containerID="41086eee39271c9b9ac349422e281435bce9a411dfc75da6c1de4ddee23fa4f6"
Jul 02 14:36:16.670968 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:36:16.670939    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41086eee39271c9b9ac349422e281435bce9a411dfc75da6c1de4ddee23fa4f6\": container with ID starting with 41086eee39271c9b9ac349422e281435bce9a411dfc75da6c1de4ddee23fa4f6 not found: ID does not exist" containerID="41086eee39271c9b9ac349422e281435bce9a411dfc75da6c1de4ddee23fa4f6"
Jul 02 14:36:16.671025 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.670977    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41086eee39271c9b9ac349422e281435bce9a411dfc75da6c1de4ddee23fa4f6"} err="failed to get container status \"41086eee39271c9b9ac349422e281435bce9a411dfc75da6c1de4ddee23fa4f6\": rpc error: code = NotFound desc = could not find container \"41086eee39271c9b9ac349422e281435bce9a411dfc75da6c1de4ddee23fa4f6\": container with ID starting with 41086eee39271c9b9ac349422e281435bce9a411dfc75da6c1de4ddee23fa4f6 not found: ID does not exist"
Jul 02 14:36:16.671025 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.670994    2582 scope.go:117] "RemoveContainer" containerID="4ede94999651f3a2c939cca5e0e4311ac06be75e4d50a3d27744e0f088153faa"
Jul 02 14:36:16.671219 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:36:16.671204    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ede94999651f3a2c939cca5e0e4311ac06be75e4d50a3d27744e0f088153faa\": container with ID starting with 4ede94999651f3a2c939cca5e0e4311ac06be75e4d50a3d27744e0f088153faa not found: ID does not exist" containerID="4ede94999651f3a2c939cca5e0e4311ac06be75e4d50a3d27744e0f088153faa"
Jul 02 14:36:16.671258 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.671222    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ede94999651f3a2c939cca5e0e4311ac06be75e4d50a3d27744e0f088153faa"} err="failed to get container status \"4ede94999651f3a2c939cca5e0e4311ac06be75e4d50a3d27744e0f088153faa\": rpc error: code = NotFound desc = could not find container \"4ede94999651f3a2c939cca5e0e4311ac06be75e4d50a3d27744e0f088153faa\": container with ID starting with 4ede94999651f3a2c939cca5e0e4311ac06be75e4d50a3d27744e0f088153faa not found: ID does not exist"
Jul 02 14:36:16.671258 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.671240    2582 scope.go:117] "RemoveContainer" containerID="a005237320cdb48649b96065998153eb514a1884fc9161b4049f4b3cb95cb99b"
Jul 02 14:36:16.671433 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.671416    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a005237320cdb48649b96065998153eb514a1884fc9161b4049f4b3cb95cb99b"} err="failed to get container status \"a005237320cdb48649b96065998153eb514a1884fc9161b4049f4b3cb95cb99b\": rpc error: code = NotFound desc = could not find container \"a005237320cdb48649b96065998153eb514a1884fc9161b4049f4b3cb95cb99b\": container with ID starting with a005237320cdb48649b96065998153eb514a1884fc9161b4049f4b3cb95cb99b not found: ID does not exist"
Jul 02 14:36:16.671471 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.671433    2582 scope.go:117] "RemoveContainer" containerID="41086eee39271c9b9ac349422e281435bce9a411dfc75da6c1de4ddee23fa4f6"
Jul 02 14:36:16.671622 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.671606    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41086eee39271c9b9ac349422e281435bce9a411dfc75da6c1de4ddee23fa4f6"} err="failed to get container status \"41086eee39271c9b9ac349422e281435bce9a411dfc75da6c1de4ddee23fa4f6\": rpc error: code = NotFound desc = could not find container \"41086eee39271c9b9ac349422e281435bce9a411dfc75da6c1de4ddee23fa4f6\": container with ID starting with 41086eee39271c9b9ac349422e281435bce9a411dfc75da6c1de4ddee23fa4f6 not found: ID does not exist"
Jul 02 14:36:16.671663 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.671622    2582 scope.go:117] "RemoveContainer" containerID="4ede94999651f3a2c939cca5e0e4311ac06be75e4d50a3d27744e0f088153faa"
Jul 02 14:36:16.671788 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.671772    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ede94999651f3a2c939cca5e0e4311ac06be75e4d50a3d27744e0f088153faa"} err="failed to get container status \"4ede94999651f3a2c939cca5e0e4311ac06be75e4d50a3d27744e0f088153faa\": rpc error: code = NotFound desc = could not find container \"4ede94999651f3a2c939cca5e0e4311ac06be75e4d50a3d27744e0f088153faa\": container with ID starting with 4ede94999651f3a2c939cca5e0e4311ac06be75e4d50a3d27744e0f088153faa not found: ID does not exist"
Jul 02 14:36:16.751583 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.751548    2582 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a9eaf80e-5300-4198-956e-0f00b1984d04-cabundle-cert\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:36:16.751583 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.751584    2582 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a9eaf80e-5300-4198-956e-0f00b1984d04-kserve-provision-location\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:36:16.751777 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.751600    2582 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a9eaf80e-5300-4198-956e-0f00b1984d04-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:36:16.751777 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.751614    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fg75v\" (UniqueName: \"kubernetes.io/projected/a9eaf80e-5300-4198-956e-0f00b1984d04-kube-api-access-fg75v\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:36:16.751777 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.751625    2582 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a9eaf80e-5300-4198-956e-0f00b1984d04-proxy-tls\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:36:16.969727 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.969697    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f"]
Jul 02 14:36:16.973762 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:16.973740    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-mgc9f"]
Jul 02 14:36:18.475049 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:18.475019    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9eaf80e-5300-4198-956e-0f00b1984d04" path="/var/lib/kubelet/pods/a9eaf80e-5300-4198-956e-0f00b1984d04/volumes"
Jul 02 14:36:20.938861 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:20.938825    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb"]
Jul 02 14:36:20.939356 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:20.939313    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9eaf80e-5300-4198-956e-0f00b1984d04" containerName="kserve-container"
Jul 02 14:36:20.939356 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:20.939332    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9eaf80e-5300-4198-956e-0f00b1984d04" containerName="kserve-container"
Jul 02 14:36:20.939356 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:20.939347    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9eaf80e-5300-4198-956e-0f00b1984d04" containerName="storage-initializer"
Jul 02 14:36:20.939356 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:20.939355    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9eaf80e-5300-4198-956e-0f00b1984d04" containerName="storage-initializer"
Jul 02 14:36:20.939560 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:20.939375    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9eaf80e-5300-4198-956e-0f00b1984d04" containerName="kube-rbac-proxy"
Jul 02 14:36:20.939560 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:20.939385    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9eaf80e-5300-4198-956e-0f00b1984d04" containerName="kube-rbac-proxy"
Jul 02 14:36:20.939560 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:20.939469    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9eaf80e-5300-4198-956e-0f00b1984d04" containerName="kube-rbac-proxy"
Jul 02 14:36:20.939560 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:20.939483    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9eaf80e-5300-4198-956e-0f00b1984d04" containerName="kserve-container"
Jul 02 14:36:20.942337 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:20.942316    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb"
Jul 02 14:36:20.945260 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:20.945215    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-variants-kube-rbac-proxy-sar-config\""
Jul 02 14:36:20.945260 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:20.945245    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 02 14:36:20.945454 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:20.945264    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 02 14:36:20.945454 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:20.945307    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 02 14:36:20.945454 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:20.945226    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-variants-predictor-serving-cert\""
Jul 02 14:36:20.945454 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:20.945307    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 02 14:36:20.945454 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:20.945376    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-7rr57\""
Jul 02 14:36:20.953287 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:20.953265    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb"]
Jul 02 14:36:20.980750 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:20.980712    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/789ef6e7-73f7-47d4-89d8-19680aa71978-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb\" (UID: \"789ef6e7-73f7-47d4-89d8-19680aa71978\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb"
Jul 02 14:36:20.980750 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:20.980755    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/789ef6e7-73f7-47d4-89d8-19680aa71978-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb\" (UID: \"789ef6e7-73f7-47d4-89d8-19680aa71978\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb"
Jul 02 14:36:20.980933 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:20.980778    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/789ef6e7-73f7-47d4-89d8-19680aa71978-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb\" (UID: \"789ef6e7-73f7-47d4-89d8-19680aa71978\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb"
Jul 02 14:36:20.980933 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:20.980825    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7g5p\" (UniqueName: \"kubernetes.io/projected/789ef6e7-73f7-47d4-89d8-19680aa71978-kube-api-access-z7g5p\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb\" (UID: \"789ef6e7-73f7-47d4-89d8-19680aa71978\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb"
Jul 02 14:36:20.980933 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:20.980870    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/789ef6e7-73f7-47d4-89d8-19680aa71978-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb\" (UID: \"789ef6e7-73f7-47d4-89d8-19680aa71978\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb"
Jul 02 14:36:21.081999 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:21.081930    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/789ef6e7-73f7-47d4-89d8-19680aa71978-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb\" (UID: \"789ef6e7-73f7-47d4-89d8-19680aa71978\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb"
Jul 02 14:36:21.082201 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:21.082004    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/789ef6e7-73f7-47d4-89d8-19680aa71978-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb\" (UID: \"789ef6e7-73f7-47d4-89d8-19680aa71978\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb"
Jul 02 14:36:21.082201 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:21.082038    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/789ef6e7-73f7-47d4-89d8-19680aa71978-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb\" (UID: \"789ef6e7-73f7-47d4-89d8-19680aa71978\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb"
Jul 02 14:36:21.082201 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:21.082065    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7g5p\" (UniqueName: \"kubernetes.io/projected/789ef6e7-73f7-47d4-89d8-19680aa71978-kube-api-access-z7g5p\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb\" (UID: \"789ef6e7-73f7-47d4-89d8-19680aa71978\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb"
Jul 02 14:36:21.082201 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:21.082103    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/789ef6e7-73f7-47d4-89d8-19680aa71978-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb\" (UID: \"789ef6e7-73f7-47d4-89d8-19680aa71978\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb"
Jul 02 14:36:21.082441 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:21.082415    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/789ef6e7-73f7-47d4-89d8-19680aa71978-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb\" (UID: \"789ef6e7-73f7-47d4-89d8-19680aa71978\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb"
Jul 02 14:36:21.082777 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:21.082753    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/789ef6e7-73f7-47d4-89d8-19680aa71978-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb\" (UID: \"789ef6e7-73f7-47d4-89d8-19680aa71978\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb"
Jul 02 14:36:21.082859 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:21.082782    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/789ef6e7-73f7-47d4-89d8-19680aa71978-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb\" (UID: \"789ef6e7-73f7-47d4-89d8-19680aa71978\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb"
Jul 02 14:36:21.084618 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:21.084599    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/789ef6e7-73f7-47d4-89d8-19680aa71978-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb\" (UID: \"789ef6e7-73f7-47d4-89d8-19680aa71978\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb"
Jul 02 14:36:21.091128 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:21.091099    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7g5p\" (UniqueName: \"kubernetes.io/projected/789ef6e7-73f7-47d4-89d8-19680aa71978-kube-api-access-z7g5p\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb\" (UID: \"789ef6e7-73f7-47d4-89d8-19680aa71978\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb"
Jul 02 14:36:21.253414 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:21.253371    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb"
Jul 02 14:36:21.386165 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:21.386137    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb"]
Jul 02 14:36:21.388491 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:36:21.388458    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod789ef6e7_73f7_47d4_89d8_19680aa71978.slice/crio-4c9593112095611c084c01cc9ee1112f80b3b2e1c42ad4eac43d9e4a2f150192 WatchSource:0}: Error finding container 4c9593112095611c084c01cc9ee1112f80b3b2e1c42ad4eac43d9e4a2f150192: Status 404 returned error can't find the container with id 4c9593112095611c084c01cc9ee1112f80b3b2e1c42ad4eac43d9e4a2f150192
Jul 02 14:36:21.671891 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:21.671796    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb" event={"ID":"789ef6e7-73f7-47d4-89d8-19680aa71978","Type":"ContainerStarted","Data":"ff02ecabe044236f1982cfc6c87c17b65ed91325e5520b8067d6039fe6f91338"}
Jul 02 14:36:21.671891 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:21.671878    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb" event={"ID":"789ef6e7-73f7-47d4-89d8-19680aa71978","Type":"ContainerStarted","Data":"4c9593112095611c084c01cc9ee1112f80b3b2e1c42ad4eac43d9e4a2f150192"}
Jul 02 14:36:26.692783 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:26.692749    2582 generic.go:356] "Generic (PLEG): container finished" podID="789ef6e7-73f7-47d4-89d8-19680aa71978" containerID="ff02ecabe044236f1982cfc6c87c17b65ed91325e5520b8067d6039fe6f91338" exitCode=0
Jul 02 14:36:26.693186 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:26.692827    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb" event={"ID":"789ef6e7-73f7-47d4-89d8-19680aa71978","Type":"ContainerDied","Data":"ff02ecabe044236f1982cfc6c87c17b65ed91325e5520b8067d6039fe6f91338"}
Jul 02 14:36:27.700235 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:27.700196    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb" event={"ID":"789ef6e7-73f7-47d4-89d8-19680aa71978","Type":"ContainerStarted","Data":"2f5c071bfba9e6c62c1adb2df5ab936edb6897249a98fae0ab30361c168cad7a"}
Jul 02 14:36:27.700642 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:27.700262    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb" event={"ID":"789ef6e7-73f7-47d4-89d8-19680aa71978","Type":"ContainerStarted","Data":"b9dfa579912b5f1de25df8b25a823beb484d8b720a317daeb5ce6483ac0594f7"}
Jul 02 14:36:27.700642 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:27.700461    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb"
Jul 02 14:36:27.720154 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:27.720069    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb" podStartSLOduration=3.258848578 podStartE2EDuration="7.720049128s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.46120055s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:36:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:36:27.719509307 +0000 UTC m=+1249.670846951" watchObservedRunningTime="2026-07-02 14:36:27.720049128 +0000 UTC m=+1249.671386775"
Jul 02 14:36:28.704874 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:28.704803    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb"
Jul 02 14:36:34.717294 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:36:34.717262    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb"
Jul 02 14:38:04.475714 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:04.475638    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb"
Jul 02 14:38:11.122068 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:11.122038    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb"]
Jul 02 14:38:11.122469 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:11.122277    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb" podUID="789ef6e7-73f7-47d4-89d8-19680aa71978" containerName="kserve-container" containerID="cri-o://b9dfa579912b5f1de25df8b25a823beb484d8b720a317daeb5ce6483ac0594f7" gracePeriod=30
Jul 02 14:38:11.122469 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:11.122354    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb" podUID="789ef6e7-73f7-47d4-89d8-19680aa71978" containerName="kube-rbac-proxy" containerID="cri-o://2f5c071bfba9e6c62c1adb2df5ab936edb6897249a98fae0ab30361c168cad7a" gracePeriod=30
Jul 02 14:38:11.962682 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:11.962653    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb"
Jul 02 14:38:11.988446 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:11.988422    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/789ef6e7-73f7-47d4-89d8-19680aa71978-cabundle-cert\") pod \"789ef6e7-73f7-47d4-89d8-19680aa71978\" (UID: \"789ef6e7-73f7-47d4-89d8-19680aa71978\") "
Jul 02 14:38:11.988578 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:11.988455    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7g5p\" (UniqueName: \"kubernetes.io/projected/789ef6e7-73f7-47d4-89d8-19680aa71978-kube-api-access-z7g5p\") pod \"789ef6e7-73f7-47d4-89d8-19680aa71978\" (UID: \"789ef6e7-73f7-47d4-89d8-19680aa71978\") "
Jul 02 14:38:11.988578 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:11.988487    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/789ef6e7-73f7-47d4-89d8-19680aa71978-kserve-provision-location\") pod \"789ef6e7-73f7-47d4-89d8-19680aa71978\" (UID: \"789ef6e7-73f7-47d4-89d8-19680aa71978\") "
Jul 02 14:38:11.988578 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:11.988569    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/789ef6e7-73f7-47d4-89d8-19680aa71978-kube-rbac-proxy-sar-config\") pod \"789ef6e7-73f7-47d4-89d8-19680aa71978\" (UID: \"789ef6e7-73f7-47d4-89d8-19680aa71978\") "
Jul 02 14:38:11.988732 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:11.988613    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/789ef6e7-73f7-47d4-89d8-19680aa71978-proxy-tls\") pod \"789ef6e7-73f7-47d4-89d8-19680aa71978\" (UID: \"789ef6e7-73f7-47d4-89d8-19680aa71978\") "
Jul 02 14:38:11.988917 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:11.988889    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/789ef6e7-73f7-47d4-89d8-19680aa71978-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "789ef6e7-73f7-47d4-89d8-19680aa71978" (UID: "789ef6e7-73f7-47d4-89d8-19680aa71978"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:38:11.989270 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:11.989243    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/789ef6e7-73f7-47d4-89d8-19680aa71978-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "789ef6e7-73f7-47d4-89d8-19680aa71978" (UID: "789ef6e7-73f7-47d4-89d8-19680aa71978"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:38:11.991333 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:11.991306    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/789ef6e7-73f7-47d4-89d8-19680aa71978-kube-api-access-z7g5p" (OuterVolumeSpecName: "kube-api-access-z7g5p") pod "789ef6e7-73f7-47d4-89d8-19680aa71978" (UID: "789ef6e7-73f7-47d4-89d8-19680aa71978"). InnerVolumeSpecName "kube-api-access-z7g5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:38:11.991430 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:11.991390    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/789ef6e7-73f7-47d4-89d8-19680aa71978-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "789ef6e7-73f7-47d4-89d8-19680aa71978" (UID: "789ef6e7-73f7-47d4-89d8-19680aa71978"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 14:38:11.993409 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:11.993383    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/789ef6e7-73f7-47d4-89d8-19680aa71978-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "789ef6e7-73f7-47d4-89d8-19680aa71978" (UID: "789ef6e7-73f7-47d4-89d8-19680aa71978"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 14:38:12.090004 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:12.089888    2582 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/789ef6e7-73f7-47d4-89d8-19680aa71978-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:38:12.090004 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:12.089913    2582 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/789ef6e7-73f7-47d4-89d8-19680aa71978-proxy-tls\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:38:12.090004 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:12.089923    2582 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/789ef6e7-73f7-47d4-89d8-19680aa71978-cabundle-cert\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:38:12.090004 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:12.089933    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7g5p\" (UniqueName: \"kubernetes.io/projected/789ef6e7-73f7-47d4-89d8-19680aa71978-kube-api-access-z7g5p\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:38:12.090004 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:12.089941    2582 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/789ef6e7-73f7-47d4-89d8-19680aa71978-kserve-provision-location\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:38:12.151430 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:12.151402    2582 generic.go:356] "Generic (PLEG): container finished" podID="789ef6e7-73f7-47d4-89d8-19680aa71978" containerID="2f5c071bfba9e6c62c1adb2df5ab936edb6897249a98fae0ab30361c168cad7a" exitCode=2
Jul 02 14:38:12.151430 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:12.151427    2582 generic.go:356] "Generic (PLEG): container finished" podID="789ef6e7-73f7-47d4-89d8-19680aa71978" containerID="b9dfa579912b5f1de25df8b25a823beb484d8b720a317daeb5ce6483ac0594f7" exitCode=0
Jul 02 14:38:12.151820 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:12.151454    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb" event={"ID":"789ef6e7-73f7-47d4-89d8-19680aa71978","Type":"ContainerDied","Data":"2f5c071bfba9e6c62c1adb2df5ab936edb6897249a98fae0ab30361c168cad7a"}
Jul 02 14:38:12.151820 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:12.151463    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb"
Jul 02 14:38:12.151820 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:12.151510    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb" event={"ID":"789ef6e7-73f7-47d4-89d8-19680aa71978","Type":"ContainerDied","Data":"b9dfa579912b5f1de25df8b25a823beb484d8b720a317daeb5ce6483ac0594f7"}
Jul 02 14:38:12.151820 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:12.151535    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb" event={"ID":"789ef6e7-73f7-47d4-89d8-19680aa71978","Type":"ContainerDied","Data":"4c9593112095611c084c01cc9ee1112f80b3b2e1c42ad4eac43d9e4a2f150192"}
Jul 02 14:38:12.151820 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:12.151560    2582 scope.go:117] "RemoveContainer" containerID="2f5c071bfba9e6c62c1adb2df5ab936edb6897249a98fae0ab30361c168cad7a"
Jul 02 14:38:12.160461 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:12.160446    2582 scope.go:117] "RemoveContainer" containerID="b9dfa579912b5f1de25df8b25a823beb484d8b720a317daeb5ce6483ac0594f7"
Jul 02 14:38:12.167521 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:12.167502    2582 scope.go:117] "RemoveContainer" containerID="ff02ecabe044236f1982cfc6c87c17b65ed91325e5520b8067d6039fe6f91338"
Jul 02 14:38:12.175210 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:12.175192    2582 scope.go:117] "RemoveContainer" containerID="2f5c071bfba9e6c62c1adb2df5ab936edb6897249a98fae0ab30361c168cad7a"
Jul 02 14:38:12.175459 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:38:12.175435    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f5c071bfba9e6c62c1adb2df5ab936edb6897249a98fae0ab30361c168cad7a\": container with ID starting with 2f5c071bfba9e6c62c1adb2df5ab936edb6897249a98fae0ab30361c168cad7a not found: ID does not exist" containerID="2f5c071bfba9e6c62c1adb2df5ab936edb6897249a98fae0ab30361c168cad7a"
Jul 02 14:38:12.175564 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:12.175475    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f5c071bfba9e6c62c1adb2df5ab936edb6897249a98fae0ab30361c168cad7a"} err="failed to get container status \"2f5c071bfba9e6c62c1adb2df5ab936edb6897249a98fae0ab30361c168cad7a\": rpc error: code = NotFound desc = could not find container \"2f5c071bfba9e6c62c1adb2df5ab936edb6897249a98fae0ab30361c168cad7a\": container with ID starting with 2f5c071bfba9e6c62c1adb2df5ab936edb6897249a98fae0ab30361c168cad7a not found: ID does not exist"
Jul 02 14:38:12.175564 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:12.175492    2582 scope.go:117] "RemoveContainer" containerID="b9dfa579912b5f1de25df8b25a823beb484d8b720a317daeb5ce6483ac0594f7"
Jul 02 14:38:12.175748 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:38:12.175730    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9dfa579912b5f1de25df8b25a823beb484d8b720a317daeb5ce6483ac0594f7\": container with ID starting with b9dfa579912b5f1de25df8b25a823beb484d8b720a317daeb5ce6483ac0594f7 not found: ID does not exist" containerID="b9dfa579912b5f1de25df8b25a823beb484d8b720a317daeb5ce6483ac0594f7"
Jul 02 14:38:12.175804 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:12.175754    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9dfa579912b5f1de25df8b25a823beb484d8b720a317daeb5ce6483ac0594f7"} err="failed to get container status \"b9dfa579912b5f1de25df8b25a823beb484d8b720a317daeb5ce6483ac0594f7\": rpc error: code = NotFound desc = could not find container \"b9dfa579912b5f1de25df8b25a823beb484d8b720a317daeb5ce6483ac0594f7\": container with ID starting with b9dfa579912b5f1de25df8b25a823beb484d8b720a317daeb5ce6483ac0594f7 not found: ID does not exist"
Jul 02 14:38:12.175804 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:12.175769    2582 scope.go:117] "RemoveContainer" containerID="ff02ecabe044236f1982cfc6c87c17b65ed91325e5520b8067d6039fe6f91338"
Jul 02 14:38:12.176038 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:38:12.176018    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff02ecabe044236f1982cfc6c87c17b65ed91325e5520b8067d6039fe6f91338\": container with ID starting with ff02ecabe044236f1982cfc6c87c17b65ed91325e5520b8067d6039fe6f91338 not found: ID does not exist" containerID="ff02ecabe044236f1982cfc6c87c17b65ed91325e5520b8067d6039fe6f91338"
Jul 02 14:38:12.176123 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:12.176040    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff02ecabe044236f1982cfc6c87c17b65ed91325e5520b8067d6039fe6f91338"} err="failed to get container status \"ff02ecabe044236f1982cfc6c87c17b65ed91325e5520b8067d6039fe6f91338\": rpc error: code = NotFound desc = could not find container \"ff02ecabe044236f1982cfc6c87c17b65ed91325e5520b8067d6039fe6f91338\": container with ID starting with ff02ecabe044236f1982cfc6c87c17b65ed91325e5520b8067d6039fe6f91338 not found: ID does not exist"
Jul 02 14:38:12.176123 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:12.176054    2582 scope.go:117] "RemoveContainer" containerID="2f5c071bfba9e6c62c1adb2df5ab936edb6897249a98fae0ab30361c168cad7a"
Jul 02 14:38:12.176123 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:12.176062    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb"]
Jul 02 14:38:12.176296 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:12.176276    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f5c071bfba9e6c62c1adb2df5ab936edb6897249a98fae0ab30361c168cad7a"} err="failed to get container status \"2f5c071bfba9e6c62c1adb2df5ab936edb6897249a98fae0ab30361c168cad7a\": rpc error: code = NotFound desc = could not find container \"2f5c071bfba9e6c62c1adb2df5ab936edb6897249a98fae0ab30361c168cad7a\": container with ID starting with 2f5c071bfba9e6c62c1adb2df5ab936edb6897249a98fae0ab30361c168cad7a not found: ID does not exist"
Jul 02 14:38:12.176341 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:12.176296    2582 scope.go:117] "RemoveContainer" containerID="b9dfa579912b5f1de25df8b25a823beb484d8b720a317daeb5ce6483ac0594f7"
Jul 02 14:38:12.176495 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:12.176479    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9dfa579912b5f1de25df8b25a823beb484d8b720a317daeb5ce6483ac0594f7"} err="failed to get container status \"b9dfa579912b5f1de25df8b25a823beb484d8b720a317daeb5ce6483ac0594f7\": rpc error: code = NotFound desc = could not find container \"b9dfa579912b5f1de25df8b25a823beb484d8b720a317daeb5ce6483ac0594f7\": container with ID starting with b9dfa579912b5f1de25df8b25a823beb484d8b720a317daeb5ce6483ac0594f7 not found: ID does not exist"
Jul 02 14:38:12.176495 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:12.176495    2582 scope.go:117] "RemoveContainer" containerID="ff02ecabe044236f1982cfc6c87c17b65ed91325e5520b8067d6039fe6f91338"
Jul 02 14:38:12.176726 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:12.176708    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff02ecabe044236f1982cfc6c87c17b65ed91325e5520b8067d6039fe6f91338"} err="failed to get container status \"ff02ecabe044236f1982cfc6c87c17b65ed91325e5520b8067d6039fe6f91338\": rpc error: code = NotFound desc = could not find container \"ff02ecabe044236f1982cfc6c87c17b65ed91325e5520b8067d6039fe6f91338\": container with ID starting with ff02ecabe044236f1982cfc6c87c17b65ed91325e5520b8067d6039fe6f91338 not found: ID does not exist"
Jul 02 14:38:12.179160 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:12.179141    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-xb8vb"]
Jul 02 14:38:12.475381 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:12.475353    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="789ef6e7-73f7-47d4-89d8-19680aa71978" path="/var/lib/kubelet/pods/789ef6e7-73f7-47d4-89d8-19680aa71978/volumes"
Jul 02 14:38:16.217301 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.217268    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp"]
Jul 02 14:38:16.217662 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.217614    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="789ef6e7-73f7-47d4-89d8-19680aa71978" containerName="storage-initializer"
Jul 02 14:38:16.217662 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.217627    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="789ef6e7-73f7-47d4-89d8-19680aa71978" containerName="storage-initializer"
Jul 02 14:38:16.217662 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.217640    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="789ef6e7-73f7-47d4-89d8-19680aa71978" containerName="kserve-container"
Jul 02 14:38:16.217662 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.217648    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="789ef6e7-73f7-47d4-89d8-19680aa71978" containerName="kserve-container"
Jul 02 14:38:16.217810 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.217665    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="789ef6e7-73f7-47d4-89d8-19680aa71978" containerName="kube-rbac-proxy"
Jul 02 14:38:16.217810 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.217671    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="789ef6e7-73f7-47d4-89d8-19680aa71978" containerName="kube-rbac-proxy"
Jul 02 14:38:16.217810 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.217719    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="789ef6e7-73f7-47d4-89d8-19680aa71978" containerName="kserve-container"
Jul 02 14:38:16.217810 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.217733    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="789ef6e7-73f7-47d4-89d8-19680aa71978" containerName="kube-rbac-proxy"
Jul 02 14:38:16.222599 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.222544    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp"
Jul 02 14:38:16.226265 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.226243    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-noslash-predictor-serving-cert\""
Jul 02 14:38:16.226888 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.226869    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 02 14:38:16.227078 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.227043    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-noslash-kube-rbac-proxy-sar-config\""
Jul 02 14:38:16.227209 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.227092    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 02 14:38:16.228002 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.227978    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 02 14:38:16.228533 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.228516    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 02 14:38:16.228615 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.228566    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-7rr57\""
Jul 02 14:38:16.234706 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.234686    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp"]
Jul 02 14:38:16.322011 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.321978    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/184b7b69-e986-4349-ac93-4d8306f2edb2-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp\" (UID: \"184b7b69-e986-4349-ac93-4d8306f2edb2\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp"
Jul 02 14:38:16.322011 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.322026    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj2ct\" (UniqueName: \"kubernetes.io/projected/184b7b69-e986-4349-ac93-4d8306f2edb2-kube-api-access-dj2ct\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp\" (UID: \"184b7b69-e986-4349-ac93-4d8306f2edb2\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp"
Jul 02 14:38:16.322234 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.322067    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/184b7b69-e986-4349-ac93-4d8306f2edb2-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp\" (UID: \"184b7b69-e986-4349-ac93-4d8306f2edb2\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp"
Jul 02 14:38:16.322234 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.322100    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/184b7b69-e986-4349-ac93-4d8306f2edb2-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp\" (UID: \"184b7b69-e986-4349-ac93-4d8306f2edb2\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp"
Jul 02 14:38:16.322234 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.322151    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/184b7b69-e986-4349-ac93-4d8306f2edb2-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp\" (UID: \"184b7b69-e986-4349-ac93-4d8306f2edb2\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp"
Jul 02 14:38:16.423162 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.423121    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj2ct\" (UniqueName: \"kubernetes.io/projected/184b7b69-e986-4349-ac93-4d8306f2edb2-kube-api-access-dj2ct\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp\" (UID: \"184b7b69-e986-4349-ac93-4d8306f2edb2\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp"
Jul 02 14:38:16.423162 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.423175    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/184b7b69-e986-4349-ac93-4d8306f2edb2-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp\" (UID: \"184b7b69-e986-4349-ac93-4d8306f2edb2\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp"
Jul 02 14:38:16.423442 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.423207    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/184b7b69-e986-4349-ac93-4d8306f2edb2-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp\" (UID: \"184b7b69-e986-4349-ac93-4d8306f2edb2\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp"
Jul 02 14:38:16.423442 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.423246    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/184b7b69-e986-4349-ac93-4d8306f2edb2-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp\" (UID: \"184b7b69-e986-4349-ac93-4d8306f2edb2\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp"
Jul 02 14:38:16.423442 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.423280    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/184b7b69-e986-4349-ac93-4d8306f2edb2-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp\" (UID: \"184b7b69-e986-4349-ac93-4d8306f2edb2\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp"
Jul 02 14:38:16.423633 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.423610    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/184b7b69-e986-4349-ac93-4d8306f2edb2-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp\" (UID: \"184b7b69-e986-4349-ac93-4d8306f2edb2\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp"
Jul 02 14:38:16.423987 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.423936    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/184b7b69-e986-4349-ac93-4d8306f2edb2-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp\" (UID: \"184b7b69-e986-4349-ac93-4d8306f2edb2\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp"
Jul 02 14:38:16.423987 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.423950    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/184b7b69-e986-4349-ac93-4d8306f2edb2-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp\" (UID: \"184b7b69-e986-4349-ac93-4d8306f2edb2\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp"
Jul 02 14:38:16.425905 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.425876    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/184b7b69-e986-4349-ac93-4d8306f2edb2-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp\" (UID: \"184b7b69-e986-4349-ac93-4d8306f2edb2\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp"
Jul 02 14:38:16.432721 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.432701    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj2ct\" (UniqueName: \"kubernetes.io/projected/184b7b69-e986-4349-ac93-4d8306f2edb2-kube-api-access-dj2ct\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp\" (UID: \"184b7b69-e986-4349-ac93-4d8306f2edb2\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp"
Jul 02 14:38:16.532532 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.532437    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp"
Jul 02 14:38:16.668661 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:16.668635    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp"]
Jul 02 14:38:16.670963 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:38:16.670922    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod184b7b69_e986_4349_ac93_4d8306f2edb2.slice/crio-b9c3f08703ea40d72bb40870b6ee04253ceb3a4122637913ca14fe433dcce49d WatchSource:0}: Error finding container b9c3f08703ea40d72bb40870b6ee04253ceb3a4122637913ca14fe433dcce49d: Status 404 returned error can't find the container with id b9c3f08703ea40d72bb40870b6ee04253ceb3a4122637913ca14fe433dcce49d
Jul 02 14:38:17.175319 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:17.175285    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp" event={"ID":"184b7b69-e986-4349-ac93-4d8306f2edb2","Type":"ContainerStarted","Data":"fb49be30a0fa0907b2e53bf3f4f8b919158b7ce39a5712ee47d7fce20eebdaea"}
Jul 02 14:38:17.175541 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:17.175347    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp" event={"ID":"184b7b69-e986-4349-ac93-4d8306f2edb2","Type":"ContainerStarted","Data":"b9c3f08703ea40d72bb40870b6ee04253ceb3a4122637913ca14fe433dcce49d"}
Jul 02 14:38:22.198094 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:22.198055    2582 generic.go:356] "Generic (PLEG): container finished" podID="184b7b69-e986-4349-ac93-4d8306f2edb2" containerID="fb49be30a0fa0907b2e53bf3f4f8b919158b7ce39a5712ee47d7fce20eebdaea" exitCode=0
Jul 02 14:38:22.198503 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:22.198135    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp" event={"ID":"184b7b69-e986-4349-ac93-4d8306f2edb2","Type":"ContainerDied","Data":"fb49be30a0fa0907b2e53bf3f4f8b919158b7ce39a5712ee47d7fce20eebdaea"}
Jul 02 14:38:23.205827 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:23.205791    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp" event={"ID":"184b7b69-e986-4349-ac93-4d8306f2edb2","Type":"ContainerStarted","Data":"384ffdece61f8fd230e2624d946acff3cb908371baceedd152a4ac8db3d65a72"}
Jul 02 14:38:23.206301 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:23.205852    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp" event={"ID":"184b7b69-e986-4349-ac93-4d8306f2edb2","Type":"ContainerStarted","Data":"862b56da29e67a6cba018ffeec32749ea2b5776d75f9734d69f039b2bb2624f6"}
Jul 02 14:38:23.206301 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:23.206084    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp"
Jul 02 14:38:23.206301 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:23.206148    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp"
Jul 02 14:38:23.229044 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:23.228978    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp" podStartSLOduration=2.450820402 podStartE2EDuration="7.228932093s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.778111691s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:38:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:38:23.226441018 +0000 UTC m=+1365.177778665" watchObservedRunningTime="2026-07-02 14:38:23.228932093 +0000 UTC m=+1365.180269740"
Jul 02 14:38:29.217651 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:29.217621    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp"
Jul 02 14:38:38.557299 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:38:38.557267    2582 scope.go:117] "RemoveContainer" containerID="253eff4945778e9c4c11ac37b4c8039bf8afb9eb5bbd07b38776ee8b3205ac92"
Jul 02 14:39:52.475627 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:52.475600    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp"
Jul 02 14:39:56.336540 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:56.336510    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp"]
Jul 02 14:39:56.337018 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:56.336820    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp" podUID="184b7b69-e986-4349-ac93-4d8306f2edb2" containerName="kserve-container" containerID="cri-o://862b56da29e67a6cba018ffeec32749ea2b5776d75f9734d69f039b2bb2624f6" gracePeriod=30
Jul 02 14:39:56.337018 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:56.336843    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp" podUID="184b7b69-e986-4349-ac93-4d8306f2edb2" containerName="kube-rbac-proxy" containerID="cri-o://384ffdece61f8fd230e2624d946acff3cb908371baceedd152a4ac8db3d65a72" gracePeriod=30
Jul 02 14:39:56.598773 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:56.598684    2582 generic.go:356] "Generic (PLEG): container finished" podID="184b7b69-e986-4349-ac93-4d8306f2edb2" containerID="384ffdece61f8fd230e2624d946acff3cb908371baceedd152a4ac8db3d65a72" exitCode=2
Jul 02 14:39:56.598773 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:56.598732    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp" event={"ID":"184b7b69-e986-4349-ac93-4d8306f2edb2","Type":"ContainerDied","Data":"384ffdece61f8fd230e2624d946acff3cb908371baceedd152a4ac8db3d65a72"}
Jul 02 14:39:57.185408 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.185385    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp"
Jul 02 14:39:57.301038 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.300905    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/184b7b69-e986-4349-ac93-4d8306f2edb2-kserve-provision-location\") pod \"184b7b69-e986-4349-ac93-4d8306f2edb2\" (UID: \"184b7b69-e986-4349-ac93-4d8306f2edb2\") "
Jul 02 14:39:57.301038 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.301030    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/184b7b69-e986-4349-ac93-4d8306f2edb2-kube-rbac-proxy-sar-config\") pod \"184b7b69-e986-4349-ac93-4d8306f2edb2\" (UID: \"184b7b69-e986-4349-ac93-4d8306f2edb2\") "
Jul 02 14:39:57.301275 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.301068    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dj2ct\" (UniqueName: \"kubernetes.io/projected/184b7b69-e986-4349-ac93-4d8306f2edb2-kube-api-access-dj2ct\") pod \"184b7b69-e986-4349-ac93-4d8306f2edb2\" (UID: \"184b7b69-e986-4349-ac93-4d8306f2edb2\") "
Jul 02 14:39:57.301275 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.301095    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/184b7b69-e986-4349-ac93-4d8306f2edb2-proxy-tls\") pod \"184b7b69-e986-4349-ac93-4d8306f2edb2\" (UID: \"184b7b69-e986-4349-ac93-4d8306f2edb2\") "
Jul 02 14:39:57.301275 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.301123    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/184b7b69-e986-4349-ac93-4d8306f2edb2-cabundle-cert\") pod \"184b7b69-e986-4349-ac93-4d8306f2edb2\" (UID: \"184b7b69-e986-4349-ac93-4d8306f2edb2\") "
Jul 02 14:39:57.301510 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.301474    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/184b7b69-e986-4349-ac93-4d8306f2edb2-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "184b7b69-e986-4349-ac93-4d8306f2edb2" (UID: "184b7b69-e986-4349-ac93-4d8306f2edb2"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:39:57.301510 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.301495    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/184b7b69-e986-4349-ac93-4d8306f2edb2-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "184b7b69-e986-4349-ac93-4d8306f2edb2" (UID: "184b7b69-e986-4349-ac93-4d8306f2edb2"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:39:57.303480 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.303459    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/184b7b69-e986-4349-ac93-4d8306f2edb2-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "184b7b69-e986-4349-ac93-4d8306f2edb2" (UID: "184b7b69-e986-4349-ac93-4d8306f2edb2"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 14:39:57.303579 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.303540    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/184b7b69-e986-4349-ac93-4d8306f2edb2-kube-api-access-dj2ct" (OuterVolumeSpecName: "kube-api-access-dj2ct") pod "184b7b69-e986-4349-ac93-4d8306f2edb2" (UID: "184b7b69-e986-4349-ac93-4d8306f2edb2"). InnerVolumeSpecName "kube-api-access-dj2ct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:39:57.305732 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.305708    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/184b7b69-e986-4349-ac93-4d8306f2edb2-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "184b7b69-e986-4349-ac93-4d8306f2edb2" (UID: "184b7b69-e986-4349-ac93-4d8306f2edb2"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 14:39:57.402401 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.402368    2582 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/184b7b69-e986-4349-ac93-4d8306f2edb2-cabundle-cert\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:39:57.402401 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.402398    2582 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/184b7b69-e986-4349-ac93-4d8306f2edb2-kserve-provision-location\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:39:57.402401 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.402407    2582 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/184b7b69-e986-4349-ac93-4d8306f2edb2-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:39:57.402837 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.402416    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dj2ct\" (UniqueName: \"kubernetes.io/projected/184b7b69-e986-4349-ac93-4d8306f2edb2-kube-api-access-dj2ct\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:39:57.402837 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.402427    2582 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/184b7b69-e986-4349-ac93-4d8306f2edb2-proxy-tls\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:39:57.605356 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.605267    2582 generic.go:356] "Generic (PLEG): container finished" podID="184b7b69-e986-4349-ac93-4d8306f2edb2" containerID="862b56da29e67a6cba018ffeec32749ea2b5776d75f9734d69f039b2bb2624f6" exitCode=0
Jul 02 14:39:57.605356 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.605323    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp" event={"ID":"184b7b69-e986-4349-ac93-4d8306f2edb2","Type":"ContainerDied","Data":"862b56da29e67a6cba018ffeec32749ea2b5776d75f9734d69f039b2bb2624f6"}
Jul 02 14:39:57.605356 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.605337    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp"
Jul 02 14:39:57.605564 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.605382    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp" event={"ID":"184b7b69-e986-4349-ac93-4d8306f2edb2","Type":"ContainerDied","Data":"b9c3f08703ea40d72bb40870b6ee04253ceb3a4122637913ca14fe433dcce49d"}
Jul 02 14:39:57.605564 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.605407    2582 scope.go:117] "RemoveContainer" containerID="384ffdece61f8fd230e2624d946acff3cb908371baceedd152a4ac8db3d65a72"
Jul 02 14:39:57.619491 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.619443    2582 scope.go:117] "RemoveContainer" containerID="862b56da29e67a6cba018ffeec32749ea2b5776d75f9734d69f039b2bb2624f6"
Jul 02 14:39:57.627523 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.627502    2582 scope.go:117] "RemoveContainer" containerID="fb49be30a0fa0907b2e53bf3f4f8b919158b7ce39a5712ee47d7fce20eebdaea"
Jul 02 14:39:57.631401 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.631378    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp"]
Jul 02 14:39:57.635154 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.635132    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-fghbp"]
Jul 02 14:39:57.636041 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.636027    2582 scope.go:117] "RemoveContainer" containerID="384ffdece61f8fd230e2624d946acff3cb908371baceedd152a4ac8db3d65a72"
Jul 02 14:39:57.636319 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:39:57.636302    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"384ffdece61f8fd230e2624d946acff3cb908371baceedd152a4ac8db3d65a72\": container with ID starting with 384ffdece61f8fd230e2624d946acff3cb908371baceedd152a4ac8db3d65a72 not found: ID does not exist" containerID="384ffdece61f8fd230e2624d946acff3cb908371baceedd152a4ac8db3d65a72"
Jul 02 14:39:57.636359 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.636329    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"384ffdece61f8fd230e2624d946acff3cb908371baceedd152a4ac8db3d65a72"} err="failed to get container status \"384ffdece61f8fd230e2624d946acff3cb908371baceedd152a4ac8db3d65a72\": rpc error: code = NotFound desc = could not find container \"384ffdece61f8fd230e2624d946acff3cb908371baceedd152a4ac8db3d65a72\": container with ID starting with 384ffdece61f8fd230e2624d946acff3cb908371baceedd152a4ac8db3d65a72 not found: ID does not exist"
Jul 02 14:39:57.636359 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.636348    2582 scope.go:117] "RemoveContainer" containerID="862b56da29e67a6cba018ffeec32749ea2b5776d75f9734d69f039b2bb2624f6"
Jul 02 14:39:57.636602 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:39:57.636574    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"862b56da29e67a6cba018ffeec32749ea2b5776d75f9734d69f039b2bb2624f6\": container with ID starting with 862b56da29e67a6cba018ffeec32749ea2b5776d75f9734d69f039b2bb2624f6 not found: ID does not exist" containerID="862b56da29e67a6cba018ffeec32749ea2b5776d75f9734d69f039b2bb2624f6"
Jul 02 14:39:57.636644 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.636601    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"862b56da29e67a6cba018ffeec32749ea2b5776d75f9734d69f039b2bb2624f6"} err="failed to get container status \"862b56da29e67a6cba018ffeec32749ea2b5776d75f9734d69f039b2bb2624f6\": rpc error: code = NotFound desc = could not find container \"862b56da29e67a6cba018ffeec32749ea2b5776d75f9734d69f039b2bb2624f6\": container with ID starting with 862b56da29e67a6cba018ffeec32749ea2b5776d75f9734d69f039b2bb2624f6 not found: ID does not exist"
Jul 02 14:39:57.636644 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.636617    2582 scope.go:117] "RemoveContainer" containerID="fb49be30a0fa0907b2e53bf3f4f8b919158b7ce39a5712ee47d7fce20eebdaea"
Jul 02 14:39:57.636928 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:39:57.636905    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb49be30a0fa0907b2e53bf3f4f8b919158b7ce39a5712ee47d7fce20eebdaea\": container with ID starting with fb49be30a0fa0907b2e53bf3f4f8b919158b7ce39a5712ee47d7fce20eebdaea not found: ID does not exist" containerID="fb49be30a0fa0907b2e53bf3f4f8b919158b7ce39a5712ee47d7fce20eebdaea"
Jul 02 14:39:57.636990 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:57.636934    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb49be30a0fa0907b2e53bf3f4f8b919158b7ce39a5712ee47d7fce20eebdaea"} err="failed to get container status \"fb49be30a0fa0907b2e53bf3f4f8b919158b7ce39a5712ee47d7fce20eebdaea\": rpc error: code = NotFound desc = could not find container \"fb49be30a0fa0907b2e53bf3f4f8b919158b7ce39a5712ee47d7fce20eebdaea\": container with ID starting with fb49be30a0fa0907b2e53bf3f4f8b919158b7ce39a5712ee47d7fce20eebdaea not found: ID does not exist"
Jul 02 14:39:58.477370 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:39:58.477339    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="184b7b69-e986-4349-ac93-4d8306f2edb2" path="/var/lib/kubelet/pods/184b7b69-e986-4349-ac93-4d8306f2edb2/volumes"
Jul 02 14:40:01.501040 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.501009    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"]
Jul 02 14:40:01.501421 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.501297    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="184b7b69-e986-4349-ac93-4d8306f2edb2" containerName="storage-initializer"
Jul 02 14:40:01.501421 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.501308    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="184b7b69-e986-4349-ac93-4d8306f2edb2" containerName="storage-initializer"
Jul 02 14:40:01.501421 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.501316    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="184b7b69-e986-4349-ac93-4d8306f2edb2" containerName="kserve-container"
Jul 02 14:40:01.501421 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.501322    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="184b7b69-e986-4349-ac93-4d8306f2edb2" containerName="kserve-container"
Jul 02 14:40:01.501421 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.501331    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="184b7b69-e986-4349-ac93-4d8306f2edb2" containerName="kube-rbac-proxy"
Jul 02 14:40:01.501421 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.501336    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="184b7b69-e986-4349-ac93-4d8306f2edb2" containerName="kube-rbac-proxy"
Jul 02 14:40:01.501421 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.501390    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="184b7b69-e986-4349-ac93-4d8306f2edb2" containerName="kube-rbac-proxy"
Jul 02 14:40:01.501421 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.501397    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="184b7b69-e986-4349-ac93-4d8306f2edb2" containerName="kserve-container"
Jul 02 14:40:01.506623 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.506603    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"
Jul 02 14:40:01.513272 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.513245    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 02 14:40:01.513391 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.513336    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 02 14:40:01.513452 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.513426    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-7rr57\""
Jul 02 14:40:01.513691 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.513675    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-ts-v1-predictor-serving-cert\""
Jul 02 14:40:01.514588 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.514574    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 02 14:40:01.514689 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.514675    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-ts-v1-kube-rbac-proxy-sar-config\""
Jul 02 14:40:01.514735 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.514678    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 02 14:40:01.530056 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.530024    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"]
Jul 02 14:40:01.634909 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.634873    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4a441638-1e97-4917-b1ff-b60ab487772f-cabundle-cert\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b\" (UID: \"4a441638-1e97-4917-b1ff-b60ab487772f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"
Jul 02 14:40:01.635125 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.634979    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4a441638-1e97-4917-b1ff-b60ab487772f-kserve-provision-location\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b\" (UID: \"4a441638-1e97-4917-b1ff-b60ab487772f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"
Jul 02 14:40:01.635125 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.635013    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjpbq\" (UniqueName: \"kubernetes.io/projected/4a441638-1e97-4917-b1ff-b60ab487772f-kube-api-access-vjpbq\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b\" (UID: \"4a441638-1e97-4917-b1ff-b60ab487772f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"
Jul 02 14:40:01.635125 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.635039    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4a441638-1e97-4917-b1ff-b60ab487772f-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b\" (UID: \"4a441638-1e97-4917-b1ff-b60ab487772f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"
Jul 02 14:40:01.635125 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.635077    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4a441638-1e97-4917-b1ff-b60ab487772f-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b\" (UID: \"4a441638-1e97-4917-b1ff-b60ab487772f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"
Jul 02 14:40:01.736344 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.736309    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4a441638-1e97-4917-b1ff-b60ab487772f-kserve-provision-location\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b\" (UID: \"4a441638-1e97-4917-b1ff-b60ab487772f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"
Jul 02 14:40:01.736344 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.736345    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjpbq\" (UniqueName: \"kubernetes.io/projected/4a441638-1e97-4917-b1ff-b60ab487772f-kube-api-access-vjpbq\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b\" (UID: \"4a441638-1e97-4917-b1ff-b60ab487772f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"
Jul 02 14:40:01.736541 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.736367    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4a441638-1e97-4917-b1ff-b60ab487772f-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b\" (UID: \"4a441638-1e97-4917-b1ff-b60ab487772f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"
Jul 02 14:40:01.736541 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.736389    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4a441638-1e97-4917-b1ff-b60ab487772f-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b\" (UID: \"4a441638-1e97-4917-b1ff-b60ab487772f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"
Jul 02 14:40:01.736541 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.736417    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4a441638-1e97-4917-b1ff-b60ab487772f-cabundle-cert\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b\" (UID: \"4a441638-1e97-4917-b1ff-b60ab487772f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"
Jul 02 14:40:01.736685 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:40:01.736572    2582 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-serving-cert: secret "isvc-autogluon-ts-v1-predictor-serving-cert" not found
Jul 02 14:40:01.736685 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:40:01.736674    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a441638-1e97-4917-b1ff-b60ab487772f-proxy-tls podName:4a441638-1e97-4917-b1ff-b60ab487772f nodeName:}" failed. No retries permitted until 2026-07-02 14:40:02.236653757 +0000 UTC m=+1464.187991380 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/4a441638-1e97-4917-b1ff-b60ab487772f-proxy-tls") pod "isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b" (UID: "4a441638-1e97-4917-b1ff-b60ab487772f") : secret "isvc-autogluon-ts-v1-predictor-serving-cert" not found
Jul 02 14:40:01.736800 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.736773    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4a441638-1e97-4917-b1ff-b60ab487772f-kserve-provision-location\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b\" (UID: \"4a441638-1e97-4917-b1ff-b60ab487772f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"
Jul 02 14:40:01.737062 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.737042    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4a441638-1e97-4917-b1ff-b60ab487772f-cabundle-cert\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b\" (UID: \"4a441638-1e97-4917-b1ff-b60ab487772f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"
Jul 02 14:40:01.737163 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.737148    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4a441638-1e97-4917-b1ff-b60ab487772f-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b\" (UID: \"4a441638-1e97-4917-b1ff-b60ab487772f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"
Jul 02 14:40:01.746397 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:01.746371    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjpbq\" (UniqueName: \"kubernetes.io/projected/4a441638-1e97-4917-b1ff-b60ab487772f-kube-api-access-vjpbq\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b\" (UID: \"4a441638-1e97-4917-b1ff-b60ab487772f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"
Jul 02 14:40:02.240091 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:02.240055    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4a441638-1e97-4917-b1ff-b60ab487772f-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b\" (UID: \"4a441638-1e97-4917-b1ff-b60ab487772f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"
Jul 02 14:40:02.242515 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:02.242495    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4a441638-1e97-4917-b1ff-b60ab487772f-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b\" (UID: \"4a441638-1e97-4917-b1ff-b60ab487772f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"
Jul 02 14:40:02.416890 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:02.416846    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"
Jul 02 14:40:02.556015 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:02.555978    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"]
Jul 02 14:40:02.558240 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:40:02.558216    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a441638_1e97_4917_b1ff_b60ab487772f.slice/crio-556ffa460b0b79bfee2b11f25d6ca796cd577e25db0acfc355d2b44617faaa93 WatchSource:0}: Error finding container 556ffa460b0b79bfee2b11f25d6ca796cd577e25db0acfc355d2b44617faaa93: Status 404 returned error can't find the container with id 556ffa460b0b79bfee2b11f25d6ca796cd577e25db0acfc355d2b44617faaa93
Jul 02 14:40:02.629906 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:02.629873    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b" event={"ID":"4a441638-1e97-4917-b1ff-b60ab487772f","Type":"ContainerStarted","Data":"77bda9b9c5d1029e24940d71183b15f50fdf3a57fdd596e3847c6962bee13a4e"}
Jul 02 14:40:02.630088 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:02.629971    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b" event={"ID":"4a441638-1e97-4917-b1ff-b60ab487772f","Type":"ContainerStarted","Data":"556ffa460b0b79bfee2b11f25d6ca796cd577e25db0acfc355d2b44617faaa93"}
Jul 02 14:40:11.667855 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:11.667819    2582 generic.go:356] "Generic (PLEG): container finished" podID="4a441638-1e97-4917-b1ff-b60ab487772f" containerID="77bda9b9c5d1029e24940d71183b15f50fdf3a57fdd596e3847c6962bee13a4e" exitCode=0
Jul 02 14:40:11.668262 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:11.667872    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b" event={"ID":"4a441638-1e97-4917-b1ff-b60ab487772f","Type":"ContainerDied","Data":"77bda9b9c5d1029e24940d71183b15f50fdf3a57fdd596e3847c6962bee13a4e"}
Jul 02 14:40:12.677531 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:12.677471    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b" event={"ID":"4a441638-1e97-4917-b1ff-b60ab487772f","Type":"ContainerStarted","Data":"7249446eb004c3bd2a42f7f186b787677f172e255ad3144339d67e17c9ddd17f"}
Jul 02 14:40:12.678024 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:12.677610    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b" event={"ID":"4a441638-1e97-4917-b1ff-b60ab487772f","Type":"ContainerStarted","Data":"4cb76a38109c2eaec3fc9eff0da70d04f634fa07c244d89ed26b7cd4a1987e97"}
Jul 02 14:40:12.678894 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:12.678859    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"
Jul 02 14:40:12.679038 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:12.678942    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"
Jul 02 14:40:12.680971 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:12.680926    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b" podUID="4a441638-1e97-4917-b1ff-b60ab487772f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 02 14:40:12.699760 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:12.699697    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b" podStartSLOduration=2.869715366 podStartE2EDuration="11.699676243s" totalImagesPullingTime="0s" totalInitContainerRuntime="8.829960877s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:40:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:40:12.697435358 +0000 UTC m=+1474.648773005" watchObservedRunningTime="2026-07-02 14:40:12.699676243 +0000 UTC m=+1474.651013891"
Jul 02 14:40:13.682217 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:13.682180    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b" podUID="4a441638-1e97-4917-b1ff-b60ab487772f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 02 14:40:18.687298 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:18.687265    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"
Jul 02 14:40:18.687807 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:18.687783    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b" podUID="4a441638-1e97-4917-b1ff-b60ab487772f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Jul 02 14:40:28.688763 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:28.688733    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"
Jul 02 14:40:31.556482 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:31.556451    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"]
Jul 02 14:40:31.556901 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:31.556677    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b" podUID="4a441638-1e97-4917-b1ff-b60ab487772f" containerName="kserve-container" containerID="cri-o://4cb76a38109c2eaec3fc9eff0da70d04f634fa07c244d89ed26b7cd4a1987e97" gracePeriod=30
Jul 02 14:40:31.556901 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:31.556746    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b" podUID="4a441638-1e97-4917-b1ff-b60ab487772f" containerName="kube-rbac-proxy" containerID="cri-o://7249446eb004c3bd2a42f7f186b787677f172e255ad3144339d67e17c9ddd17f" gracePeriod=30
Jul 02 14:40:31.758389 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:31.758357    2582 generic.go:356] "Generic (PLEG): container finished" podID="4a441638-1e97-4917-b1ff-b60ab487772f" containerID="7249446eb004c3bd2a42f7f186b787677f172e255ad3144339d67e17c9ddd17f" exitCode=2
Jul 02 14:40:31.758585 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:31.758426    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b" event={"ID":"4a441638-1e97-4917-b1ff-b60ab487772f","Type":"ContainerDied","Data":"7249446eb004c3bd2a42f7f186b787677f172e255ad3144339d67e17c9ddd17f"}
Jul 02 14:40:32.401556 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.401532    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"
Jul 02 14:40:32.492576 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.492550    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4a441638-1e97-4917-b1ff-b60ab487772f-kserve-provision-location\") pod \"4a441638-1e97-4917-b1ff-b60ab487772f\" (UID: \"4a441638-1e97-4917-b1ff-b60ab487772f\") "
Jul 02 14:40:32.492739 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.492597    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4a441638-1e97-4917-b1ff-b60ab487772f-cabundle-cert\") pod \"4a441638-1e97-4917-b1ff-b60ab487772f\" (UID: \"4a441638-1e97-4917-b1ff-b60ab487772f\") "
Jul 02 14:40:32.492739 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.492617    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4a441638-1e97-4917-b1ff-b60ab487772f-kube-rbac-proxy-sar-config\") pod \"4a441638-1e97-4917-b1ff-b60ab487772f\" (UID: \"4a441638-1e97-4917-b1ff-b60ab487772f\") "
Jul 02 14:40:32.492739 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.492652    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4a441638-1e97-4917-b1ff-b60ab487772f-proxy-tls\") pod \"4a441638-1e97-4917-b1ff-b60ab487772f\" (UID: \"4a441638-1e97-4917-b1ff-b60ab487772f\") "
Jul 02 14:40:32.492739 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.492675    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjpbq\" (UniqueName: \"kubernetes.io/projected/4a441638-1e97-4917-b1ff-b60ab487772f-kube-api-access-vjpbq\") pod \"4a441638-1e97-4917-b1ff-b60ab487772f\" (UID: \"4a441638-1e97-4917-b1ff-b60ab487772f\") "
Jul 02 14:40:32.493066 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.493044    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a441638-1e97-4917-b1ff-b60ab487772f-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "4a441638-1e97-4917-b1ff-b60ab487772f" (UID: "4a441638-1e97-4917-b1ff-b60ab487772f"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:40:32.493130 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.493065    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a441638-1e97-4917-b1ff-b60ab487772f-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "4a441638-1e97-4917-b1ff-b60ab487772f" (UID: "4a441638-1e97-4917-b1ff-b60ab487772f"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 14:40:32.493485 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.493460    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a441638-1e97-4917-b1ff-b60ab487772f-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "4a441638-1e97-4917-b1ff-b60ab487772f" (UID: "4a441638-1e97-4917-b1ff-b60ab487772f"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 14:40:32.494897 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.494869    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a441638-1e97-4917-b1ff-b60ab487772f-kube-api-access-vjpbq" (OuterVolumeSpecName: "kube-api-access-vjpbq") pod "4a441638-1e97-4917-b1ff-b60ab487772f" (UID: "4a441638-1e97-4917-b1ff-b60ab487772f"). InnerVolumeSpecName "kube-api-access-vjpbq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 14:40:32.495082 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.494932    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a441638-1e97-4917-b1ff-b60ab487772f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "4a441638-1e97-4917-b1ff-b60ab487772f" (UID: "4a441638-1e97-4917-b1ff-b60ab487772f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 14:40:32.593356 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.593327    2582 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4a441638-1e97-4917-b1ff-b60ab487772f-kserve-provision-location\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:40:32.593356 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.593354    2582 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4a441638-1e97-4917-b1ff-b60ab487772f-cabundle-cert\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:40:32.593719 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.593364    2582 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4a441638-1e97-4917-b1ff-b60ab487772f-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:40:32.593719 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.593377    2582 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4a441638-1e97-4917-b1ff-b60ab487772f-proxy-tls\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:40:32.593719 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.593386    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjpbq\" (UniqueName: \"kubernetes.io/projected/4a441638-1e97-4917-b1ff-b60ab487772f-kube-api-access-vjpbq\") on node \"ip-10-0-138-160.ec2.internal\" DevicePath \"\""
Jul 02 14:40:32.764839 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.764750    2582 generic.go:356] "Generic (PLEG): container finished" podID="4a441638-1e97-4917-b1ff-b60ab487772f" containerID="4cb76a38109c2eaec3fc9eff0da70d04f634fa07c244d89ed26b7cd4a1987e97" exitCode=0
Jul 02 14:40:32.764839 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.764804    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"
Jul 02 14:40:32.765029 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.764835    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b" event={"ID":"4a441638-1e97-4917-b1ff-b60ab487772f","Type":"ContainerDied","Data":"4cb76a38109c2eaec3fc9eff0da70d04f634fa07c244d89ed26b7cd4a1987e97"}
Jul 02 14:40:32.765029 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.764899    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b" event={"ID":"4a441638-1e97-4917-b1ff-b60ab487772f","Type":"ContainerDied","Data":"556ffa460b0b79bfee2b11f25d6ca796cd577e25db0acfc355d2b44617faaa93"}
Jul 02 14:40:32.765029 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.764928    2582 scope.go:117] "RemoveContainer" containerID="7249446eb004c3bd2a42f7f186b787677f172e255ad3144339d67e17c9ddd17f"
Jul 02 14:40:32.774337 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.774322    2582 scope.go:117] "RemoveContainer" containerID="4cb76a38109c2eaec3fc9eff0da70d04f634fa07c244d89ed26b7cd4a1987e97"
Jul 02 14:40:32.781589 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.781570    2582 scope.go:117] "RemoveContainer" containerID="77bda9b9c5d1029e24940d71183b15f50fdf3a57fdd596e3847c6962bee13a4e"
Jul 02 14:40:32.788821 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.788802    2582 scope.go:117] "RemoveContainer" containerID="7249446eb004c3bd2a42f7f186b787677f172e255ad3144339d67e17c9ddd17f"
Jul 02 14:40:32.789112 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:40:32.789083    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7249446eb004c3bd2a42f7f186b787677f172e255ad3144339d67e17c9ddd17f\": container with ID starting with 7249446eb004c3bd2a42f7f186b787677f172e255ad3144339d67e17c9ddd17f not found: ID does not exist" containerID="7249446eb004c3bd2a42f7f186b787677f172e255ad3144339d67e17c9ddd17f"
Jul 02 14:40:32.789209 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.789119    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7249446eb004c3bd2a42f7f186b787677f172e255ad3144339d67e17c9ddd17f"} err="failed to get container status \"7249446eb004c3bd2a42f7f186b787677f172e255ad3144339d67e17c9ddd17f\": rpc error: code = NotFound desc = could not find container \"7249446eb004c3bd2a42f7f186b787677f172e255ad3144339d67e17c9ddd17f\": container with ID starting with 7249446eb004c3bd2a42f7f186b787677f172e255ad3144339d67e17c9ddd17f not found: ID does not exist"
Jul 02 14:40:32.789209 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.789138    2582 scope.go:117] "RemoveContainer" containerID="4cb76a38109c2eaec3fc9eff0da70d04f634fa07c244d89ed26b7cd4a1987e97"
Jul 02 14:40:32.789408 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:40:32.789388    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cb76a38109c2eaec3fc9eff0da70d04f634fa07c244d89ed26b7cd4a1987e97\": container with ID starting with 4cb76a38109c2eaec3fc9eff0da70d04f634fa07c244d89ed26b7cd4a1987e97 not found: ID does not exist" containerID="4cb76a38109c2eaec3fc9eff0da70d04f634fa07c244d89ed26b7cd4a1987e97"
Jul 02 14:40:32.789460 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.789414    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cb76a38109c2eaec3fc9eff0da70d04f634fa07c244d89ed26b7cd4a1987e97"} err="failed to get container status \"4cb76a38109c2eaec3fc9eff0da70d04f634fa07c244d89ed26b7cd4a1987e97\": rpc error: code = NotFound desc = could not find container \"4cb76a38109c2eaec3fc9eff0da70d04f634fa07c244d89ed26b7cd4a1987e97\": container with ID starting with 4cb76a38109c2eaec3fc9eff0da70d04f634fa07c244d89ed26b7cd4a1987e97 not found: ID does not exist"
Jul 02 14:40:32.789460 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.789429    2582 scope.go:117] "RemoveContainer" containerID="77bda9b9c5d1029e24940d71183b15f50fdf3a57fdd596e3847c6962bee13a4e"
Jul 02 14:40:32.789664 ip-10-0-138-160 kubenswrapper[2582]: E0702 14:40:32.789650    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77bda9b9c5d1029e24940d71183b15f50fdf3a57fdd596e3847c6962bee13a4e\": container with ID starting with 77bda9b9c5d1029e24940d71183b15f50fdf3a57fdd596e3847c6962bee13a4e not found: ID does not exist" containerID="77bda9b9c5d1029e24940d71183b15f50fdf3a57fdd596e3847c6962bee13a4e"
Jul 02 14:40:32.789729 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.789666    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77bda9b9c5d1029e24940d71183b15f50fdf3a57fdd596e3847c6962bee13a4e"} err="failed to get container status \"77bda9b9c5d1029e24940d71183b15f50fdf3a57fdd596e3847c6962bee13a4e\": rpc error: code = NotFound desc = could not find container \"77bda9b9c5d1029e24940d71183b15f50fdf3a57fdd596e3847c6962bee13a4e\": container with ID starting with 77bda9b9c5d1029e24940d71183b15f50fdf3a57fdd596e3847c6962bee13a4e not found: ID does not exist"
Jul 02 14:40:32.792760 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.792740    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"]
Jul 02 14:40:32.803544 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:32.800308    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-n4c4b"]
Jul 02 14:40:34.474370 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:34.474331    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a441638-1e97-4917-b1ff-b60ab487772f" path="/var/lib/kubelet/pods/4a441638-1e97-4917-b1ff-b60ab487772f/volumes"
Jul 02 14:40:38.516215 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:38.516189    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnwb5_ca909057-dd63-451d-9c05-b671ca5ae671/ovn-acl-logging/0.log"
Jul 02 14:40:38.516793 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:38.516776    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnwb5_ca909057-dd63-451d-9c05-b671ca5ae671/ovn-acl-logging/0.log"
Jul 02 14:40:38.581013 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:38.580988    2582 scope.go:117] "RemoveContainer" containerID="5d48f1c7daffa538f4719de3ed88653b1db866cd077accd830cc61d28c67c0dc"
Jul 02 14:40:38.588766 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:40:38.588744    2582 scope.go:117] "RemoveContainer" containerID="6dd97d2828c25c670258208d0cd785bb370113cded00b60347008163ed9247ed"
Jul 02 14:41:05.684386 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:05.684294    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sd8sv/must-gather-c7dgp"]
Jul 02 14:41:05.684977 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:05.684733    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a441638-1e97-4917-b1ff-b60ab487772f" containerName="kserve-container"
Jul 02 14:41:05.684977 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:05.684750    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a441638-1e97-4917-b1ff-b60ab487772f" containerName="kserve-container"
Jul 02 14:41:05.684977 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:05.684769    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a441638-1e97-4917-b1ff-b60ab487772f" containerName="kube-rbac-proxy"
Jul 02 14:41:05.684977 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:05.684777    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a441638-1e97-4917-b1ff-b60ab487772f" containerName="kube-rbac-proxy"
Jul 02 14:41:05.684977 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:05.684793    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a441638-1e97-4917-b1ff-b60ab487772f" containerName="storage-initializer"
Jul 02 14:41:05.684977 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:05.684803    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a441638-1e97-4917-b1ff-b60ab487772f" containerName="storage-initializer"
Jul 02 14:41:05.684977 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:05.684881    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a441638-1e97-4917-b1ff-b60ab487772f" containerName="kserve-container"
Jul 02 14:41:05.684977 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:05.684894    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a441638-1e97-4917-b1ff-b60ab487772f" containerName="kube-rbac-proxy"
Jul 02 14:41:05.689543 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:05.689519    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sd8sv/must-gather-c7dgp"
Jul 02 14:41:05.692386 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:05.692366    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-sd8sv\"/\"default-dockercfg-sb4hv\""
Jul 02 14:41:05.693825 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:05.693809    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sd8sv\"/\"kube-root-ca.crt\""
Jul 02 14:41:05.693903 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:05.693861    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sd8sv\"/\"openshift-service-ca.crt\""
Jul 02 14:41:05.697326 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:05.697303    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sd8sv/must-gather-c7dgp"]
Jul 02 14:41:05.724185 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:05.724148    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpql9\" (UniqueName: \"kubernetes.io/projected/596851df-85ca-496f-a826-18074d32e4c6-kube-api-access-mpql9\") pod \"must-gather-c7dgp\" (UID: \"596851df-85ca-496f-a826-18074d32e4c6\") " pod="openshift-must-gather-sd8sv/must-gather-c7dgp"
Jul 02 14:41:05.724354 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:05.724193    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/596851df-85ca-496f-a826-18074d32e4c6-must-gather-output\") pod \"must-gather-c7dgp\" (UID: \"596851df-85ca-496f-a826-18074d32e4c6\") " pod="openshift-must-gather-sd8sv/must-gather-c7dgp"
Jul 02 14:41:05.825162 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:05.825128    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpql9\" (UniqueName: \"kubernetes.io/projected/596851df-85ca-496f-a826-18074d32e4c6-kube-api-access-mpql9\") pod \"must-gather-c7dgp\" (UID: \"596851df-85ca-496f-a826-18074d32e4c6\") " pod="openshift-must-gather-sd8sv/must-gather-c7dgp"
Jul 02 14:41:05.825287 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:05.825174    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/596851df-85ca-496f-a826-18074d32e4c6-must-gather-output\") pod \"must-gather-c7dgp\" (UID: \"596851df-85ca-496f-a826-18074d32e4c6\") " pod="openshift-must-gather-sd8sv/must-gather-c7dgp"
Jul 02 14:41:05.825505 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:05.825490    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/596851df-85ca-496f-a826-18074d32e4c6-must-gather-output\") pod \"must-gather-c7dgp\" (UID: \"596851df-85ca-496f-a826-18074d32e4c6\") " pod="openshift-must-gather-sd8sv/must-gather-c7dgp"
Jul 02 14:41:05.833592 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:05.833564    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpql9\" (UniqueName: \"kubernetes.io/projected/596851df-85ca-496f-a826-18074d32e4c6-kube-api-access-mpql9\") pod \"must-gather-c7dgp\" (UID: \"596851df-85ca-496f-a826-18074d32e4c6\") " pod="openshift-must-gather-sd8sv/must-gather-c7dgp"
Jul 02 14:41:05.999136 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:05.999107    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sd8sv/must-gather-c7dgp"
Jul 02 14:41:06.132722 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:06.132698    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sd8sv/must-gather-c7dgp"]
Jul 02 14:41:06.135550 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:41:06.135520    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod596851df_85ca_496f_a826_18074d32e4c6.slice/crio-7881c1faa744b6f4b9e3ec9602db3980e40e40beec5290864cb2f1f41d3dec69 WatchSource:0}: Error finding container 7881c1faa744b6f4b9e3ec9602db3980e40e40beec5290864cb2f1f41d3dec69: Status 404 returned error can't find the container with id 7881c1faa744b6f4b9e3ec9602db3980e40e40beec5290864cb2f1f41d3dec69
Jul 02 14:41:06.137608 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:06.137589    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 14:41:06.905233 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:06.905196    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sd8sv/must-gather-c7dgp" event={"ID":"596851df-85ca-496f-a826-18074d32e4c6","Type":"ContainerStarted","Data":"7881c1faa744b6f4b9e3ec9602db3980e40e40beec5290864cb2f1f41d3dec69"}
Jul 02 14:41:07.913891 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:07.913536    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sd8sv/must-gather-c7dgp" event={"ID":"596851df-85ca-496f-a826-18074d32e4c6","Type":"ContainerStarted","Data":"47fcbaab59a9fb3b6d37079fdc62faa3e86add09c379f888d959069ac73aa11c"}
Jul 02 14:41:07.913891 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:07.913594    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sd8sv/must-gather-c7dgp" event={"ID":"596851df-85ca-496f-a826-18074d32e4c6","Type":"ContainerStarted","Data":"aba53e5ecb629e74268036dd8dc745d869dc270449d5cbe009d41bbe1131a31a"}
Jul 02 14:41:07.932707 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:07.932642    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-sd8sv/must-gather-c7dgp" podStartSLOduration=1.8323229140000001 podStartE2EDuration="2.932619318s" totalImagesPullingTime="1.100296404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:41:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:41:07.929985568 +0000 UTC m=+1529.881323215" watchObservedRunningTime="2026-07-02 14:41:07.932619318 +0000 UTC m=+1529.883956965"
Jul 02 14:41:08.805474 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:08.805440    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-p64q5_2bb3f329-4286-46e2-8a7d-450769d1aa15/global-pull-secret-syncer/0.log"
Jul 02 14:41:08.926486 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:08.926458    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-r6m5m_fd59e931-7137-4fa6-8b4a-3e759e2eebf1/konnectivity-agent/0.log"
Jul 02 14:41:09.007608 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:09.007578    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-160.ec2.internal_e0538fe0bb3712feb96e32d2faab018b/haproxy/0.log"
Jul 02 14:41:13.117620 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:13.117575    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-vqtb2_32981337-e63c-486f-b119-cb9fc0fe22f8/kube-state-metrics/0.log"
Jul 02 14:41:13.146138 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:13.146100    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-vqtb2_32981337-e63c-486f-b119-cb9fc0fe22f8/kube-rbac-proxy-main/0.log"
Jul 02 14:41:13.174110 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:13.173983    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-vqtb2_32981337-e63c-486f-b119-cb9fc0fe22f8/kube-rbac-proxy-self/0.log"
Jul 02 14:41:13.214791 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:13.214764    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7f787b8f48-9lc74_07974ea0-3f57-4174-b4bf-65639138b2f6/metrics-server/0.log"
Jul 02 14:41:13.249691 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:13.249637    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-cd8656b8-qhbds_b8fc3257-7848-48da-9eab-8db8a9bd26c7/monitoring-plugin/0.log"
Jul 02 14:41:13.472941 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:13.472911    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d2qn4_dd865561-ea6f-49e2-abd3-84f6870125ff/node-exporter/0.log"
Jul 02 14:41:13.502164 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:13.502134    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d2qn4_dd865561-ea6f-49e2-abd3-84f6870125ff/kube-rbac-proxy/0.log"
Jul 02 14:41:13.524589 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:13.524560    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d2qn4_dd865561-ea6f-49e2-abd3-84f6870125ff/init-textfile/0.log"
Jul 02 14:41:17.229974 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:17.229920    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sd8sv/perf-node-gather-daemonset-rf9cj"]
Jul 02 14:41:17.235878 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:17.235842    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sd8sv/perf-node-gather-daemonset-rf9cj"
Jul 02 14:41:17.248518 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:17.248431    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sd8sv/perf-node-gather-daemonset-rf9cj"]
Jul 02 14:41:17.329682 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:17.329649    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b528d23e-39ae-4a70-b064-9d7d970aa75d-podres\") pod \"perf-node-gather-daemonset-rf9cj\" (UID: \"b528d23e-39ae-4a70-b064-9d7d970aa75d\") " pod="openshift-must-gather-sd8sv/perf-node-gather-daemonset-rf9cj"
Jul 02 14:41:17.329881 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:17.329689    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b528d23e-39ae-4a70-b064-9d7d970aa75d-proc\") pod \"perf-node-gather-daemonset-rf9cj\" (UID: \"b528d23e-39ae-4a70-b064-9d7d970aa75d\") " pod="openshift-must-gather-sd8sv/perf-node-gather-daemonset-rf9cj"
Jul 02 14:41:17.329881 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:17.329718    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b528d23e-39ae-4a70-b064-9d7d970aa75d-lib-modules\") pod \"perf-node-gather-daemonset-rf9cj\" (UID: \"b528d23e-39ae-4a70-b064-9d7d970aa75d\") " pod="openshift-must-gather-sd8sv/perf-node-gather-daemonset-rf9cj"
Jul 02 14:41:17.329881 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:17.329736    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9vcv\" (UniqueName: \"kubernetes.io/projected/b528d23e-39ae-4a70-b064-9d7d970aa75d-kube-api-access-b9vcv\") pod \"perf-node-gather-daemonset-rf9cj\" (UID: \"b528d23e-39ae-4a70-b064-9d7d970aa75d\") " pod="openshift-must-gather-sd8sv/perf-node-gather-daemonset-rf9cj"
Jul 02 14:41:17.329881 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:17.329770    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b528d23e-39ae-4a70-b064-9d7d970aa75d-sys\") pod \"perf-node-gather-daemonset-rf9cj\" (UID: \"b528d23e-39ae-4a70-b064-9d7d970aa75d\") " pod="openshift-must-gather-sd8sv/perf-node-gather-daemonset-rf9cj"
Jul 02 14:41:17.431015 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:17.430945    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b528d23e-39ae-4a70-b064-9d7d970aa75d-sys\") pod \"perf-node-gather-daemonset-rf9cj\" (UID: \"b528d23e-39ae-4a70-b064-9d7d970aa75d\") " pod="openshift-must-gather-sd8sv/perf-node-gather-daemonset-rf9cj"
Jul 02 14:41:17.431269 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:17.431219    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b528d23e-39ae-4a70-b064-9d7d970aa75d-sys\") pod \"perf-node-gather-daemonset-rf9cj\" (UID: \"b528d23e-39ae-4a70-b064-9d7d970aa75d\") " pod="openshift-must-gather-sd8sv/perf-node-gather-daemonset-rf9cj"
Jul 02 14:41:17.431405 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:17.431340    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b528d23e-39ae-4a70-b064-9d7d970aa75d-podres\") pod \"perf-node-gather-daemonset-rf9cj\" (UID: \"b528d23e-39ae-4a70-b064-9d7d970aa75d\") " pod="openshift-must-gather-sd8sv/perf-node-gather-daemonset-rf9cj"
Jul 02 14:41:17.431908 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:17.431498    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b528d23e-39ae-4a70-b064-9d7d970aa75d-proc\") pod \"perf-node-gather-daemonset-rf9cj\" (UID: \"b528d23e-39ae-4a70-b064-9d7d970aa75d\") " pod="openshift-must-gather-sd8sv/perf-node-gather-daemonset-rf9cj"
Jul 02 14:41:17.431908 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:17.431512    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b528d23e-39ae-4a70-b064-9d7d970aa75d-podres\") pod \"perf-node-gather-daemonset-rf9cj\" (UID: \"b528d23e-39ae-4a70-b064-9d7d970aa75d\") " pod="openshift-must-gather-sd8sv/perf-node-gather-daemonset-rf9cj"
Jul 02 14:41:17.431908 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:17.431549    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b528d23e-39ae-4a70-b064-9d7d970aa75d-lib-modules\") pod \"perf-node-gather-daemonset-rf9cj\" (UID: \"b528d23e-39ae-4a70-b064-9d7d970aa75d\") " pod="openshift-must-gather-sd8sv/perf-node-gather-daemonset-rf9cj"
Jul 02 14:41:17.431908 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:17.431577    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9vcv\" (UniqueName: \"kubernetes.io/projected/b528d23e-39ae-4a70-b064-9d7d970aa75d-kube-api-access-b9vcv\") pod \"perf-node-gather-daemonset-rf9cj\" (UID: \"b528d23e-39ae-4a70-b064-9d7d970aa75d\") " pod="openshift-must-gather-sd8sv/perf-node-gather-daemonset-rf9cj"
Jul 02 14:41:17.431908 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:17.431590    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b528d23e-39ae-4a70-b064-9d7d970aa75d-proc\") pod \"perf-node-gather-daemonset-rf9cj\" (UID: \"b528d23e-39ae-4a70-b064-9d7d970aa75d\") " pod="openshift-must-gather-sd8sv/perf-node-gather-daemonset-rf9cj"
Jul 02 14:41:17.431908 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:17.431692    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b528d23e-39ae-4a70-b064-9d7d970aa75d-lib-modules\") pod \"perf-node-gather-daemonset-rf9cj\" (UID: \"b528d23e-39ae-4a70-b064-9d7d970aa75d\") " pod="openshift-must-gather-sd8sv/perf-node-gather-daemonset-rf9cj"
Jul 02 14:41:17.441181 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:17.441151    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9vcv\" (UniqueName: \"kubernetes.io/projected/b528d23e-39ae-4a70-b064-9d7d970aa75d-kube-api-access-b9vcv\") pod \"perf-node-gather-daemonset-rf9cj\" (UID: \"b528d23e-39ae-4a70-b064-9d7d970aa75d\") " pod="openshift-must-gather-sd8sv/perf-node-gather-daemonset-rf9cj"
Jul 02 14:41:17.548979 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:17.548887    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sd8sv/perf-node-gather-daemonset-rf9cj"
Jul 02 14:41:17.733998 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:17.733947    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sd8sv/perf-node-gather-daemonset-rf9cj"]
Jul 02 14:41:17.736170 ip-10-0-138-160 kubenswrapper[2582]: W0702 14:41:17.736139    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb528d23e_39ae_4a70_b064_9d7d970aa75d.slice/crio-4a56bf0616004b7a7c53405cdc3acda51b53066f40a9b648fed10f8dc1dfdf68 WatchSource:0}: Error finding container 4a56bf0616004b7a7c53405cdc3acda51b53066f40a9b648fed10f8dc1dfdf68: Status 404 returned error can't find the container with id 4a56bf0616004b7a7c53405cdc3acda51b53066f40a9b648fed10f8dc1dfdf68
Jul 02 14:41:17.973621 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:17.973576    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sd8sv/perf-node-gather-daemonset-rf9cj" event={"ID":"b528d23e-39ae-4a70-b064-9d7d970aa75d","Type":"ContainerStarted","Data":"6970c2b9cf9d4ce0c9fe337c9351eb264f10ed69aaa0461bba4ba36c2e0b76c9"}
Jul 02 14:41:17.973798 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:17.973654    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sd8sv/perf-node-gather-daemonset-rf9cj" event={"ID":"b528d23e-39ae-4a70-b064-9d7d970aa75d","Type":"ContainerStarted","Data":"4a56bf0616004b7a7c53405cdc3acda51b53066f40a9b648fed10f8dc1dfdf68"}
Jul 02 14:41:17.973798 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:17.973679    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-sd8sv/perf-node-gather-daemonset-rf9cj"
Jul 02 14:41:17.987946 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:17.987922    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-295qn_977cb808-d191-4c87-b63b-8efe4dc2d00f/dns/0.log"
Jul 02 14:41:17.991590 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:17.991545    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-sd8sv/perf-node-gather-daemonset-rf9cj" podStartSLOduration=0.991529388 podStartE2EDuration="991.529388ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:41:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:41:17.990715669 +0000 UTC m=+1539.942053319" watchObservedRunningTime="2026-07-02 14:41:17.991529388 +0000 UTC m=+1539.942867091"
Jul 02 14:41:18.010881 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:18.010858    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-295qn_977cb808-d191-4c87-b63b-8efe4dc2d00f/kube-rbac-proxy/0.log"
Jul 02 14:41:18.137426 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:18.137391    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-f8pbz_76237012-8c88-464a-a19a-dfc41536ea5c/dns-node-resolver/0.log"
Jul 02 14:41:18.713702 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:18.713670    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rtn2p_4cb188f8-101f-4d6f-b035-90cf16106a5e/node-ca/0.log"
Jul 02 14:41:20.053839 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:20.053806    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rlkvc_cc14ce21-e15d-484d-b934-bb9409c53413/serve-healthcheck-canary/0.log"
Jul 02 14:41:20.618465 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:20.618432    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-srnvs_196e2ce2-d807-46d8-81e5-ff0b627c3b08/kube-rbac-proxy/0.log"
Jul 02 14:41:20.643044 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:20.643013    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-srnvs_196e2ce2-d807-46d8-81e5-ff0b627c3b08/exporter/0.log"
Jul 02 14:41:20.673407 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:20.673363    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-srnvs_196e2ce2-d807-46d8-81e5-ff0b627c3b08/extractor/0.log"
Jul 02 14:41:22.813261 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:22.813217    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-fd65d7d6-7tpvc_41b77d29-74e2-4d81-9f5a-a8c31be03d7e/server/0.log"
Jul 02 14:41:22.890066 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:22.890034    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-84b54b7f97-fffqj_52a07e67-58cb-48fe-8dfe-dce1949d9c2f/manager/0.log"
Jul 02 14:41:23.089664 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:23.089581    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-64568bcd49-5dv4p_e0a3971e-4c32-4087-9dd6-73c316a9e1ec/seaweedfs/0.log"
Jul 02 14:41:23.115540 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:23.115505    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-tls-custom-8b44d767d-qcwpj_3a5e2d1b-fe84-4da2-a2ef-c20c2b100e53/seaweedfs-tls-custom/0.log"
Jul 02 14:41:23.141294 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:23.141264    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-tls-serving-67b7477c6d-gkts6_40676d2b-47dc-4415-8958-6498fc987001/seaweedfs-tls-serving/0.log"
Jul 02 14:41:23.992082 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:23.992042    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-sd8sv/perf-node-gather-daemonset-rf9cj"
Jul 02 14:41:27.450361 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:27.450322    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-786f9fbc5-zmsd5_5b2261d4-525c-425a-abd6-ee77b01d6392/migrator/0.log"
Jul 02 14:41:27.475547 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:27.475518    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-786f9fbc5-zmsd5_5b2261d4-525c-425a-abd6-ee77b01d6392/graceful-termination/0.log"
Jul 02 14:41:29.168632 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:29.168599    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p96q8_2ff40ff7-e770-4d86-b327-b437a9cfa15a/kube-multus-additional-cni-plugins/0.log"
Jul 02 14:41:29.191864 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:29.191835    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p96q8_2ff40ff7-e770-4d86-b327-b437a9cfa15a/egress-router-binary-copy/0.log"
Jul 02 14:41:29.215264 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:29.215232    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p96q8_2ff40ff7-e770-4d86-b327-b437a9cfa15a/cni-plugins/0.log"
Jul 02 14:41:29.239432 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:29.239405    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p96q8_2ff40ff7-e770-4d86-b327-b437a9cfa15a/bond-cni-plugin/0.log"
Jul 02 14:41:29.262776 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:29.262751    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p96q8_2ff40ff7-e770-4d86-b327-b437a9cfa15a/routeoverride-cni/0.log"
Jul 02 14:41:29.288537 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:29.288465    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p96q8_2ff40ff7-e770-4d86-b327-b437a9cfa15a/whereabouts-cni-bincopy/0.log"
Jul 02 14:41:29.310859 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:29.310822    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p96q8_2ff40ff7-e770-4d86-b327-b437a9cfa15a/whereabouts-cni/0.log"
Jul 02 14:41:29.371782 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:29.371750    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-r2rq9_929e19e8-c3ec-4f7a-810f-cd805cf058f6/kube-multus/0.log"
Jul 02 14:41:29.422192 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:29.422165    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lpdzr_ada3d319-cd6a-42ad-b8f9-c04da51705ef/network-metrics-daemon/0.log"
Jul 02 14:41:29.443785 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:29.443760    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lpdzr_ada3d319-cd6a-42ad-b8f9-c04da51705ef/kube-rbac-proxy/0.log"
Jul 02 14:41:30.576236 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:30.576204    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnwb5_ca909057-dd63-451d-9c05-b671ca5ae671/ovn-controller/0.log"
Jul 02 14:41:30.597939 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:30.597892    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnwb5_ca909057-dd63-451d-9c05-b671ca5ae671/ovn-acl-logging/0.log"
Jul 02 14:41:30.597939 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:30.597892    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnwb5_ca909057-dd63-451d-9c05-b671ca5ae671/ovn-acl-logging/1.log"
Jul 02 14:41:30.617860 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:30.617820    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnwb5_ca909057-dd63-451d-9c05-b671ca5ae671/kube-rbac-proxy-node/0.log"
Jul 02 14:41:30.642360 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:30.642333    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnwb5_ca909057-dd63-451d-9c05-b671ca5ae671/kube-rbac-proxy-ovn-metrics/0.log"
Jul 02 14:41:30.665091 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:30.665028    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnwb5_ca909057-dd63-451d-9c05-b671ca5ae671/northd/0.log"
Jul 02 14:41:30.693553 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:30.693513    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnwb5_ca909057-dd63-451d-9c05-b671ca5ae671/nbdb/0.log"
Jul 02 14:41:30.718616 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:30.718588    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnwb5_ca909057-dd63-451d-9c05-b671ca5ae671/sbdb/0.log"
Jul 02 14:41:30.762859 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:30.762830    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cnwb5_ca909057-dd63-451d-9c05-b671ca5ae671/ovnkube-controller/0.log"
Jul 02 14:41:32.044744 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:32.044710    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-7gvk9_81c88401-37f6-493c-80ad-c65003c62e57/network-check-target-container/0.log"
Jul 02 14:41:33.063526 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:33.063499    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-v5829_b96a56ee-b8ff-41b1-8470-13914745bc35/iptables-alerter/0.log"
Jul 02 14:41:33.741634 ip-10-0-138-160 kubenswrapper[2582]: I0702 14:41:33.741605    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-q6zgx_9b8d6940-2ec0-442f-ae0f-0e7221348432/tuned/0.log"