Jul 08 16:34:40.525641 ip-10-0-139-237 systemd[1]: Starting Kubernetes Kubelet...
Jul 08 16:34:40.955570 ip-10-0-139-237 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 16:34:40.955570 ip-10-0-139-237 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 08 16:34:40.955570 ip-10-0-139-237 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 16:34:40.955570 ip-10-0-139-237 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 08 16:34:40.955570 ip-10-0-139-237 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 16:34:40.956923 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.955601    2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 08 16:34:40.959287 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959262    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 16:34:40.959287 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959283    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 16:34:40.959287 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959288    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 16:34:40.959287 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959292    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 16:34:40.959287 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959296    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 16:34:40.959590 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959302    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 16:34:40.959590 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959306    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 16:34:40.959590 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959325    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 16:34:40.959590 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959330    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 16:34:40.959590 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959334    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 16:34:40.959590 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959338    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 16:34:40.959590 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959341    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 16:34:40.959590 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959345    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 16:34:40.959590 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959348    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 16:34:40.959590 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959351    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 16:34:40.959590 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959355    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 16:34:40.959590 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959362    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 16:34:40.959590 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959367    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 16:34:40.959590 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959371    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 16:34:40.959590 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959375    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 16:34:40.959590 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959380    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 16:34:40.959590 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959384    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 16:34:40.959590 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959388    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 16:34:40.959590 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959392    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 16:34:40.959590 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959396    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 16:34:40.960427 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959400    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 16:34:40.960427 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959404    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 16:34:40.960427 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959407    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 16:34:40.960427 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959411    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 16:34:40.960427 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959415    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 16:34:40.960427 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959419    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 16:34:40.960427 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959423    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 16:34:40.960427 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959427    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 16:34:40.960427 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959431    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 16:34:40.960427 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959436    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 16:34:40.960427 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959440    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 16:34:40.960427 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959444    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 16:34:40.960427 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959449    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 16:34:40.960427 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959454    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 16:34:40.960427 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959458    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 16:34:40.960427 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959463    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 16:34:40.960427 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959468    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 16:34:40.960427 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959472    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 16:34:40.960427 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959477    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 16:34:40.960427 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959481    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 16:34:40.961030 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959485    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 16:34:40.961030 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959489    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 16:34:40.961030 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959493    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 16:34:40.961030 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959498    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 16:34:40.961030 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959502    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 16:34:40.961030 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959506    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 16:34:40.961030 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959510    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 16:34:40.961030 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959514    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 16:34:40.961030 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959519    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 16:34:40.961030 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959523    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 16:34:40.961030 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959527    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 16:34:40.961030 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959531    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 16:34:40.961030 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959536    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 16:34:40.961030 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959540    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 16:34:40.961030 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959546    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 16:34:40.961030 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959550    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 16:34:40.961030 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959553    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 16:34:40.961030 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959558    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 16:34:40.961030 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959563    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 16:34:40.961030 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959567    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 16:34:40.961593 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959573    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 16:34:40.961593 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959580    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 16:34:40.961593 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959585    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 16:34:40.961593 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959590    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 16:34:40.961593 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959594    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 16:34:40.961593 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959602    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 16:34:40.961593 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959608    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 16:34:40.961593 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959612    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 16:34:40.961593 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959617    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 16:34:40.961593 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959621    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 16:34:40.961593 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959625    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 16:34:40.961593 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959629    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 16:34:40.961593 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959633    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 16:34:40.961593 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959638    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 16:34:40.961593 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959642    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 16:34:40.961593 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959646    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 16:34:40.961593 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959650    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 16:34:40.961593 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959654    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 16:34:40.961593 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959659    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 16:34:40.962062 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959663    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 16:34:40.962062 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959667    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 16:34:40.962062 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959672    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 16:34:40.962062 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959675    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 16:34:40.962062 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959682    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 16:34:40.962062 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959688    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 16:34:40.962062 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959693    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 16:34:40.962062 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959697    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 16:34:40.962062 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959701    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 16:34:40.962062 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959705    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 16:34:40.962062 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959710    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 16:34:40.962062 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959714    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 16:34:40.962062 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959718    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 16:34:40.962062 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959723    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 16:34:40.962062 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.959727    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 16:34:40.962755 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962373    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 16:34:40.962755 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962385    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 16:34:40.962755 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962390    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 16:34:40.962755 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962395    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 16:34:40.962755 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962400    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 16:34:40.962755 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962406    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 16:34:40.962755 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962411    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 16:34:40.962755 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962416    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 16:34:40.962755 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962421    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 16:34:40.962755 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962425    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 16:34:40.962755 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962429    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 16:34:40.962755 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962434    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 16:34:40.962755 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962438    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 16:34:40.962755 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962442    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 16:34:40.962755 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962447    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 16:34:40.962755 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962451    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 16:34:40.962755 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962455    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 16:34:40.962755 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962459    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 16:34:40.962755 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962463    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 16:34:40.962755 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962467    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 16:34:40.963618 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962471    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 16:34:40.963618 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962476    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 16:34:40.963618 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962480    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 16:34:40.963618 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962484    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 16:34:40.963618 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962488    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 16:34:40.963618 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962492    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 16:34:40.963618 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962496    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 16:34:40.963618 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962501    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 16:34:40.963618 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962505    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 16:34:40.963618 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962509    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 16:34:40.963618 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962514    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 16:34:40.963618 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962518    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 16:34:40.963618 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962522    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 16:34:40.963618 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962527    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 16:34:40.963618 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962531    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 16:34:40.963618 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962538    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 16:34:40.963618 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962544    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 16:34:40.963618 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962548    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 16:34:40.963618 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962555    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 16:34:40.964494 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962560    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 16:34:40.964494 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962565    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 16:34:40.964494 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962569    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 16:34:40.964494 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962575    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 16:34:40.964494 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962580    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 16:34:40.964494 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962585    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 16:34:40.964494 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962589    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 16:34:40.964494 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962593    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 16:34:40.964494 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962598    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 16:34:40.964494 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962602    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 16:34:40.964494 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962607    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 16:34:40.964494 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962611    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 16:34:40.964494 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962615    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 16:34:40.964494 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962619    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 16:34:40.964494 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962623    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 16:34:40.964494 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962627    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 16:34:40.964494 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962632    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 16:34:40.964494 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962636    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 16:34:40.964494 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962641    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 16:34:40.964494 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962645    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 16:34:40.965395 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962649    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 16:34:40.965395 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962654    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 16:34:40.965395 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962658    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 16:34:40.965395 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962662    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 16:34:40.965395 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962668    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 16:34:40.965395 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962674    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 16:34:40.965395 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962680    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 16:34:40.965395 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962684    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 16:34:40.965395 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962688    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 16:34:40.965395 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962693    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 16:34:40.965395 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962699    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 16:34:40.965395 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962703    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 16:34:40.965395 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962716    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 16:34:40.965395 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962721    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 16:34:40.965395 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962725    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 16:34:40.965395 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962730    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 16:34:40.965395 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962734    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 16:34:40.965395 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962738    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 16:34:40.965395 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962742    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 16:34:40.965395 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962746    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 16:34:40.965993 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962750    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 16:34:40.965993 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962754    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 16:34:40.965993 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962759    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 16:34:40.965993 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962763    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 16:34:40.965993 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962767    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 16:34:40.965993 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962771    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 16:34:40.965993 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962777    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 16:34:40.965993 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962782    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 16:34:40.965993 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962787    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 16:34:40.965993 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962792    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 16:34:40.965993 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962797    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 16:34:40.965993 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962801    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 16:34:40.965993 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962805    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 16:34:40.965993 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962810    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 16:34:40.965993 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962814    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 16:34:40.965993 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962818    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 16:34:40.965993 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962822    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 16:34:40.965993 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962826    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 16:34:40.965993 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962831    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 16:34:40.966524 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.962835    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 16:34:40.966524 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963522    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 16:34:40.966524 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963531    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 16:34:40.966524 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963535    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 16:34:40.966524 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963539    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 16:34:40.966524 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963544    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 16:34:40.966524 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963549    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 16:34:40.966524 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963553    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 16:34:40.966524 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963558    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 16:34:40.966524 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963563    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 16:34:40.966524 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963567    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 16:34:40.966524 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963571    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 16:34:40.966524 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963575    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 16:34:40.966524 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963579    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 16:34:40.966524 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963583    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 16:34:40.966524 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963588    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 16:34:40.966524 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963593    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 16:34:40.966524 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963597    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 16:34:40.966524 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963601    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 16:34:40.966524 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963605    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 16:34:40.967110 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963609    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 16:34:40.967110 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963613    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 16:34:40.967110 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963617    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 16:34:40.967110 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963621    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 16:34:40.967110 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963625    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 16:34:40.967110 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963629    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 16:34:40.967110 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963633    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 16:34:40.967110 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963636    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 16:34:40.967110 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963641    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 16:34:40.967110 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963644    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 16:34:40.967110 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963649    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 16:34:40.967110 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963654    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 16:34:40.967110 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963658    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 16:34:40.967110 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963663    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 16:34:40.967110 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963667    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 16:34:40.967110 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963671    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 16:34:40.967110 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963675    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 16:34:40.967110 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963679    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 16:34:40.967110 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963689    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 16:34:40.967745 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963693    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 16:34:40.967745 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963700    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 16:34:40.967745 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963705    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 16:34:40.967745 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963710    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 16:34:40.967745 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963714    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 16:34:40.967745 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963718    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 16:34:40.967745 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963722    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 16:34:40.967745 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963727    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 16:34:40.967745 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963731    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 16:34:40.967745 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963736    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 16:34:40.967745 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963742    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 16:34:40.967745 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963746    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 16:34:40.967745 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963751    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 16:34:40.967745 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963755    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 16:34:40.967745 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963759    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 16:34:40.967745 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963764    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 16:34:40.967745 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963768    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 16:34:40.967745 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963772    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 16:34:40.967745 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963776    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 16:34:40.968209 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963780    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 16:34:40.968209 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963784    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 16:34:40.968209 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963788    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 16:34:40.968209 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963793    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 16:34:40.968209 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963797    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 16:34:40.968209 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963804    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 16:34:40.968209 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963809    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 16:34:40.968209 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963815    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 16:34:40.968209 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963819    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 16:34:40.968209 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963824    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 16:34:40.968209 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963829    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 16:34:40.968209 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963833    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 16:34:40.968209 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963837    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 16:34:40.968209 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963849    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 16:34:40.968209 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963853    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 16:34:40.968209 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963857    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 16:34:40.968209 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963861    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 16:34:40.968209 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963865    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 16:34:40.968209 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963870    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 16:34:40.968721 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963875    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 16:34:40.968721 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963878    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 16:34:40.968721 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963883    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 16:34:40.968721 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963887    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 16:34:40.968721 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963892    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 16:34:40.968721 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963896    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 16:34:40.968721 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963900    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 16:34:40.968721 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963908    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 16:34:40.968721 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963914    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 16:34:40.968721 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963919    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 16:34:40.968721 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963923    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 16:34:40.968721 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963928    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 16:34:40.968721 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963932    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 16:34:40.968721 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963936    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 16:34:40.968721 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963940    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 16:34:40.968721 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963944    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 16:34:40.968721 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963948    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 16:34:40.968721 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963952    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 16:34:40.968721 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963958    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 16:34:40.968721 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963962    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 16:34:40.969254 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963966    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 16:34:40.969254 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963970    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 16:34:40.969254 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.963974    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 16:34:40.969254 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964094    2582 flags.go:64] FLAG: --address="0.0.0.0"
Jul 08 16:34:40.969254 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964107    2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 08 16:34:40.969254 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964116    2582 flags.go:64] FLAG: --anonymous-auth="true"
Jul 08 16:34:40.969254 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964123    2582 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 08 16:34:40.969254 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964135    2582 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 08 16:34:40.969254 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964141    2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 08 16:34:40.969254 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964149    2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 08 16:34:40.969254 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964156    2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 08 16:34:40.969254 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964161    2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 08 16:34:40.969254 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964166    2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 08 16:34:40.969254 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964171    2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 08 16:34:40.969254 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964177    2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 08 16:34:40.969254 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964185    2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 08 16:34:40.969254 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964190    2582 flags.go:64] FLAG: --cgroup-root=""
Jul 08 16:34:40.969254 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964195    2582 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 08 16:34:40.969254 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964199    2582 flags.go:64] FLAG: --client-ca-file=""
Jul 08 16:34:40.969254 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964204    2582 flags.go:64] FLAG: --cloud-provider="external"
Jul 08 16:34:40.969254 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964209    2582 flags.go:64] FLAG: --cluster-dns="[]"
Jul 08 16:34:40.969254 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964216    2582 flags.go:64] FLAG: --cluster-domain=""
Jul 08 16:34:40.969254 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964220    2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 08 16:34:40.969254 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964226    2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 08 16:34:40.969866 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964231    2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 08 16:34:40.969866 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964236    2582 flags.go:64] FLAG: --container-log-max-files="5"
Jul 08 16:34:40.969866 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964242    2582 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 08 16:34:40.969866 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964247    2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 08 16:34:40.969866 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964252    2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 08 16:34:40.969866 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964257    2582 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 08 16:34:40.969866 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964266    2582 flags.go:64] FLAG: --contention-profiling="false"
Jul 08 16:34:40.969866 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964271    2582 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 08 16:34:40.969866 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964276    2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 08 16:34:40.969866 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964281    2582 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 08 16:34:40.969866 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964285    2582 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 08 16:34:40.969866 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964292    2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 08 16:34:40.969866 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964297    2582 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 08 16:34:40.969866 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964302    2582 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 08 16:34:40.969866 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964306    2582 flags.go:64] FLAG: --enable-load-reader="false"
Jul 08 16:34:40.969866 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964331    2582 flags.go:64] FLAG: --enable-server="true"
Jul 08 16:34:40.969866 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964337    2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 08 16:34:40.969866 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964344    2582 flags.go:64] FLAG: --event-burst="100"
Jul 08 16:34:40.969866 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964349    2582 flags.go:64] FLAG: --event-qps="50"
Jul 08 16:34:40.969866 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964353    2582 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 08 16:34:40.969866 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964359    2582 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 08 16:34:40.969866 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964363    2582 flags.go:64] FLAG: --eviction-hard=""
Jul 08 16:34:40.969866 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964369    2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 08 16:34:40.969866 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964375    2582 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 08 16:34:40.970604 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964380    2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 08 16:34:40.970604 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964385    2582 flags.go:64] FLAG: --eviction-soft=""
Jul 08 16:34:40.970604 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964390    2582 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 08 16:34:40.970604 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964396    2582 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 08 16:34:40.970604 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964400    2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 08 16:34:40.970604 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964405    2582 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 08 16:34:40.970604 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964409    2582 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 08 16:34:40.970604 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964414    2582 flags.go:64] FLAG: --fail-swap-on="true"
Jul 08 16:34:40.970604 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964418    2582 flags.go:64] FLAG: --feature-gates=""
Jul 08 16:34:40.970604 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964425    2582 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 08 16:34:40.970604 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964429    2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 08 16:34:40.970604 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964434    2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 08 16:34:40.970604 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964439    2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 08 16:34:40.970604 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964444    2582 flags.go:64] FLAG: --healthz-port="10248"
Jul 08 16:34:40.970604 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964451    2582 flags.go:64] FLAG: --help="false"
Jul 08 16:34:40.970604 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964455    2582 flags.go:64] FLAG: --hostname-override="ip-10-0-139-237.ec2.internal"
Jul 08 16:34:40.970604 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964460    2582 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 08 16:34:40.970604 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964465    2582 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 08 16:34:40.970604 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964470    2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 08 16:34:40.970604 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964475    2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 08 16:34:40.970604 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964481    2582 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 08 16:34:40.970604 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964486    2582 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 08 16:34:40.970604 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964490    2582 flags.go:64] FLAG: --image-service-endpoint=""
Jul 08 16:34:40.971218 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964495    2582 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 08 16:34:40.971218 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964499    2582 flags.go:64] FLAG: --kube-api-burst="100"
Jul 08 16:34:40.971218 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964504    2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 08 16:34:40.971218 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964509    2582 flags.go:64] FLAG: --kube-api-qps="50"
Jul 08 16:34:40.971218 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964514    2582 flags.go:64] FLAG: --kube-reserved=""
Jul 08 16:34:40.971218 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964519    2582 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 08 16:34:40.971218 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964523    2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 08 16:34:40.971218 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964529    2582 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 08 16:34:40.971218 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964534    2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 08 16:34:40.971218 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964538    2582 flags.go:64] FLAG: --lock-file=""
Jul 08 16:34:40.971218 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964543    2582 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 08 16:34:40.971218 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964547    2582 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 08 16:34:40.971218 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964552    2582 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 08 16:34:40.971218 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964561    2582 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 08 16:34:40.971218 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964565    2582 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 08 16:34:40.971218 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964570    2582 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 08 16:34:40.971218 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964574    2582 flags.go:64] FLAG: --logging-format="text"
Jul 08 16:34:40.971218 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964579    2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 08 16:34:40.971218 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964584    2582 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 08 16:34:40.971218 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964588    2582 flags.go:64] FLAG: --manifest-url=""
Jul 08 16:34:40.971218 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964593    2582 flags.go:64] FLAG: --manifest-url-header=""
Jul 08 16:34:40.971218 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964599    2582 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 08 16:34:40.971218 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964604    2582 flags.go:64] FLAG: --max-open-files="1000000"
Jul 08 16:34:40.971218 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964612    2582 flags.go:64] FLAG: --max-pods="110"
Jul 08 16:34:40.971218 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964617    2582 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 08 16:34:40.971843 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964622    2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 08 16:34:40.971843 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964627    2582 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 08 16:34:40.971843 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964632    2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 08 16:34:40.971843 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964636    2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 08 16:34:40.971843 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964641    2582 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 08 16:34:40.971843 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964646    2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 08 16:34:40.971843 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964657    2582 flags.go:64] FLAG: --node-status-max-images="50"
Jul 08 16:34:40.971843 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964663    2582 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 08 16:34:40.971843 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964668    2582 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 08 16:34:40.971843 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964673    2582 flags.go:64] FLAG: --pod-cidr=""
Jul 08 16:34:40.971843 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964686    2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 08 16:34:40.971843 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964694    2582 flags.go:64] FLAG: --pod-manifest-path=""
Jul 08 16:34:40.971843 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964700    2582 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 08 16:34:40.971843 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964705    2582 flags.go:64] FLAG: --pods-per-core="0"
Jul 08 16:34:40.971843 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964709    2582 flags.go:64] FLAG: --port="10250"
Jul 08 16:34:40.971843 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964714    2582 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 08 16:34:40.971843 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964718    2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0cf7b8f7f0cd042e7"
Jul 08 16:34:40.971843 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964723    2582 flags.go:64] FLAG: --qos-reserved=""
Jul 08 16:34:40.971843 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964728    2582 flags.go:64] FLAG: --read-only-port="10255"
Jul 08 16:34:40.971843 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964733    2582 flags.go:64] FLAG: --register-node="true"
Jul 08 16:34:40.971843 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964737    2582 flags.go:64] FLAG: --register-with-taints=""
Jul 08 16:34:40.971843 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964743    2582 flags.go:64] FLAG: --registry-burst="10"
Jul 08 16:34:40.971843 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964748    2582 flags.go:64] FLAG: --registry-qps="5"
Jul 08 16:34:40.971843 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964753    2582 flags.go:64] FLAG: --reserved-cpus=""
Jul 08 16:34:40.972445 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964757    2582 flags.go:64] FLAG: --reserved-memory=""
Jul 08 16:34:40.972445 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964763    2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 08 16:34:40.972445 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964768    2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 08 16:34:40.972445 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964773    2582 flags.go:64] FLAG: --rotate-certificates="false"
Jul 08 16:34:40.972445 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964778    2582 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 08 16:34:40.972445 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964782    2582 flags.go:64] FLAG: --runonce="false"
Jul 08 16:34:40.972445 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964789    2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 08 16:34:40.972445 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964794    2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 08 16:34:40.972445 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964799    2582 flags.go:64] FLAG: --seccomp-default="false"
Jul 08 16:34:40.972445 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964803    2582 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 08 16:34:40.972445 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964808    2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 08 16:34:40.972445 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964813    2582 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 08 16:34:40.972445 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964817    2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 08 16:34:40.972445 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964822    2582 flags.go:64] FLAG: --storage-driver-password="root"
Jul 08 16:34:40.972445 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964827    2582 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 08 16:34:40.972445 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964831    2582 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 08 16:34:40.972445 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964844    2582 flags.go:64] FLAG: --storage-driver-user="root"
Jul 08 16:34:40.972445 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964849    2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 08 16:34:40.972445 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964854    2582 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 08 16:34:40.972445 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964866    2582 flags.go:64] FLAG: --system-cgroups=""
Jul 08 16:34:40.972445 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964871    2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 08 16:34:40.972445 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964880    2582 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 08 16:34:40.972445 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964885    2582 flags.go:64] FLAG: --tls-cert-file=""
Jul 08 16:34:40.972445 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964889    2582 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 08 16:34:40.972445 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964901    2582 flags.go:64] FLAG: --tls-min-version=""
Jul 08 16:34:40.973046 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964906    2582 flags.go:64] FLAG: --tls-private-key-file=""
Jul 08 16:34:40.973046 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964911    2582 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 08 16:34:40.973046 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964916    2582 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 08 16:34:40.973046 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964920    2582 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 08 16:34:40.973046 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964925    2582 flags.go:64] FLAG: --v="2"
Jul 08 16:34:40.973046 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964931    2582 flags.go:64] FLAG: --version="false"
Jul 08 16:34:40.973046 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964937    2582 flags.go:64] FLAG: --vmodule=""
Jul 08 16:34:40.973046 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964943    2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 08 16:34:40.973046 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.964948    2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 08 16:34:40.973046 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965134    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 16:34:40.973046 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965140    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 16:34:40.973046 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965145    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 16:34:40.973046 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965150    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 16:34:40.973046 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965157    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 16:34:40.973046 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965161    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 16:34:40.973046 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965165    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 16:34:40.973046 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965170    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 16:34:40.973046 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965175    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 16:34:40.973046 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965179    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 16:34:40.973046 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965183    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 16:34:40.973046 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965186    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 16:34:40.973046 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965191    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 16:34:40.973676 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965195    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 16:34:40.973676 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965199    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 16:34:40.973676 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965203    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 16:34:40.973676 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965208    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 16:34:40.973676 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965212    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 16:34:40.973676 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965217    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 16:34:40.973676 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965221    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 16:34:40.973676 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965225    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 16:34:40.973676 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965229    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 16:34:40.973676 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965234    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 16:34:40.973676 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965238    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 16:34:40.973676 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965242    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 16:34:40.973676 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965246    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 16:34:40.973676 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965251    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 16:34:40.973676 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965255    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 16:34:40.973676 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965259    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 16:34:40.973676 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965263    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 16:34:40.973676 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965267    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 16:34:40.973676 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965271    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 16:34:40.973676 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965275    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 16:34:40.974218 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965280    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 16:34:40.974218 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965284    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 16:34:40.974218 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965289    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 16:34:40.974218 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965294    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 16:34:40.974218 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965299    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 16:34:40.974218 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965303    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 16:34:40.974218 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965307    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 16:34:40.974218 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965329    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 16:34:40.974218 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965333    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 16:34:40.974218 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965337    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 16:34:40.974218 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965342    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 16:34:40.974218 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965347    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 16:34:40.974218 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965351    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 16:34:40.974218 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965355    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 16:34:40.974218 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965358    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 16:34:40.974218 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965364    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 16:34:40.974218 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965368    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 16:34:40.974218 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965372    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 16:34:40.974218 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965383    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 16:34:40.974740 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965387    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 16:34:40.974740 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965391    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 16:34:40.974740 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965395    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 16:34:40.974740 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965400    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 16:34:40.974740 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965404    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 16:34:40.974740 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965408    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 16:34:40.974740 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965412    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 16:34:40.974740 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965416    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 16:34:40.974740 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965421    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 16:34:40.974740 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965424    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 16:34:40.974740 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965429    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 16:34:40.974740 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965433    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 16:34:40.974740 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965438    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 16:34:40.974740 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965442    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 16:34:40.974740 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965446    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 16:34:40.974740 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965451    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 16:34:40.974740 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965455    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 16:34:40.974740 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965462    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 16:34:40.974740 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965467    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 16:34:40.975202 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965472    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 16:34:40.975202 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965477    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 16:34:40.975202 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965481    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 16:34:40.975202 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965486    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 16:34:40.975202 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965490    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 16:34:40.975202 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965495    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 16:34:40.975202 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965499    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 16:34:40.975202 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965503    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 16:34:40.975202 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965507    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 16:34:40.975202 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965512    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 16:34:40.975202 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965516    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 16:34:40.975202 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965520    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 16:34:40.975202 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965525    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 16:34:40.975202 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965530    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 16:34:40.975202 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965534    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 16:34:40.975202 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965538    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 16:34:40.975202 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965542    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 16:34:40.975202 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965546    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 16:34:40.975202 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965551    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 16:34:40.975202 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965556    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 16:34:40.975703 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965560    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 16:34:40.975703 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965564    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 16:34:40.975703 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965570    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 16:34:40.975703 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965576    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 16:34:40.975703 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965581    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 16:34:40.975703 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965585    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 16:34:40.975703 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965591    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 16:34:40.975703 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.965596    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 16:34:40.975703 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.966429    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 16:34:40.975703 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.975671    2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 08 16:34:40.975703 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.975687    2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 08 16:34:40.975703 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.975701    2582 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 08 16:34:40.975703 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.975707    2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 08 16:34:40.976108 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975772    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 16:34:40.976108 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975777    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 16:34:40.976108 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975781    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 16:34:40.976108 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975784    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 16:34:40.976108 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975787    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 16:34:40.976108 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975790    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 16:34:40.976108 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975793    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 16:34:40.976108 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975796    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 16:34:40.976108 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975798    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 16:34:40.976108 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975801    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 16:34:40.976108 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975804    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 16:34:40.976108 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975806    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 16:34:40.976108 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975814    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 16:34:40.976108 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975817    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 16:34:40.976108 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975820    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 16:34:40.976108 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975822    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 16:34:40.976108 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975828    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 16:34:40.976108 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975831    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 16:34:40.976108 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975833    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 16:34:40.976108 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975836    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 16:34:40.976609 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975838    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 16:34:40.976609 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975842    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 16:34:40.976609 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975846    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 16:34:40.976609 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975849    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 16:34:40.976609 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975852    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 16:34:40.976609 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975855    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 16:34:40.976609 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975858    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 16:34:40.976609 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975860    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 16:34:40.976609 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975863    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 16:34:40.976609 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975865    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 16:34:40.976609 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975868    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 16:34:40.976609 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975870    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 16:34:40.976609 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975873    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 16:34:40.976609 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975876    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 16:34:40.976609 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975879    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 16:34:40.976609 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975881    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 16:34:40.976609 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975883    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 16:34:40.976609 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975886    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 16:34:40.976609 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975888    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 16:34:40.977071 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975891    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 16:34:40.977071 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975894    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 16:34:40.977071 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975896    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 16:34:40.977071 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975898    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 16:34:40.977071 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975901    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 16:34:40.977071 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975904    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 16:34:40.977071 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975912    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 16:34:40.977071 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975914    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 16:34:40.977071 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975917    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 16:34:40.977071 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975919    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 16:34:40.977071 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975921    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 16:34:40.977071 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975924    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 16:34:40.977071 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975926    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 16:34:40.977071 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975929    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 16:34:40.977071 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975931    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 16:34:40.977071 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975934    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 16:34:40.977071 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975936    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 16:34:40.977071 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975939    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 16:34:40.977071 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975943    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 16:34:40.977071 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975947    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 16:34:40.977647 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975949    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 16:34:40.977647 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975952    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 16:34:40.977647 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975954    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 16:34:40.977647 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975957    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 16:34:40.977647 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975960    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 16:34:40.977647 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975962    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 16:34:40.977647 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975966    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 16:34:40.977647 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975969    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 16:34:40.977647 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975972    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 16:34:40.977647 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975974    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 16:34:40.977647 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975977    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 16:34:40.977647 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975980    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 16:34:40.977647 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975982    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 16:34:40.977647 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975984    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 16:34:40.977647 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975987    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 16:34:40.977647 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975989    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 16:34:40.977647 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975992    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 16:34:40.977647 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975996    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 16:34:40.977647 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.975998    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 16:34:40.978136 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976007    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 16:34:40.978136 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976010    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 16:34:40.978136 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976012    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 16:34:40.978136 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976015    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 16:34:40.978136 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976018    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 16:34:40.978136 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976020    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 16:34:40.978136 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976023    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 16:34:40.978136 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976025    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 16:34:40.978136 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976028    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 16:34:40.978136 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976030    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 16:34:40.978136 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976032    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 16:34:40.978136 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976035    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 16:34:40.978136 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976038    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 16:34:40.978136 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976047    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 16:34:40.978136 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976049    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 16:34:40.978136 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976052    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 16:34:40.978136 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976054    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 16:34:40.978136 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976057    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 16:34:40.978136 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976059    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 16:34:40.978136 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976063    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 16:34:40.978645 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976065    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 16:34:40.978645 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.976071    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 16:34:40.978645 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976227    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 16:34:40.978645 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976232    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 16:34:40.978645 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976235    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 16:34:40.978645 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976238    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 16:34:40.978645 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976241    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 16:34:40.978645 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976244    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 16:34:40.978645 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976247    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 16:34:40.978645 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976250    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 16:34:40.978645 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976253    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 16:34:40.978645 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976255    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 16:34:40.978645 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976258    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 16:34:40.978645 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976266    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 16:34:40.978645 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976268    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 16:34:40.979006 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976271    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 16:34:40.979006 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976274    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 16:34:40.979006 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976277    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 16:34:40.979006 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976280    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 16:34:40.979006 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976283    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 16:34:40.979006 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976285    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 16:34:40.979006 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976288    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 16:34:40.979006 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976291    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 16:34:40.979006 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976293    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 16:34:40.979006 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976295    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 16:34:40.979006 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976298    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 16:34:40.979006 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976300    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 16:34:40.979006 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976303    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 16:34:40.979006 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976305    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 16:34:40.979006 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976308    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 16:34:40.979006 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976328    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 16:34:40.979006 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976331    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 16:34:40.979006 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976334    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 16:34:40.979006 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976336    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 16:34:40.979006 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976339    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 16:34:40.979511 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976341    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 16:34:40.979511 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976344    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 16:34:40.979511 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976347    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 16:34:40.979511 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976350    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 16:34:40.979511 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976352    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 16:34:40.979511 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976355    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 16:34:40.979511 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976357    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 16:34:40.979511 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976359    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 16:34:40.979511 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976362    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 16:34:40.979511 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976364    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 16:34:40.979511 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976368    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 16:34:40.979511 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976377    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 16:34:40.979511 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976380    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 16:34:40.979511 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976383    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 16:34:40.979511 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976386    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 16:34:40.979511 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976388    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 16:34:40.979511 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976391    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 16:34:40.979511 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976394    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 16:34:40.979511 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976397    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 16:34:40.979961 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976400    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 16:34:40.979961 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976403    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 16:34:40.979961 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976406    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 16:34:40.979961 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976409    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 16:34:40.979961 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976411    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 16:34:40.979961 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976415    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 16:34:40.979961 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976418    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 16:34:40.979961 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976421    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 16:34:40.979961 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976424    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 16:34:40.979961 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976427    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 16:34:40.979961 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976430    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 16:34:40.979961 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976432    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 16:34:40.979961 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976435    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 16:34:40.979961 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976437    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 16:34:40.979961 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976440    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 16:34:40.979961 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976443    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 16:34:40.979961 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976445    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 16:34:40.979961 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976448    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 16:34:40.979961 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976450    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 16:34:40.980439 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976453    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 16:34:40.980439 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976455    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 16:34:40.980439 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976457    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 16:34:40.980439 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976460    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 16:34:40.980439 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976463    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 16:34:40.980439 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976465    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 16:34:40.980439 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976473    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 16:34:40.980439 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976475    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 16:34:40.980439 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976478    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 16:34:40.980439 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976481    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 16:34:40.980439 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976483    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 16:34:40.980439 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976485    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 16:34:40.980439 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976488    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 16:34:40.980439 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976491    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 16:34:40.980439 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976493    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 16:34:40.980439 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976496    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 16:34:40.980439 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976498    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 16:34:40.980439 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976500    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 16:34:40.980439 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976503    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 16:34:40.980439 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976506    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 16:34:40.980918 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976508    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 16:34:40.980918 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976511    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 16:34:40.980918 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976513    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 16:34:40.980918 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976516    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 16:34:40.980918 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976519    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 16:34:40.980918 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976521    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 16:34:40.980918 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976524    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 16:34:40.980918 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:40.976526    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 16:34:40.980918 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.976531    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 16:34:40.980918 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.976662    2582 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 08 16:34:40.980918 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.978792    2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 08 16:34:40.980918 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.980468    2582 server.go:1019] "Starting client certificate rotation"
Jul 08 16:34:40.980918 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.980568    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 08 16:34:40.980918 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.980623    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 08 16:34:40.985304 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.985276    2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 08 16:34:40.986581 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.986564    2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 08 16:34:40.997297 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:40.997279    2582 log.go:25] "Validated CRI v1 runtime API"
Jul 08 16:34:41.001381 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.001359    2582 log.go:25] "Validated CRI v1 image API"
Jul 08 16:34:41.002117 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.002104    2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 08 16:34:41.004732 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.004711    2582 fs.go:135] Filesystem UUIDs: map[07897908-c968-4ecb-95ae-cb80c70cb1d4:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 d2b3402c-b012-465b-b7e8-d329bb2d2498:/dev/nvme0n1p3]
Jul 08 16:34:41.004794 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.004730    2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 08 16:34:41.011826 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.011709    2582 manager.go:216] Machine: {Timestamp:2026-07-08 16:34:41.008629027 +0000 UTC m=+0.349594096 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096064 MemoryCapacity:32812167168 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25aafe6b99d691d183dee3202a24af SystemUUID:ec25aafe-6b99-d691-d183-dee3202a24af BootID:093b0816-3f58-40f7-b27d-5b11528d2dc9 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:26:47:31:e3:0d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:26:47:31:e3:0d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ea:a7:ad:cc:4b:a8 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812167168 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 08 16:34:41.011826 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.011824    2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 08 16:34:41.011936 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.011911    2582 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 08 16:34:41.012085 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.012068    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 08 16:34:41.013206 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.013167    2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 08 16:34:41.013370 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.013207    2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-237.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 08 16:34:41.014539 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.014526    2582 topology_manager.go:138] "Creating topology manager with none policy"
Jul 08 16:34:41.014584 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.014546    2582 container_manager_linux.go:309] "Creating device plugin manager"
Jul 08 16:34:41.014584 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.014554    2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 08 16:34:41.015086 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.015076    2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 08 16:34:41.016352 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.016343    2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 08 16:34:41.019196 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.019184    2582 state_mem.go:36] "Initialized new in-memory state store"
Jul 08 16:34:41.019351 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.019340    2582 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 08 16:34:41.019403 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.019377    2582 kubelet.go:486] "Attempting to sync node with API server"
Jul 08 16:34:41.019403 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.019391    2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 08 16:34:41.019480 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.019404    2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 08 16:34:41.019480 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.019413    2582 kubelet.go:398] "Adding apiserver pod source"
Jul 08 16:34:41.019480 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.019425    2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 08 16:34:41.020194 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.020163    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 08 16:34:41.020194 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.020193    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 08 16:34:41.023781 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.023762    2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 08 16:34:41.025559 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.025546    2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 08 16:34:41.025602 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.025577    2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 08 16:34:41.026616 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.026606    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 08 16:34:41.026664 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.026622    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 08 16:34:41.026664 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.026628    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 08 16:34:41.026664 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.026634    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 08 16:34:41.026664 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.026643    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 08 16:34:41.026664 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.026649    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 08 16:34:41.026664 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.026661    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 08 16:34:41.026825 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.026669    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 08 16:34:41.026825 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.026676    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 08 16:34:41.026825 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.026682    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 08 16:34:41.026825 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.026690    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 08 16:34:41.026825 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.026695    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 08 16:34:41.027460 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.027450    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 08 16:34:41.027497 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.027462    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 08 16:34:41.029286 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.029266    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 08 16:34:41.029820 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:41.029794    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 08 16:34:41.029869 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:41.029794    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 08 16:34:41.030799 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.030787    2582 server.go:1268] "Started kubelet"
Jul 08 16:34:41.030960 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.030901    2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 08 16:34:41.031059 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.030984    2582 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 08 16:34:41.031059 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.030997    2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 08 16:34:41.031695 ip-10-0-139-237 systemd[1]: Started Kubernetes Kubelet.
Jul 08 16:34:41.032131 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.032115    2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 08 16:34:41.032592 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.032553    2582 server.go:310] "Adding debug handlers to kubelet server"
Jul 08 16:34:41.034358 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.034308    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nznjk"
Jul 08 16:34:41.037075 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:41.036247    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-237.ec2.internal.18c05db436a16b80  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-237.ec2.internal,UID:ip-10-0-139-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-237.ec2.internal,},FirstTimestamp:2026-07-08 16:34:41.030761344 +0000 UTC m=+0.371726414,LastTimestamp:2026-07-08 16:34:41.030761344 +0000 UTC m=+0.371726414,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-237.ec2.internal,}"
Jul 08 16:34:41.037759 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.037747    2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 08 16:34:41.037821 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.037771    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 08 16:34:41.037907 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:41.037890    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-237.ec2.internal\" not found"
Jul 08 16:34:41.038558 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.038533    2582 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 08 16:34:41.038558 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.038559    2582 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 08 16:34:41.038689 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.038533    2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 08 16:34:41.038737 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.038728    2582 reconstruct.go:97] "Volume reconstruction finished"
Jul 08 16:34:41.038737 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.038737    2582 reconciler.go:29] "Reconciler: start to sync state"
Jul 08 16:34:41.038828 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.038757    2582 factory.go:55] Registering systemd factory
Jul 08 16:34:41.038828 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.038771    2582 factory.go:223] Registration of the systemd container factory successfully
Jul 08 16:34:41.039015 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.038999    2582 factory.go:153] Registering CRI-O factory
Jul 08 16:34:41.039068 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.039019    2582 factory.go:223] Registration of the crio container factory successfully
Jul 08 16:34:41.039106 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.039070    2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 08 16:34:41.039106 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.039089    2582 factory.go:103] Registering Raw factory
Jul 08 16:34:41.039106 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.039102    2582 manager.go:1196] Started watching for new ooms in manager
Jul 08 16:34:41.039787 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.039757    2582 manager.go:349] Starting recovery of all containers
Jul 08 16:34:41.039787 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:41.039779    2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 08 16:34:41.040388 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.040355    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nznjk"
Jul 08 16:34:41.050132 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.050102    2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 16:34:41.053723 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:41.053536    2582 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-237.ec2.internal\" not found" node="ip-10-0-139-237.ec2.internal"
Jul 08 16:34:41.054237 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.054220    2582 manager.go:354] Recovery completed
Jul 08 16:34:41.058714 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.058701    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 16:34:41.060908 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.060893    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 16:34:41.060978 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.060922    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 16:34:41.060978 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.060933    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-237.ec2.internal" event="NodeHasSufficientPID"
Jul 08 16:34:41.061538 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.061524    2582 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 08 16:34:41.061538 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.061533    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 16:34:41.061667 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.061535    2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 08 16:34:41.061667 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.061610    2582 state_mem.go:36] "Initialized new in-memory state store"
Jul 08 16:34:41.062202 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.062187    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 16:34:41.062250 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.062217    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 16:34:41.062250 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.062228    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-237.ec2.internal" event="NodeHasSufficientPID"
Jul 08 16:34:41.062677 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.062665    2582 policy_none.go:49] "None policy: Start"
Jul 08 16:34:41.062722 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.062682    2582 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 08 16:34:41.062722 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.062692    2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 08 16:34:41.063836 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.063825    2582 policy_none.go:47] "Start"
Jul 08 16:34:41.096895 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.096873    2582 manager.go:337] "Starting Device Plugin manager"
Jul 08 16:34:41.096999 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:41.096914    2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 08 16:34:41.096999 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.096927    2582 server.go:87] "Starting device plugin registration server"
Jul 08 16:34:41.097252 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.097241    2582 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 08 16:34:41.097288 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.097255    2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 08 16:34:41.097441 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.097425    2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 08 16:34:41.097549 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.097538    2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 08 16:34:41.097602 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.097550    2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 08 16:34:41.098053 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:41.098034    2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 08 16:34:41.098177 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:41.098079    2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-237.ec2.internal\" not found"
Jul 08 16:34:41.197843 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.197824    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 16:34:41.198805 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.198785    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 16:34:41.198927 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.198823    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 16:34:41.198927 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.198838    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-237.ec2.internal" event="NodeHasSufficientPID"
Jul 08 16:34:41.198927 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.198873    2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-237.ec2.internal"
Jul 08 16:34:41.209564 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.209504    2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-237.ec2.internal"
Jul 08 16:34:41.209564 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:41.209529    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-237.ec2.internal\": node \"ip-10-0-139-237.ec2.internal\" not found"
Jul 08 16:34:41.209743 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.209638    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 08 16:34:41.211004 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.210990    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 08 16:34:41.211078 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.211024    2582 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 08 16:34:41.211078 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.211043    2582 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 08 16:34:41.211078 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:41.211074    2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 08 16:34:41.213418 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.213392    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 16:34:41.261947 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:41.261915    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-237.ec2.internal\" not found"
Jul 08 16:34:41.311629 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.311575    2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-237.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-237.ec2.internal"]
Jul 08 16:34:41.311774 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.311672    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 16:34:41.312655 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.312637    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 16:34:41.312736 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.312671    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 16:34:41.312736 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.312685    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-237.ec2.internal" event="NodeHasSufficientPID"
Jul 08 16:34:41.313929 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.313916    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 16:34:41.314054 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.314038    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-237.ec2.internal"
Jul 08 16:34:41.314108 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.314070    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 16:34:41.314663 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.314639    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 16:34:41.314777 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.314672    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 16:34:41.314777 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.314685    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-237.ec2.internal" event="NodeHasSufficientPID"
Jul 08 16:34:41.314777 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.314725    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 16:34:41.314777 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.314760    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 16:34:41.314777 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.314771    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-237.ec2.internal" event="NodeHasSufficientPID"
Jul 08 16:34:41.315816 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.315799    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-237.ec2.internal"
Jul 08 16:34:41.315906 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.315829    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 16:34:41.316469 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.316455    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-237.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 16:34:41.316545 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.316478    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-237.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 16:34:41.316545 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.316487    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-237.ec2.internal" event="NodeHasSufficientPID"
Jul 08 16:34:41.337838 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:41.337818    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-237.ec2.internal\" not found" node="ip-10-0-139-237.ec2.internal"
Jul 08 16:34:41.339956 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.339938    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9d4667db5036e322bfdaf5880d0290f1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-237.ec2.internal\" (UID: \"9d4667db5036e322bfdaf5880d0290f1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-237.ec2.internal"
Jul 08 16:34:41.340059 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.339963    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d4667db5036e322bfdaf5880d0290f1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-237.ec2.internal\" (UID: \"9d4667db5036e322bfdaf5880d0290f1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-237.ec2.internal"
Jul 08 16:34:41.340059 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.339981    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/81aa7add2915dbb7aae72d3a67b06d72-config\") pod \"kube-apiserver-proxy-ip-10-0-139-237.ec2.internal\" (UID: \"81aa7add2915dbb7aae72d3a67b06d72\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-237.ec2.internal"
Jul 08 16:34:41.341771 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:41.341756    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-237.ec2.internal\" not found" node="ip-10-0-139-237.ec2.internal"
Jul 08 16:34:41.362541 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:41.362519    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-237.ec2.internal\" not found"
Jul 08 16:34:41.440115 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.440078    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d4667db5036e322bfdaf5880d0290f1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-237.ec2.internal\" (UID: \"9d4667db5036e322bfdaf5880d0290f1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-237.ec2.internal"
Jul 08 16:34:41.440115 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.440114    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/81aa7add2915dbb7aae72d3a67b06d72-config\") pod \"kube-apiserver-proxy-ip-10-0-139-237.ec2.internal\" (UID: \"81aa7add2915dbb7aae72d3a67b06d72\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-237.ec2.internal"
Jul 08 16:34:41.440332 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.440131    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9d4667db5036e322bfdaf5880d0290f1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-237.ec2.internal\" (UID: \"9d4667db5036e322bfdaf5880d0290f1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-237.ec2.internal"
Jul 08 16:34:41.440332 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.440168    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/81aa7add2915dbb7aae72d3a67b06d72-config\") pod \"kube-apiserver-proxy-ip-10-0-139-237.ec2.internal\" (UID: \"81aa7add2915dbb7aae72d3a67b06d72\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-237.ec2.internal"
Jul 08 16:34:41.440332 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.440177    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d4667db5036e322bfdaf5880d0290f1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-237.ec2.internal\" (UID: \"9d4667db5036e322bfdaf5880d0290f1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-237.ec2.internal"
Jul 08 16:34:41.440332 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.440213    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9d4667db5036e322bfdaf5880d0290f1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-237.ec2.internal\" (UID: \"9d4667db5036e322bfdaf5880d0290f1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-237.ec2.internal"
Jul 08 16:34:41.463262 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:41.463200    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-237.ec2.internal\" not found"
Jul 08 16:34:41.563914 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:41.563882    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-237.ec2.internal\" not found"
Jul 08 16:34:41.640201 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.640182    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-237.ec2.internal"
Jul 08 16:34:41.644790 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.644590    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-237.ec2.internal"
Jul 08 16:34:41.664892 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:41.664867    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-237.ec2.internal\" not found"
Jul 08 16:34:41.765467 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:41.765374    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-237.ec2.internal\" not found"
Jul 08 16:34:41.865886 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:41.865859    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-237.ec2.internal\" not found"
Jul 08 16:34:41.966372 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:41.966347    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-237.ec2.internal\" not found"
Jul 08 16:34:41.979533 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.979506    2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 08 16:34:41.979680 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.979664    2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 08 16:34:41.979734 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.979689    2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 08 16:34:41.992466 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:41.992444    2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 16:34:42.006942 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.006915    2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 16:34:42.020120 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.020060    2582 apiserver.go:52] "Watching apiserver"
Jul 08 16:34:42.027425 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.027405    2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 08 16:34:42.027735 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.027714    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5pm","openshift-cluster-node-tuning-operator/tuned-p46xh","openshift-image-registry/node-ca-bpx78","openshift-multus/multus-jgp2d","openshift-multus/network-metrics-daemon-z82nm","openshift-network-diagnostics/network-check-target-bkqjt","kube-system/global-pull-secret-syncer-4vtkl","kube-system/konnectivity-agent-957gg","openshift-dns/node-resolver-pq4sj","openshift-multus/multus-additional-cni-plugins-mw6h7","openshift-network-operator/iptables-alerter-8hh26","openshift-ovn-kubernetes/ovnkube-node-kvhpr"]
Jul 08 16:34:42.029198 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.029185    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5pm"
Jul 08 16:34:42.030194 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.030174    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.032870 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.032849    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bpx78"
Jul 08 16:34:42.033616 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.033598    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 08 16:34:42.033705 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.033686    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 08 16:34:42.033705 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.033689    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 08 16:34:42.033782 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.033723    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pld78\""
Jul 08 16:34:42.034089 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.034077    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.035110 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.035095    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 08 16:34:42.035177 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.035114    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 08 16:34:42.035177 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.035142    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-s67jr\""
Jul 08 16:34:42.035268 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.035254    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:34:42.035375 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:42.035354    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z82nm" podUID="2796ef11-ffb4-4bff-b73b-38d727eb45f2"
Jul 08 16:34:42.036391 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.036375    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:34:42.036457 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:42.036439    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkqjt" podUID="c98cc144-3322-4f83-b571-9772b5a63d49"
Jul 08 16:34:42.036683 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.036595    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 08 16:34:42.036683 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.036639    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 08 16:34:42.037067 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.037054    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-r2k9g\""
Jul 08 16:34:42.037533 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.037515    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 08 16:34:42.037788 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.037655    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 08 16:34:42.037788 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.037708    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 08 16:34:42.037788 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.037711    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 08 16:34:42.037788 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.037734    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-2xsnp\""
Jul 08 16:34:42.038030 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.037825    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 08 16:34:42.038030 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.037935    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-237.ec2.internal"
Jul 08 16:34:42.038030 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.038003    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 08 16:34:42.038141 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.037938    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:34:42.038141 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:42.038108    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtkl" podUID="8eb2abc0-b610-4f3b-b6dc-02131ad29e35"
Jul 08 16:34:42.039051 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.039035    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-957gg"
Jul 08 16:34:42.040781 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.040766    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pq4sj"
Jul 08 16:34:42.041561 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.041545    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 08 16:34:42.041693 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.041675    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 08 16:34:42.041885 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.041872    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mw6h7"
Jul 08 16:34:42.042451 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.042423    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5zqjt\""
Jul 08 16:34:42.044179 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.043818    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f877b6b1-282c-4c50-95d6-530371d41a15-konnectivity-ca\") pod \"konnectivity-agent-957gg\" (UID: \"f877b6b1-282c-4c50-95d6-530371d41a15\") " pod="kube-system/konnectivity-agent-957gg"
Jul 08 16:34:42.044179 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.043846    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmxvw\" (UniqueName: \"kubernetes.io/projected/d45b9a7f-3c63-4f52-863f-5d128fd07d93-kube-api-access-nmxvw\") pod \"node-ca-bpx78\" (UID: \"d45b9a7f-3c63-4f52-863f-5d128fd07d93\") " pod="openshift-image-registry/node-ca-bpx78"
Jul 08 16:34:42.044179 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.043865    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2796ef11-ffb4-4bff-b73b-38d727eb45f2-metrics-certs\") pod \"network-metrics-daemon-z82nm\" (UID: \"2796ef11-ffb4-4bff-b73b-38d727eb45f2\") " pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:34:42.044179 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.043889    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-kubelet-config\") pod \"global-pull-secret-syncer-4vtkl\" (UID: \"8eb2abc0-b610-4f3b-b6dc-02131ad29e35\") " pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:34:42.044179 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.043923    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-dbus\") pod \"global-pull-secret-syncer-4vtkl\" (UID: \"8eb2abc0-b610-4f3b-b6dc-02131ad29e35\") " pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:34:42.044179 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.043995    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-host-var-lib-cni-multus\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.044179 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.044047    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-hostroot\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.044179 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.044083    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-host-run-multus-certs\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.044179 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.044122    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkwp8\" (UniqueName: \"kubernetes.io/projected/6c79a3b7-b6b3-4264-8c73-36fc32701b6e-kube-api-access-dkwp8\") pod \"aws-ebs-csi-driver-node-ns5pm\" (UID: \"6c79a3b7-b6b3-4264-8c73-36fc32701b6e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5pm"
Jul 08 16:34:42.044654 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.044201    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-original-pull-secret\") pod \"global-pull-secret-syncer-4vtkl\" (UID: \"8eb2abc0-b610-4f3b-b6dc-02131ad29e35\") " pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:34:42.044654 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.044232    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-system-cni-dir\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.044654 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.044259    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-host-run-k8s-cni-cncf-io\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.044654 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.044293    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6c79a3b7-b6b3-4264-8c73-36fc32701b6e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ns5pm\" (UID: \"6c79a3b7-b6b3-4264-8c73-36fc32701b6e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5pm"
Jul 08 16:34:42.044654 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.044340    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6c79a3b7-b6b3-4264-8c73-36fc32701b6e-socket-dir\") pod \"aws-ebs-csi-driver-node-ns5pm\" (UID: \"6c79a3b7-b6b3-4264-8c73-36fc32701b6e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5pm"
Jul 08 16:34:42.044654 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.044359    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6c79a3b7-b6b3-4264-8c73-36fc32701b6e-sys-fs\") pod \"aws-ebs-csi-driver-node-ns5pm\" (UID: \"6c79a3b7-b6b3-4264-8c73-36fc32701b6e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5pm"
Jul 08 16:34:42.044654 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.044377    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9txx\" (UniqueName: \"kubernetes.io/projected/2796ef11-ffb4-4bff-b73b-38d727eb45f2-kube-api-access-k9txx\") pod \"network-metrics-daemon-z82nm\" (UID: \"2796ef11-ffb4-4bff-b73b-38d727eb45f2\") " pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:34:42.044654 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.044459    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6c79a3b7-b6b3-4264-8c73-36fc32701b6e-registration-dir\") pod \"aws-ebs-csi-driver-node-ns5pm\" (UID: \"6c79a3b7-b6b3-4264-8c73-36fc32701b6e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5pm"
Jul 08 16:34:42.044654 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.044548    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-etc-modprobe-d\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.044654 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.044585    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-var-lib-kubelet\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.044654 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.044641    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-host\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.045136 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.044693    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d45b9a7f-3c63-4f52-863f-5d128fd07d93-host\") pod \"node-ca-bpx78\" (UID: \"d45b9a7f-3c63-4f52-863f-5d128fd07d93\") " pod="openshift-image-registry/node-ca-bpx78"
Jul 08 16:34:42.045136 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.044733    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-os-release\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.045136 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.044790    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8hh26"
Jul 08 16:34:42.045136 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.044790    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-multus-socket-dir-parent\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.045136 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.044824    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-host-var-lib-kubelet\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.045136 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.044802    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 16:29:41 +0000 UTC" deadline="2027-12-03 00:30:52.538815732 +0000 UTC"
Jul 08 16:34:42.045136 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.044843    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12295h56m10.493976279s"
Jul 08 16:34:42.045136 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.044875    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6c79a3b7-b6b3-4264-8c73-36fc32701b6e-device-dir\") pod \"aws-ebs-csi-driver-node-ns5pm\" (UID: \"6c79a3b7-b6b3-4264-8c73-36fc32701b6e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5pm"
Jul 08 16:34:42.045136 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.044909    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-etc-sysconfig\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.045136 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.044987    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-etc-sysctl-d\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.045136 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.045026    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-etc-sysctl-conf\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.045742 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.045088    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-run\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.045803 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.045743    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wlvf8\""
Jul 08 16:34:42.045854 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.045796    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-multus-cni-dir\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.045854 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.045840    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f877b6b1-282c-4c50-95d6-530371d41a15-agent-certs\") pod \"konnectivity-agent-957gg\" (UID: \"f877b6b1-282c-4c50-95d6-530371d41a15\") " pod="kube-system/konnectivity-agent-957gg"
Jul 08 16:34:42.045939 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.045873    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-etc-systemd\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.045939 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.045923    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-sys\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.046029 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.045963    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/782cdc3e-e8c4-4d37-a22e-832fac199082-etc-tuned\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.046080 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.046051    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/782cdc3e-e8c4-4d37-a22e-832fac199082-tmp\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.046133 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.046078    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p2pz\" (UniqueName: \"kubernetes.io/projected/782cdc3e-e8c4-4d37-a22e-832fac199082-kube-api-access-8p2pz\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.046183 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.046140    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 08 16:34:42.046234 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.046190    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-host-var-lib-cni-bin\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.046282 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.046233    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-etc-kubernetes\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.046351 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.046284    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-lib-modules\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.046423 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.046359    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d45b9a7f-3c63-4f52-863f-5d128fd07d93-serviceca\") pod \"node-ca-bpx78\" (UID: \"d45b9a7f-3c63-4f52-863f-5d128fd07d93\") " pod="openshift-image-registry/node-ca-bpx78"
Jul 08 16:34:42.046423 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.046404    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.046563 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.046441    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 08 16:34:42.046563 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.046479    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqv9w\" (UniqueName: \"kubernetes.io/projected/c98cc144-3322-4f83-b571-9772b5a63d49-kube-api-access-kqv9w\") pod \"network-check-target-bkqjt\" (UID: \"c98cc144-3322-4f83-b571-9772b5a63d49\") " pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:34:42.046853 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.046832    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-host-run-netns\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.046921 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.046880    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-multus-conf-dir\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.046921 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.046908    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a2b65945-693a-42f8-9955-f484a3f96d10-multus-daemon-config\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.047012 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.046929    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-etc-kubernetes\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.047012 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.046953    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s7x8\" (UniqueName: \"kubernetes.io/projected/a2b65945-693a-42f8-9955-f484a3f96d10-kube-api-access-2s7x8\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.047012 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.046979    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6c79a3b7-b6b3-4264-8c73-36fc32701b6e-etc-selinux\") pod \"aws-ebs-csi-driver-node-ns5pm\" (UID: \"6c79a3b7-b6b3-4264-8c73-36fc32701b6e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5pm"
Jul 08 16:34:42.047139 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.047012    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-cnibin\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.047139 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.047043    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a2b65945-693a-42f8-9955-f484a3f96d10-cni-binary-copy\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.048161 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.048146    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bdg5t\""
Jul 08 16:34:42.048910 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.048893    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 08 16:34:42.048953 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.048920    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-g6dzc\""
Jul 08 16:34:42.049209 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.049192    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 08 16:34:42.049255 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.049208    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 08 16:34:42.049255 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.049208    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 08 16:34:42.049430 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.049412    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 08 16:34:42.049542 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.049455    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 08 16:34:42.049542 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.049522    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 08 16:34:42.049680 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.049666    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-q7cbp\""
Jul 08 16:34:42.049733 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.049686    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 08 16:34:42.049783 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.049730    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 08 16:34:42.049783 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.049762    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 08 16:34:42.050254 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.050240    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 08 16:34:42.054481 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.054464    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 08 16:34:42.054615 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.054595    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-237.ec2.internal"
Jul 08 16:34:42.055986 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.055972    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-237.ec2.internal"]
Jul 08 16:34:42.056665 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.056649    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 08 16:34:42.063204 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.063188    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 08 16:34:42.063283 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.063264    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-139-237.ec2.internal"]
Jul 08 16:34:42.087233 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.087210    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-65kw7"
Jul 08 16:34:42.095093 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.095076    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-65kw7"
Jul 08 16:34:42.139336 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.139297    2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 08 16:34:42.147837 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.147814    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-os-release\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.147946 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.147846    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-multus-socket-dir-parent\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.147946 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.147865    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-host-var-lib-kubelet\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.147946 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.147912    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-host-var-lib-kubelet\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.147946 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.147924    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6c79a3b7-b6b3-4264-8c73-36fc32701b6e-device-dir\") pod \"aws-ebs-csi-driver-node-ns5pm\" (UID: \"6c79a3b7-b6b3-4264-8c73-36fc32701b6e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5pm"
Jul 08 16:34:42.147946 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.147936    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-os-release\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.147946 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.147938    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-multus-socket-dir-parent\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.148112 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.147974    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-etc-sysconfig\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.148112 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.147988    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6c79a3b7-b6b3-4264-8c73-36fc32701b6e-device-dir\") pod \"aws-ebs-csi-driver-node-ns5pm\" (UID: \"6c79a3b7-b6b3-4264-8c73-36fc32701b6e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5pm"
Jul 08 16:34:42.148112 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148000    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-etc-sysctl-d\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.148112 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148030    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-run\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.148112 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148047    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a12a9aeb-4a05-41b1-9f81-dc6dc476fda8-host-slash\") pod \"iptables-alerter-8hh26\" (UID: \"a12a9aeb-4a05-41b1-9f81-dc6dc476fda8\") " pod="openshift-network-operator/iptables-alerter-8hh26"
Jul 08 16:34:42.148112 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148069    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-multus-cni-dir\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.148112 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148077    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-run\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.148112 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148086    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f877b6b1-282c-4c50-95d6-530371d41a15-agent-certs\") pod \"konnectivity-agent-957gg\" (UID: \"f877b6b1-282c-4c50-95d6-530371d41a15\") " pod="kube-system/konnectivity-agent-957gg"
Jul 08 16:34:42.148437 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148101    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-host-var-lib-cni-bin\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.148437 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148047    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-etc-sysconfig\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.148437 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148141    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-etc-kubernetes\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.148437 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148133    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-etc-sysctl-d\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.148437 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148173    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-multus-cni-dir\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.148437 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148193    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-host-var-lib-cni-bin\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.148437 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148242    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-etc-kubernetes\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.148437 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148264    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.148437 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148287    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/19df7751-4b62-4ae5-92a6-0bbda11fec85-ovn-node-metrics-cert\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.148437 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148340    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqv9w\" (UniqueName: \"kubernetes.io/projected/c98cc144-3322-4f83-b571-9772b5a63d49-kube-api-access-kqv9w\") pod \"network-check-target-bkqjt\" (UID: \"c98cc144-3322-4f83-b571-9772b5a63d49\") " pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:34:42.148437 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148356    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a2b65945-693a-42f8-9955-f484a3f96d10-multus-daemon-config\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.148437 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148370    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6c79a3b7-b6b3-4264-8c73-36fc32701b6e-etc-selinux\") pod \"aws-ebs-csi-driver-node-ns5pm\" (UID: \"6c79a3b7-b6b3-4264-8c73-36fc32701b6e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5pm"
Jul 08 16:34:42.148437 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148400    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2796ef11-ffb4-4bff-b73b-38d727eb45f2-metrics-certs\") pod \"network-metrics-daemon-z82nm\" (UID: \"2796ef11-ffb4-4bff-b73b-38d727eb45f2\") " pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:34:42.148437 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148433    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-host-kubelet\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.148437 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148423    2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 08 16:34:42.149139 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148465    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-host-slash\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.149139 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:42.148474    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 16:34:42.149139 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148502    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-run-openvswitch\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.149139 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148507    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6c79a3b7-b6b3-4264-8c73-36fc32701b6e-etc-selinux\") pod \"aws-ebs-csi-driver-node-ns5pm\" (UID: \"6c79a3b7-b6b3-4264-8c73-36fc32701b6e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5pm"
Jul 08 16:34:42.149139 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148529    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f877b6b1-282c-4c50-95d6-530371d41a15-konnectivity-ca\") pod \"konnectivity-agent-957gg\" (UID: \"f877b6b1-282c-4c50-95d6-530371d41a15\") " pod="kube-system/konnectivity-agent-957gg"
Jul 08 16:34:42.149139 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148559    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-run-ovn\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.149139 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:42.148631    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2796ef11-ffb4-4bff-b73b-38d727eb45f2-metrics-certs podName:2796ef11-ffb4-4bff-b73b-38d727eb45f2 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:42.648590909 +0000 UTC m=+1.989555988 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2796ef11-ffb4-4bff-b73b-38d727eb45f2-metrics-certs") pod "network-metrics-daemon-z82nm" (UID: "2796ef11-ffb4-4bff-b73b-38d727eb45f2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 16:34:42.149139 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148664    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ecc650f2-27aa-4300-b61a-8328a41d599d-hosts-file\") pod \"node-resolver-pq4sj\" (UID: \"ecc650f2-27aa-4300-b61a-8328a41d599d\") " pod="openshift-dns/node-resolver-pq4sj"
Jul 08 16:34:42.149139 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148693    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a12a9aeb-4a05-41b1-9f81-dc6dc476fda8-iptables-alerter-script\") pod \"iptables-alerter-8hh26\" (UID: \"a12a9aeb-4a05-41b1-9f81-dc6dc476fda8\") " pod="openshift-network-operator/iptables-alerter-8hh26"
Jul 08 16:34:42.149139 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148718    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-dbus\") pod \"global-pull-secret-syncer-4vtkl\" (UID: \"8eb2abc0-b610-4f3b-b6dc-02131ad29e35\") " pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:34:42.149139 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148733    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-host-var-lib-cni-multus\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.149139 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148753    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-host-run-multus-certs\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.149139 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148776    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-host-run-ovn-kubernetes\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.149139 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148800    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-dbus\") pod \"global-pull-secret-syncer-4vtkl\" (UID: \"8eb2abc0-b610-4f3b-b6dc-02131ad29e35\") " pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:34:42.149139 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148809    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-host-var-lib-cni-multus\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.149139 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148802    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/19df7751-4b62-4ae5-92a6-0bbda11fec85-env-overrides\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.149139 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148844    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-system-cni-dir\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.149996 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148850    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-host-run-multus-certs\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.149996 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148860    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-host-run-k8s-cni-cncf-io\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.149996 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148892    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6c79a3b7-b6b3-4264-8c73-36fc32701b6e-socket-dir\") pod \"aws-ebs-csi-driver-node-ns5pm\" (UID: \"6c79a3b7-b6b3-4264-8c73-36fc32701b6e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5pm"
Jul 08 16:34:42.149996 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148913    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-host-run-k8s-cni-cncf-io\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.149996 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148923    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9txx\" (UniqueName: \"kubernetes.io/projected/2796ef11-ffb4-4bff-b73b-38d727eb45f2-kube-api-access-k9txx\") pod \"network-metrics-daemon-z82nm\" (UID: \"2796ef11-ffb4-4bff-b73b-38d727eb45f2\") " pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:34:42.149996 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148951    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-etc-modprobe-d\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.149996 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148967    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a2b65945-693a-42f8-9955-f484a3f96d10-multus-daemon-config\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.149996 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148997    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6c79a3b7-b6b3-4264-8c73-36fc32701b6e-socket-dir\") pod \"aws-ebs-csi-driver-node-ns5pm\" (UID: \"6c79a3b7-b6b3-4264-8c73-36fc32701b6e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5pm"
Jul 08 16:34:42.149996 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.149014    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-host\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.149996 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.148974    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-host\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.149996 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.149052    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d45b9a7f-3c63-4f52-863f-5d128fd07d93-host\") pod \"node-ca-bpx78\" (UID: \"d45b9a7f-3c63-4f52-863f-5d128fd07d93\") " pod="openshift-image-registry/node-ca-bpx78"
Jul 08 16:34:42.149996 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.149073    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f877b6b1-282c-4c50-95d6-530371d41a15-konnectivity-ca\") pod \"konnectivity-agent-957gg\" (UID: \"f877b6b1-282c-4c50-95d6-530371d41a15\") " pod="kube-system/konnectivity-agent-957gg"
Jul 08 16:34:42.149996 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.149077    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-etc-modprobe-d\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.149996 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.149096    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d45b9a7f-3c63-4f52-863f-5d128fd07d93-host\") pod \"node-ca-bpx78\" (UID: \"d45b9a7f-3c63-4f52-863f-5d128fd07d93\") " pod="openshift-image-registry/node-ca-bpx78"
Jul 08 16:34:42.149996 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.149082    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-var-lib-openvswitch\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.149996 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.149122    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d9ddf360-c5ae-4ef7-a947-59405bf4e8db-cnibin\") pod \"multus-additional-cni-plugins-mw6h7\" (UID: \"d9ddf360-c5ae-4ef7-a947-59405bf4e8db\") " pod="openshift-multus/multus-additional-cni-plugins-mw6h7"
Jul 08 16:34:42.149996 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.149149    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vts62\" (UniqueName: \"kubernetes.io/projected/a12a9aeb-4a05-41b1-9f81-dc6dc476fda8-kube-api-access-vts62\") pod \"iptables-alerter-8hh26\" (UID: \"a12a9aeb-4a05-41b1-9f81-dc6dc476fda8\") " pod="openshift-network-operator/iptables-alerter-8hh26"
Jul 08 16:34:42.151507 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.149150    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-system-cni-dir\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.151507 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.149201    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-systemd-units\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.151507 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.149229    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-etc-sysctl-conf\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.151507 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.149255    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/782cdc3e-e8c4-4d37-a22e-832fac199082-etc-tuned\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.151507 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.149278    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/782cdc3e-e8c4-4d37-a22e-832fac199082-tmp\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.151507 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.149422    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-etc-sysctl-conf\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.151507 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.149754    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-node-log\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.151507 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.149786    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-host-cni-netd\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.151507 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.149813    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-etc-systemd\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.151507 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.149836    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-sys\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.151507 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.149859    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p2pz\" (UniqueName: \"kubernetes.io/projected/782cdc3e-e8c4-4d37-a22e-832fac199082-kube-api-access-8p2pz\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.151507 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.149882    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/19df7751-4b62-4ae5-92a6-0bbda11fec85-ovnkube-config\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.151507 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.149906    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-lib-modules\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.151507 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.149926    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-sys\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.151507 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.149973    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-etc-systemd\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.151507 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.150007    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d45b9a7f-3c63-4f52-863f-5d128fd07d93-serviceca\") pod \"node-ca-bpx78\" (UID: \"d45b9a7f-3c63-4f52-863f-5d128fd07d93\") " pod="openshift-image-registry/node-ca-bpx78"
Jul 08 16:34:42.151507 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.150034    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-run-systemd\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.151507 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.150051    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-lib-modules\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.152241 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.150059    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d9ddf360-c5ae-4ef7-a947-59405bf4e8db-os-release\") pod \"multus-additional-cni-plugins-mw6h7\" (UID: \"d9ddf360-c5ae-4ef7-a947-59405bf4e8db\") " pod="openshift-multus/multus-additional-cni-plugins-mw6h7"
Jul 08 16:34:42.152241 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.150088    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-host-run-netns\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.152241 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.150111    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-multus-conf-dir\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.152241 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.150132    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-etc-kubernetes\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.152241 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.150169    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s7x8\" (UniqueName: \"kubernetes.io/projected/a2b65945-693a-42f8-9955-f484a3f96d10-kube-api-access-2s7x8\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.152241 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.150193    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-etc-openvswitch\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.152241 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.150217    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/19df7751-4b62-4ae5-92a6-0bbda11fec85-ovnkube-script-lib\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.152241 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.150230    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-host-run-netns\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.152241 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.150239    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-cnibin\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.152241 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.150273    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a2b65945-693a-42f8-9955-f484a3f96d10-cni-binary-copy\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.152241 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.150290    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-cnibin\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.152241 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.150328    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmxvw\" (UniqueName: \"kubernetes.io/projected/d45b9a7f-3c63-4f52-863f-5d128fd07d93-kube-api-access-nmxvw\") pod \"node-ca-bpx78\" (UID: \"d45b9a7f-3c63-4f52-863f-5d128fd07d93\") " pod="openshift-image-registry/node-ca-bpx78"
Jul 08 16:34:42.152241 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.150353    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-multus-conf-dir\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.152241 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.150361    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96mh7\" (UniqueName: \"kubernetes.io/projected/ecc650f2-27aa-4300-b61a-8328a41d599d-kube-api-access-96mh7\") pod \"node-resolver-pq4sj\" (UID: \"ecc650f2-27aa-4300-b61a-8328a41d599d\") " pod="openshift-dns/node-resolver-pq4sj"
Jul 08 16:34:42.152241 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.150391    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d9ddf360-c5ae-4ef7-a947-59405bf4e8db-system-cni-dir\") pod \"multus-additional-cni-plugins-mw6h7\" (UID: \"d9ddf360-c5ae-4ef7-a947-59405bf4e8db\") " pod="openshift-multus/multus-additional-cni-plugins-mw6h7"
Jul 08 16:34:42.152241 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.150389    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-etc-kubernetes\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.152241 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.150787    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d45b9a7f-3c63-4f52-863f-5d128fd07d93-serviceca\") pod \"node-ca-bpx78\" (UID: \"d45b9a7f-3c63-4f52-863f-5d128fd07d93\") " pod="openshift-image-registry/node-ca-bpx78"
Jul 08 16:34:42.152241 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.150801    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d9ddf360-c5ae-4ef7-a947-59405bf4e8db-cni-binary-copy\") pod \"multus-additional-cni-plugins-mw6h7\" (UID: \"d9ddf360-c5ae-4ef7-a947-59405bf4e8db\") " pod="openshift-multus/multus-additional-cni-plugins-mw6h7"
Jul 08 16:34:42.153062 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.151164    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a2b65945-693a-42f8-9955-f484a3f96d10-cni-binary-copy\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.153062 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.151193    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d9ddf360-c5ae-4ef7-a947-59405bf4e8db-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mw6h7\" (UID: \"d9ddf360-c5ae-4ef7-a947-59405bf4e8db\") " pod="openshift-multus/multus-additional-cni-plugins-mw6h7"
Jul 08 16:34:42.153062 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.151865    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-kubelet-config\") pod \"global-pull-secret-syncer-4vtkl\" (UID: \"8eb2abc0-b610-4f3b-b6dc-02131ad29e35\") " pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:34:42.153062 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.151924    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-hostroot\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.153062 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.151952    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkwp8\" (UniqueName: \"kubernetes.io/projected/6c79a3b7-b6b3-4264-8c73-36fc32701b6e-kube-api-access-dkwp8\") pod \"aws-ebs-csi-driver-node-ns5pm\" (UID: \"6c79a3b7-b6b3-4264-8c73-36fc32701b6e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5pm"
Jul 08 16:34:42.153062 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.151982    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-host-run-netns\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.153062 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.152012    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2vlb\" (UniqueName: \"kubernetes.io/projected/d9ddf360-c5ae-4ef7-a947-59405bf4e8db-kube-api-access-h2vlb\") pod \"multus-additional-cni-plugins-mw6h7\" (UID: \"d9ddf360-c5ae-4ef7-a947-59405bf4e8db\") " pod="openshift-multus/multus-additional-cni-plugins-mw6h7"
Jul 08 16:34:42.153062 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.152045    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-original-pull-secret\") pod \"global-pull-secret-syncer-4vtkl\" (UID: \"8eb2abc0-b610-4f3b-b6dc-02131ad29e35\") " pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:34:42.153062 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.152073    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6c79a3b7-b6b3-4264-8c73-36fc32701b6e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ns5pm\" (UID: \"6c79a3b7-b6b3-4264-8c73-36fc32701b6e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5pm"
Jul 08 16:34:42.153062 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.152095    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6c79a3b7-b6b3-4264-8c73-36fc32701b6e-sys-fs\") pod \"aws-ebs-csi-driver-node-ns5pm\" (UID: \"6c79a3b7-b6b3-4264-8c73-36fc32701b6e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5pm"
Jul 08 16:34:42.153062 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.152122    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7dqq\" (UniqueName: \"kubernetes.io/projected/19df7751-4b62-4ae5-92a6-0bbda11fec85-kube-api-access-c7dqq\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.153062 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.152149    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecc650f2-27aa-4300-b61a-8328a41d599d-tmp-dir\") pod \"node-resolver-pq4sj\" (UID: \"ecc650f2-27aa-4300-b61a-8328a41d599d\") " pod="openshift-dns/node-resolver-pq4sj"
Jul 08 16:34:42.153062 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.152176    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d9ddf360-c5ae-4ef7-a947-59405bf4e8db-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mw6h7\" (UID: \"d9ddf360-c5ae-4ef7-a947-59405bf4e8db\") " pod="openshift-multus/multus-additional-cni-plugins-mw6h7"
Jul 08 16:34:42.153062 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.152205    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d9ddf360-c5ae-4ef7-a947-59405bf4e8db-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mw6h7\" (UID: \"d9ddf360-c5ae-4ef7-a947-59405bf4e8db\") " pod="openshift-multus/multus-additional-cni-plugins-mw6h7"
Jul 08 16:34:42.153062 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.152237    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6c79a3b7-b6b3-4264-8c73-36fc32701b6e-registration-dir\") pod \"aws-ebs-csi-driver-node-ns5pm\" (UID: \"6c79a3b7-b6b3-4264-8c73-36fc32701b6e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5pm"
Jul 08 16:34:42.153062 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.152267    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-var-lib-kubelet\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.153764 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.152346    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/782cdc3e-e8c4-4d37-a22e-832fac199082-etc-tuned\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.153764 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.152300    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-log-socket\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.153764 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.152435    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-host-cni-bin\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.153764 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.152451    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f877b6b1-282c-4c50-95d6-530371d41a15-agent-certs\") pod \"konnectivity-agent-957gg\" (UID: \"f877b6b1-282c-4c50-95d6-530371d41a15\") " pod="kube-system/konnectivity-agent-957gg"
Jul 08 16:34:42.153764 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.152528    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/782cdc3e-e8c4-4d37-a22e-832fac199082-tmp\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.153764 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:42.152568    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 16:34:42.153764 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:42.152624    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-original-pull-secret podName:8eb2abc0-b610-4f3b-b6dc-02131ad29e35 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:42.652607656 +0000 UTC m=+1.993572712 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-original-pull-secret") pod "global-pull-secret-syncer-4vtkl" (UID: "8eb2abc0-b610-4f3b-b6dc-02131ad29e35") : object "kube-system"/"original-pull-secret" not registered
Jul 08 16:34:42.153764 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.152622    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6c79a3b7-b6b3-4264-8c73-36fc32701b6e-sys-fs\") pod \"aws-ebs-csi-driver-node-ns5pm\" (UID: \"6c79a3b7-b6b3-4264-8c73-36fc32701b6e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5pm"
Jul 08 16:34:42.153764 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.152622    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6c79a3b7-b6b3-4264-8c73-36fc32701b6e-registration-dir\") pod \"aws-ebs-csi-driver-node-ns5pm\" (UID: \"6c79a3b7-b6b3-4264-8c73-36fc32701b6e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5pm"
Jul 08 16:34:42.153764 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.152683    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6c79a3b7-b6b3-4264-8c73-36fc32701b6e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ns5pm\" (UID: \"6c79a3b7-b6b3-4264-8c73-36fc32701b6e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5pm"
Jul 08 16:34:42.153764 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.152697    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/782cdc3e-e8c4-4d37-a22e-832fac199082-var-lib-kubelet\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.153764 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.152810    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-kubelet-config\") pod \"global-pull-secret-syncer-4vtkl\" (UID: \"8eb2abc0-b610-4f3b-b6dc-02131ad29e35\") " pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:34:42.153764 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.152819    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a2b65945-693a-42f8-9955-f484a3f96d10-hostroot\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.160902 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:42.160882    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 16:34:42.160902 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:42.160903    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 16:34:42.161049 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:42.160913    2582 projected.go:196] Error preparing data for projected volume kube-api-access-kqv9w for pod openshift-network-diagnostics/network-check-target-bkqjt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 16:34:42.161049 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:42.160981    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c98cc144-3322-4f83-b571-9772b5a63d49-kube-api-access-kqv9w podName:c98cc144-3322-4f83-b571-9772b5a63d49 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:42.660961746 +0000 UTC m=+2.001926815 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kqv9w" (UniqueName: "kubernetes.io/projected/c98cc144-3322-4f83-b571-9772b5a63d49-kube-api-access-kqv9w") pod "network-check-target-bkqjt" (UID: "c98cc144-3322-4f83-b571-9772b5a63d49") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 16:34:42.164998 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.164961    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9txx\" (UniqueName: \"kubernetes.io/projected/2796ef11-ffb4-4bff-b73b-38d727eb45f2-kube-api-access-k9txx\") pod \"network-metrics-daemon-z82nm\" (UID: \"2796ef11-ffb4-4bff-b73b-38d727eb45f2\") " pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:34:42.165079 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.165034    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p2pz\" (UniqueName: \"kubernetes.io/projected/782cdc3e-e8c4-4d37-a22e-832fac199082-kube-api-access-8p2pz\") pod \"tuned-p46xh\" (UID: \"782cdc3e-e8c4-4d37-a22e-832fac199082\") " pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.165527 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.165500    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmxvw\" (UniqueName: \"kubernetes.io/projected/d45b9a7f-3c63-4f52-863f-5d128fd07d93-kube-api-access-nmxvw\") pod \"node-ca-bpx78\" (UID: \"d45b9a7f-3c63-4f52-863f-5d128fd07d93\") " pod="openshift-image-registry/node-ca-bpx78"
Jul 08 16:34:42.165869 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.165845    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s7x8\" (UniqueName: \"kubernetes.io/projected/a2b65945-693a-42f8-9955-f484a3f96d10-kube-api-access-2s7x8\") pod \"multus-jgp2d\" (UID: \"a2b65945-693a-42f8-9955-f484a3f96d10\") " pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.165908 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.165869    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkwp8\" (UniqueName: \"kubernetes.io/projected/6c79a3b7-b6b3-4264-8c73-36fc32701b6e-kube-api-access-dkwp8\") pod \"aws-ebs-csi-driver-node-ns5pm\" (UID: \"6c79a3b7-b6b3-4264-8c73-36fc32701b6e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5pm"
Jul 08 16:34:42.252991 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.252950    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-run-ovn\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.252991 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.252988    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ecc650f2-27aa-4300-b61a-8328a41d599d-hosts-file\") pod \"node-resolver-pq4sj\" (UID: \"ecc650f2-27aa-4300-b61a-8328a41d599d\") " pod="openshift-dns/node-resolver-pq4sj"
Jul 08 16:34:42.253229 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253010    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a12a9aeb-4a05-41b1-9f81-dc6dc476fda8-iptables-alerter-script\") pod \"iptables-alerter-8hh26\" (UID: \"a12a9aeb-4a05-41b1-9f81-dc6dc476fda8\") " pod="openshift-network-operator/iptables-alerter-8hh26"
Jul 08 16:34:42.253229 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253027    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-host-run-ovn-kubernetes\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.253229 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253044    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/19df7751-4b62-4ae5-92a6-0bbda11fec85-env-overrides\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.253229 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253067    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-var-lib-openvswitch\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.253229 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253072    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-run-ovn\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.253229 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253089    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d9ddf360-c5ae-4ef7-a947-59405bf4e8db-cnibin\") pod \"multus-additional-cni-plugins-mw6h7\" (UID: \"d9ddf360-c5ae-4ef7-a947-59405bf4e8db\") " pod="openshift-multus/multus-additional-cni-plugins-mw6h7"
Jul 08 16:34:42.253229 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253097    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ecc650f2-27aa-4300-b61a-8328a41d599d-hosts-file\") pod \"node-resolver-pq4sj\" (UID: \"ecc650f2-27aa-4300-b61a-8328a41d599d\") " pod="openshift-dns/node-resolver-pq4sj"
Jul 08 16:34:42.253229 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253113    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vts62\" (UniqueName: \"kubernetes.io/projected/a12a9aeb-4a05-41b1-9f81-dc6dc476fda8-kube-api-access-vts62\") pod \"iptables-alerter-8hh26\" (UID: \"a12a9aeb-4a05-41b1-9f81-dc6dc476fda8\") " pod="openshift-network-operator/iptables-alerter-8hh26"
Jul 08 16:34:42.253229 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253143    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d9ddf360-c5ae-4ef7-a947-59405bf4e8db-cnibin\") pod \"multus-additional-cni-plugins-mw6h7\" (UID: \"d9ddf360-c5ae-4ef7-a947-59405bf4e8db\") " pod="openshift-multus/multus-additional-cni-plugins-mw6h7"
Jul 08 16:34:42.253229 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253152    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-systemd-units\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.253229 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253098    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-host-run-ovn-kubernetes\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.253229 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253162    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-var-lib-openvswitch\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.253229 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253179    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-node-log\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.253229 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253198    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-host-cni-netd\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.253229 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253202    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-systemd-units\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.253229 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253210    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-node-log\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.253229 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253217    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/19df7751-4b62-4ae5-92a6-0bbda11fec85-ovnkube-config\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.254039 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253261    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-run-systemd\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.254039 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253277    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d9ddf360-c5ae-4ef7-a947-59405bf4e8db-os-release\") pod \"multus-additional-cni-plugins-mw6h7\" (UID: \"d9ddf360-c5ae-4ef7-a947-59405bf4e8db\") " pod="openshift-multus/multus-additional-cni-plugins-mw6h7"
Jul 08 16:34:42.254039 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253259    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-host-cni-netd\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.254039 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253303    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-etc-openvswitch\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.254039 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253347    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/19df7751-4b62-4ae5-92a6-0bbda11fec85-ovnkube-script-lib\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.254039 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253365    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-run-systemd\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.254039 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253370    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-etc-openvswitch\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.254039 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253376    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96mh7\" (UniqueName: \"kubernetes.io/projected/ecc650f2-27aa-4300-b61a-8328a41d599d-kube-api-access-96mh7\") pod \"node-resolver-pq4sj\" (UID: \"ecc650f2-27aa-4300-b61a-8328a41d599d\") " pod="openshift-dns/node-resolver-pq4sj"
Jul 08 16:34:42.254039 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253424    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d9ddf360-c5ae-4ef7-a947-59405bf4e8db-system-cni-dir\") pod \"multus-additional-cni-plugins-mw6h7\" (UID: \"d9ddf360-c5ae-4ef7-a947-59405bf4e8db\") " pod="openshift-multus/multus-additional-cni-plugins-mw6h7"
Jul 08 16:34:42.254039 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253441    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d9ddf360-c5ae-4ef7-a947-59405bf4e8db-os-release\") pod \"multus-additional-cni-plugins-mw6h7\" (UID: \"d9ddf360-c5ae-4ef7-a947-59405bf4e8db\") " pod="openshift-multus/multus-additional-cni-plugins-mw6h7"
Jul 08 16:34:42.254039 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253518    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d9ddf360-c5ae-4ef7-a947-59405bf4e8db-cni-binary-copy\") pod \"multus-additional-cni-plugins-mw6h7\" (UID: \"d9ddf360-c5ae-4ef7-a947-59405bf4e8db\") " pod="openshift-multus/multus-additional-cni-plugins-mw6h7"
Jul 08 16:34:42.254039 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253544    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d9ddf360-c5ae-4ef7-a947-59405bf4e8db-system-cni-dir\") pod \"multus-additional-cni-plugins-mw6h7\" (UID: \"d9ddf360-c5ae-4ef7-a947-59405bf4e8db\") " pod="openshift-multus/multus-additional-cni-plugins-mw6h7"
Jul 08 16:34:42.254039 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253608    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d9ddf360-c5ae-4ef7-a947-59405bf4e8db-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mw6h7\" (UID: \"d9ddf360-c5ae-4ef7-a947-59405bf4e8db\") " pod="openshift-multus/multus-additional-cni-plugins-mw6h7"
Jul 08 16:34:42.254039 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253638    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/19df7751-4b62-4ae5-92a6-0bbda11fec85-env-overrides\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.254039 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253668    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-host-run-netns\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.254039 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253701    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2vlb\" (UniqueName: \"kubernetes.io/projected/d9ddf360-c5ae-4ef7-a947-59405bf4e8db-kube-api-access-h2vlb\") pod \"multus-additional-cni-plugins-mw6h7\" (UID: \"d9ddf360-c5ae-4ef7-a947-59405bf4e8db\") " pod="openshift-multus/multus-additional-cni-plugins-mw6h7"
Jul 08 16:34:42.254039 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253708    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a12a9aeb-4a05-41b1-9f81-dc6dc476fda8-iptables-alerter-script\") pod \"iptables-alerter-8hh26\" (UID: \"a12a9aeb-4a05-41b1-9f81-dc6dc476fda8\") " pod="openshift-network-operator/iptables-alerter-8hh26"
Jul 08 16:34:42.254615 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253742    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7dqq\" (UniqueName: \"kubernetes.io/projected/19df7751-4b62-4ae5-92a6-0bbda11fec85-kube-api-access-c7dqq\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.254615 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253702    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-host-run-netns\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.254615 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253766    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecc650f2-27aa-4300-b61a-8328a41d599d-tmp-dir\") pod \"node-resolver-pq4sj\" (UID: \"ecc650f2-27aa-4300-b61a-8328a41d599d\") " pod="openshift-dns/node-resolver-pq4sj"
Jul 08 16:34:42.254615 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253791    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d9ddf360-c5ae-4ef7-a947-59405bf4e8db-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mw6h7\" (UID: \"d9ddf360-c5ae-4ef7-a947-59405bf4e8db\") " pod="openshift-multus/multus-additional-cni-plugins-mw6h7"
Jul 08 16:34:42.254615 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253763    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/19df7751-4b62-4ae5-92a6-0bbda11fec85-ovnkube-config\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.254615 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253815    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d9ddf360-c5ae-4ef7-a947-59405bf4e8db-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mw6h7\" (UID: \"d9ddf360-c5ae-4ef7-a947-59405bf4e8db\") " pod="openshift-multus/multus-additional-cni-plugins-mw6h7"
Jul 08 16:34:42.254615 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253840    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-log-socket\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.254615 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253864    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-host-cni-bin\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.254615 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253895    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a12a9aeb-4a05-41b1-9f81-dc6dc476fda8-host-slash\") pod \"iptables-alerter-8hh26\" (UID: \"a12a9aeb-4a05-41b1-9f81-dc6dc476fda8\") " pod="openshift-network-operator/iptables-alerter-8hh26"
Jul 08 16:34:42.254615 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253927    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-log-socket\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.254615 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253936    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.254615 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253960    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/19df7751-4b62-4ae5-92a6-0bbda11fec85-ovn-node-metrics-cert\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.254615 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.253973    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d9ddf360-c5ae-4ef7-a947-59405bf4e8db-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mw6h7\" (UID: \"d9ddf360-c5ae-4ef7-a947-59405bf4e8db\") " pod="openshift-multus/multus-additional-cni-plugins-mw6h7"
Jul 08 16:34:42.254615 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.254009    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a12a9aeb-4a05-41b1-9f81-dc6dc476fda8-host-slash\") pod \"iptables-alerter-8hh26\" (UID: \"a12a9aeb-4a05-41b1-9f81-dc6dc476fda8\") " pod="openshift-network-operator/iptables-alerter-8hh26"
Jul 08 16:34:42.254615 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.254017    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-host-kubelet\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.254615 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.254024    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.254615 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.254041    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/19df7751-4b62-4ae5-92a6-0bbda11fec85-ovnkube-script-lib\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.255124 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.254055    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-host-cni-bin\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.255124 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.254059    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecc650f2-27aa-4300-b61a-8328a41d599d-tmp-dir\") pod \"node-resolver-pq4sj\" (UID: \"ecc650f2-27aa-4300-b61a-8328a41d599d\") " pod="openshift-dns/node-resolver-pq4sj"
Jul 08 16:34:42.255124 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.254043    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-host-slash\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.255124 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.254084    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-host-slash\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.255124 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.254090    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-host-kubelet\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.255124 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.254106    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-run-openvswitch\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.255124 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.254152    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d9ddf360-c5ae-4ef7-a947-59405bf4e8db-cni-binary-copy\") pod \"multus-additional-cni-plugins-mw6h7\" (UID: \"d9ddf360-c5ae-4ef7-a947-59405bf4e8db\") " pod="openshift-multus/multus-additional-cni-plugins-mw6h7"
Jul 08 16:34:42.255124 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.254171    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/19df7751-4b62-4ae5-92a6-0bbda11fec85-run-openvswitch\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.255124 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.254416    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d9ddf360-c5ae-4ef7-a947-59405bf4e8db-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mw6h7\" (UID: \"d9ddf360-c5ae-4ef7-a947-59405bf4e8db\") " pod="openshift-multus/multus-additional-cni-plugins-mw6h7"
Jul 08 16:34:42.255124 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.254574    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d9ddf360-c5ae-4ef7-a947-59405bf4e8db-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mw6h7\" (UID: \"d9ddf360-c5ae-4ef7-a947-59405bf4e8db\") " pod="openshift-multus/multus-additional-cni-plugins-mw6h7"
Jul 08 16:34:42.256235 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.256214    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/19df7751-4b62-4ae5-92a6-0bbda11fec85-ovn-node-metrics-cert\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.262006 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.261983    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vts62\" (UniqueName: \"kubernetes.io/projected/a12a9aeb-4a05-41b1-9f81-dc6dc476fda8-kube-api-access-vts62\") pod \"iptables-alerter-8hh26\" (UID: \"a12a9aeb-4a05-41b1-9f81-dc6dc476fda8\") " pod="openshift-network-operator/iptables-alerter-8hh26"
Jul 08 16:34:42.262340 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.262305    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2vlb\" (UniqueName: \"kubernetes.io/projected/d9ddf360-c5ae-4ef7-a947-59405bf4e8db-kube-api-access-h2vlb\") pod \"multus-additional-cni-plugins-mw6h7\" (UID: \"d9ddf360-c5ae-4ef7-a947-59405bf4e8db\") " pod="openshift-multus/multus-additional-cni-plugins-mw6h7"
Jul 08 16:34:42.263848 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.263831    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96mh7\" (UniqueName: \"kubernetes.io/projected/ecc650f2-27aa-4300-b61a-8328a41d599d-kube-api-access-96mh7\") pod \"node-resolver-pq4sj\" (UID: \"ecc650f2-27aa-4300-b61a-8328a41d599d\") " pod="openshift-dns/node-resolver-pq4sj"
Jul 08 16:34:42.267333 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.267297    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7dqq\" (UniqueName: \"kubernetes.io/projected/19df7751-4b62-4ae5-92a6-0bbda11fec85-kube-api-access-c7dqq\") pod \"ovnkube-node-kvhpr\" (UID: \"19df7751-4b62-4ae5-92a6-0bbda11fec85\") " pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.348449 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.348356    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5pm"
Jul 08 16:34:42.371711 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.371653    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-p46xh"
Jul 08 16:34:42.394338 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.394296    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bpx78"
Jul 08 16:34:42.407243 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.407223    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jgp2d"
Jul 08 16:34:42.411977 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:42.411933    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd45b9a7f_3c63_4f52_863f_5d128fd07d93.slice/crio-2e2c76a60b46fa6a6d0c1b403e4967998b87f9b4bc29c512017d79bf19ce057a WatchSource:0}: Error finding container 2e2c76a60b46fa6a6d0c1b403e4967998b87f9b4bc29c512017d79bf19ce057a: Status 404 returned error can't find the container with id 2e2c76a60b46fa6a6d0c1b403e4967998b87f9b4bc29c512017d79bf19ce057a
Jul 08 16:34:42.412326 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:42.412294    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81aa7add2915dbb7aae72d3a67b06d72.slice/crio-90acc36ccff2aea6284a006f8ffd722470de7e40beb11221a76897eb0d3620d2 WatchSource:0}: Error finding container 90acc36ccff2aea6284a006f8ffd722470de7e40beb11221a76897eb0d3620d2: Status 404 returned error can't find the container with id 90acc36ccff2aea6284a006f8ffd722470de7e40beb11221a76897eb0d3620d2
Jul 08 16:34:42.413019 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:42.412999    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d4667db5036e322bfdaf5880d0290f1.slice/crio-41a7b59a427249ca768832718bd75684161685f63ad83eff506f9475d72d5ee6 WatchSource:0}: Error finding container 41a7b59a427249ca768832718bd75684161685f63ad83eff506f9475d72d5ee6: Status 404 returned error can't find the container with id 41a7b59a427249ca768832718bd75684161685f63ad83eff506f9475d72d5ee6
Jul 08 16:34:42.417291 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:42.417269    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2b65945_693a_42f8_9955_f484a3f96d10.slice/crio-40269dd558f11cdaed342dbaa51ee0776918d2066546cc419b99b948d231b98a WatchSource:0}: Error finding container 40269dd558f11cdaed342dbaa51ee0776918d2066546cc419b99b948d231b98a: Status 404 returned error can't find the container with id 40269dd558f11cdaed342dbaa51ee0776918d2066546cc419b99b948d231b98a
Jul 08 16:34:42.419822 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.419675    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 16:34:42.427474 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.427457    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-957gg"
Jul 08 16:34:42.435382 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:42.435360    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf877b6b1_282c_4c50_95d6_530371d41a15.slice/crio-137113ee83705d64a229e513ae51ccdc4b451857326097ea5ebc018e797cb8d5 WatchSource:0}: Error finding container 137113ee83705d64a229e513ae51ccdc4b451857326097ea5ebc018e797cb8d5: Status 404 returned error can't find the container with id 137113ee83705d64a229e513ae51ccdc4b451857326097ea5ebc018e797cb8d5
Jul 08 16:34:42.442868 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.442847    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pq4sj"
Jul 08 16:34:42.450300 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:42.450275    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecc650f2_27aa_4300_b61a_8328a41d599d.slice/crio-81d877ecb59d4a81a25d3ea8a2129fc40ff9d1f7d04849b68d34a4260b835bce WatchSource:0}: Error finding container 81d877ecb59d4a81a25d3ea8a2129fc40ff9d1f7d04849b68d34a4260b835bce: Status 404 returned error can't find the container with id 81d877ecb59d4a81a25d3ea8a2129fc40ff9d1f7d04849b68d34a4260b835bce
Jul 08 16:34:42.458831 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.458810    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mw6h7"
Jul 08 16:34:42.465664 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:42.465641    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9ddf360_c5ae_4ef7_a947_59405bf4e8db.slice/crio-434c6949fe6e0983390dc203840f7e93c103d31be209f8e79acc93ef8772792c WatchSource:0}: Error finding container 434c6949fe6e0983390dc203840f7e93c103d31be209f8e79acc93ef8772792c: Status 404 returned error can't find the container with id 434c6949fe6e0983390dc203840f7e93c103d31be209f8e79acc93ef8772792c
Jul 08 16:34:42.466759 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.466739    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8hh26"
Jul 08 16:34:42.474425 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:42.474398    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda12a9aeb_4a05_41b1_9f81_dc6dc476fda8.slice/crio-140ad4476e4551891a18c253ba7c0d8a36787d26315337c2b59051c7c08179c8 WatchSource:0}: Error finding container 140ad4476e4551891a18c253ba7c0d8a36787d26315337c2b59051c7c08179c8: Status 404 returned error can't find the container with id 140ad4476e4551891a18c253ba7c0d8a36787d26315337c2b59051c7c08179c8
Jul 08 16:34:42.482429 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.482411    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:34:42.489763 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:42.489741    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19df7751_4b62_4ae5_92a6_0bbda11fec85.slice/crio-8b948ec92b6a756e405028d1989641148f95f060c60e7c5f4ff01e6679945556 WatchSource:0}: Error finding container 8b948ec92b6a756e405028d1989641148f95f060c60e7c5f4ff01e6679945556: Status 404 returned error can't find the container with id 8b948ec92b6a756e405028d1989641148f95f060c60e7c5f4ff01e6679945556
Jul 08 16:34:42.657222 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.657134    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2796ef11-ffb4-4bff-b73b-38d727eb45f2-metrics-certs\") pod \"network-metrics-daemon-z82nm\" (UID: \"2796ef11-ffb4-4bff-b73b-38d727eb45f2\") " pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:34:42.657222 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.657206    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-original-pull-secret\") pod \"global-pull-secret-syncer-4vtkl\" (UID: \"8eb2abc0-b610-4f3b-b6dc-02131ad29e35\") " pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:34:42.657477 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:42.657259    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 16:34:42.657477 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:42.657301    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 16:34:42.657477 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:42.657359    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2796ef11-ffb4-4bff-b73b-38d727eb45f2-metrics-certs podName:2796ef11-ffb4-4bff-b73b-38d727eb45f2 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:43.65733762 +0000 UTC m=+2.998302677 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2796ef11-ffb4-4bff-b73b-38d727eb45f2-metrics-certs") pod "network-metrics-daemon-z82nm" (UID: "2796ef11-ffb4-4bff-b73b-38d727eb45f2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 16:34:42.657477 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:42.657382    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-original-pull-secret podName:8eb2abc0-b610-4f3b-b6dc-02131ad29e35 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:43.657373353 +0000 UTC m=+2.998338410 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-original-pull-secret") pod "global-pull-secret-syncer-4vtkl" (UID: "8eb2abc0-b610-4f3b-b6dc-02131ad29e35") : object "kube-system"/"original-pull-secret" not registered
Jul 08 16:34:42.702528 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:42.702500    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c79a3b7_b6b3_4264_8c73_36fc32701b6e.slice/crio-5ca525a3631b9591568502c9a895c7e208ad86068d4053ce0f107ddd80e7f412 WatchSource:0}: Error finding container 5ca525a3631b9591568502c9a895c7e208ad86068d4053ce0f107ddd80e7f412: Status 404 returned error can't find the container with id 5ca525a3631b9591568502c9a895c7e208ad86068d4053ce0f107ddd80e7f412
Jul 08 16:34:42.758238 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.758203    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqv9w\" (UniqueName: \"kubernetes.io/projected/c98cc144-3322-4f83-b571-9772b5a63d49-kube-api-access-kqv9w\") pod \"network-check-target-bkqjt\" (UID: \"c98cc144-3322-4f83-b571-9772b5a63d49\") " pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:34:42.758428 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:42.758401    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 16:34:42.758428 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:42.758424    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 16:34:42.758506 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:42.758433    2582 projected.go:196] Error preparing data for projected volume kube-api-access-kqv9w for pod openshift-network-diagnostics/network-check-target-bkqjt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 16:34:42.758550 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:42.758504    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c98cc144-3322-4f83-b571-9772b5a63d49-kube-api-access-kqv9w podName:c98cc144-3322-4f83-b571-9772b5a63d49 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:43.758479841 +0000 UTC m=+3.099444900 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kqv9w" (UniqueName: "kubernetes.io/projected/c98cc144-3322-4f83-b571-9772b5a63d49-kube-api-access-kqv9w") pod "network-check-target-bkqjt" (UID: "c98cc144-3322-4f83-b571-9772b5a63d49") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 16:34:42.835386 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:42.835345    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 16:34:43.097295 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:43.097181    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 16:29:42 +0000 UTC" deadline="2028-02-15 11:59:07.682998792 +0000 UTC"
Jul 08 16:34:43.097295 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:43.097243    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14083h24m24.585759817s"
Jul 08 16:34:43.172849 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:43.172814    2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 16:34:43.216267 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:34:43.216238    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod782cdc3e_e8c4_4d37_a22e_832fac199082.slice/crio-0b82d0ffbc685f22007a76178b8ac4e566a8bc0cc9d5343450356824e62f4f63 WatchSource:0}: Error finding container 0b82d0ffbc685f22007a76178b8ac4e566a8bc0cc9d5343450356824e62f4f63: Status 404 returned error can't find the container with id 0b82d0ffbc685f22007a76178b8ac4e566a8bc0cc9d5343450356824e62f4f63
Jul 08 16:34:43.239545 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:43.239355    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr" event={"ID":"19df7751-4b62-4ae5-92a6-0bbda11fec85","Type":"ContainerStarted","Data":"8b948ec92b6a756e405028d1989641148f95f060c60e7c5f4ff01e6679945556"}
Jul 08 16:34:43.241976 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:43.241944    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8hh26" event={"ID":"a12a9aeb-4a05-41b1-9f81-dc6dc476fda8","Type":"ContainerStarted","Data":"140ad4476e4551891a18c253ba7c0d8a36787d26315337c2b59051c7c08179c8"}
Jul 08 16:34:43.249927 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:43.249891    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mw6h7" event={"ID":"d9ddf360-c5ae-4ef7-a947-59405bf4e8db","Type":"ContainerStarted","Data":"434c6949fe6e0983390dc203840f7e93c103d31be209f8e79acc93ef8772792c"}
Jul 08 16:34:43.256881 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:43.256815    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bpx78" event={"ID":"d45b9a7f-3c63-4f52-863f-5d128fd07d93","Type":"ContainerStarted","Data":"2e2c76a60b46fa6a6d0c1b403e4967998b87f9b4bc29c512017d79bf19ce057a"}
Jul 08 16:34:43.263797 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:43.263766    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5pm" event={"ID":"6c79a3b7-b6b3-4264-8c73-36fc32701b6e","Type":"ContainerStarted","Data":"5ca525a3631b9591568502c9a895c7e208ad86068d4053ce0f107ddd80e7f412"}
Jul 08 16:34:43.277101 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:43.277061    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pq4sj" event={"ID":"ecc650f2-27aa-4300-b61a-8328a41d599d","Type":"ContainerStarted","Data":"81d877ecb59d4a81a25d3ea8a2129fc40ff9d1f7d04849b68d34a4260b835bce"}
Jul 08 16:34:43.303475 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:43.303434    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-957gg" event={"ID":"f877b6b1-282c-4c50-95d6-530371d41a15","Type":"ContainerStarted","Data":"137113ee83705d64a229e513ae51ccdc4b451857326097ea5ebc018e797cb8d5"}
Jul 08 16:34:43.314010 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:43.313928    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jgp2d" event={"ID":"a2b65945-693a-42f8-9955-f484a3f96d10","Type":"ContainerStarted","Data":"40269dd558f11cdaed342dbaa51ee0776918d2066546cc419b99b948d231b98a"}
Jul 08 16:34:43.324128 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:43.324002    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-237.ec2.internal" event={"ID":"81aa7add2915dbb7aae72d3a67b06d72","Type":"ContainerStarted","Data":"90acc36ccff2aea6284a006f8ffd722470de7e40beb11221a76897eb0d3620d2"}
Jul 08 16:34:43.333148 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:43.333019    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-237.ec2.internal" event={"ID":"9d4667db5036e322bfdaf5880d0290f1","Type":"ContainerStarted","Data":"41a7b59a427249ca768832718bd75684161685f63ad83eff506f9475d72d5ee6"}
Jul 08 16:34:43.338591 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:43.338557    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-p46xh" event={"ID":"782cdc3e-e8c4-4d37-a22e-832fac199082","Type":"ContainerStarted","Data":"0b82d0ffbc685f22007a76178b8ac4e566a8bc0cc9d5343450356824e62f4f63"}
Jul 08 16:34:43.666099 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:43.666014    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-original-pull-secret\") pod \"global-pull-secret-syncer-4vtkl\" (UID: \"8eb2abc0-b610-4f3b-b6dc-02131ad29e35\") " pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:34:43.666099 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:43.666091    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2796ef11-ffb4-4bff-b73b-38d727eb45f2-metrics-certs\") pod \"network-metrics-daemon-z82nm\" (UID: \"2796ef11-ffb4-4bff-b73b-38d727eb45f2\") " pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:34:43.666360 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:43.666216    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 16:34:43.666360 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:43.666280    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2796ef11-ffb4-4bff-b73b-38d727eb45f2-metrics-certs podName:2796ef11-ffb4-4bff-b73b-38d727eb45f2 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:45.666260968 +0000 UTC m=+5.007226039 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2796ef11-ffb4-4bff-b73b-38d727eb45f2-metrics-certs") pod "network-metrics-daemon-z82nm" (UID: "2796ef11-ffb4-4bff-b73b-38d727eb45f2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 16:34:43.666749 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:43.666726    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 16:34:43.666828 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:43.666790    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-original-pull-secret podName:8eb2abc0-b610-4f3b-b6dc-02131ad29e35 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:45.66677259 +0000 UTC m=+5.007737662 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-original-pull-secret") pod "global-pull-secret-syncer-4vtkl" (UID: "8eb2abc0-b610-4f3b-b6dc-02131ad29e35") : object "kube-system"/"original-pull-secret" not registered
Jul 08 16:34:43.766606 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:43.766572    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqv9w\" (UniqueName: \"kubernetes.io/projected/c98cc144-3322-4f83-b571-9772b5a63d49-kube-api-access-kqv9w\") pod \"network-check-target-bkqjt\" (UID: \"c98cc144-3322-4f83-b571-9772b5a63d49\") " pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:34:43.766787 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:43.766769    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 16:34:43.766852 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:43.766790    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 16:34:43.766852 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:43.766802    2582 projected.go:196] Error preparing data for projected volume kube-api-access-kqv9w for pod openshift-network-diagnostics/network-check-target-bkqjt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 16:34:43.766946 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:43.766856    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c98cc144-3322-4f83-b571-9772b5a63d49-kube-api-access-kqv9w podName:c98cc144-3322-4f83-b571-9772b5a63d49 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:45.766838151 +0000 UTC m=+5.107803210 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kqv9w" (UniqueName: "kubernetes.io/projected/c98cc144-3322-4f83-b571-9772b5a63d49-kube-api-access-kqv9w") pod "network-check-target-bkqjt" (UID: "c98cc144-3322-4f83-b571-9772b5a63d49") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 16:34:44.097885 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:44.097770    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 16:29:42 +0000 UTC" deadline="2028-02-25 17:15:06.681006961 +0000 UTC"
Jul 08 16:34:44.097885 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:44.097805    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14328h40m22.583206344s"
Jul 08 16:34:44.212871 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:44.212007    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:34:44.212871 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:44.212137    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtkl" podUID="8eb2abc0-b610-4f3b-b6dc-02131ad29e35"
Jul 08 16:34:44.212871 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:44.212568    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:34:44.212871 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:44.212675    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z82nm" podUID="2796ef11-ffb4-4bff-b73b-38d727eb45f2"
Jul 08 16:34:44.212871 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:44.212736    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:34:44.212871 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:44.212813    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkqjt" podUID="c98cc144-3322-4f83-b571-9772b5a63d49"
Jul 08 16:34:45.682956 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:45.682742    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2796ef11-ffb4-4bff-b73b-38d727eb45f2-metrics-certs\") pod \"network-metrics-daemon-z82nm\" (UID: \"2796ef11-ffb4-4bff-b73b-38d727eb45f2\") " pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:34:45.682956 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:45.682816    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-original-pull-secret\") pod \"global-pull-secret-syncer-4vtkl\" (UID: \"8eb2abc0-b610-4f3b-b6dc-02131ad29e35\") " pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:34:45.682956 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:45.682941    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 16:34:45.683475 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:45.683000    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-original-pull-secret podName:8eb2abc0-b610-4f3b-b6dc-02131ad29e35 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:49.682981905 +0000 UTC m=+9.023946967 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-original-pull-secret") pod "global-pull-secret-syncer-4vtkl" (UID: "8eb2abc0-b610-4f3b-b6dc-02131ad29e35") : object "kube-system"/"original-pull-secret" not registered
Jul 08 16:34:45.683546 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:45.683493    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 16:34:45.683598 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:45.683549    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2796ef11-ffb4-4bff-b73b-38d727eb45f2-metrics-certs podName:2796ef11-ffb4-4bff-b73b-38d727eb45f2 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:49.683533079 +0000 UTC m=+9.024498141 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2796ef11-ffb4-4bff-b73b-38d727eb45f2-metrics-certs") pod "network-metrics-daemon-z82nm" (UID: "2796ef11-ffb4-4bff-b73b-38d727eb45f2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 16:34:45.784180 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:45.783539    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqv9w\" (UniqueName: \"kubernetes.io/projected/c98cc144-3322-4f83-b571-9772b5a63d49-kube-api-access-kqv9w\") pod \"network-check-target-bkqjt\" (UID: \"c98cc144-3322-4f83-b571-9772b5a63d49\") " pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:34:45.784180 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:45.783722    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 16:34:45.784180 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:45.783740    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 16:34:45.784180 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:45.783753    2582 projected.go:196] Error preparing data for projected volume kube-api-access-kqv9w for pod openshift-network-diagnostics/network-check-target-bkqjt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 16:34:45.784180 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:45.783809    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c98cc144-3322-4f83-b571-9772b5a63d49-kube-api-access-kqv9w podName:c98cc144-3322-4f83-b571-9772b5a63d49 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:49.783790984 +0000 UTC m=+9.124756063 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kqv9w" (UniqueName: "kubernetes.io/projected/c98cc144-3322-4f83-b571-9772b5a63d49-kube-api-access-kqv9w") pod "network-check-target-bkqjt" (UID: "c98cc144-3322-4f83-b571-9772b5a63d49") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 16:34:46.211540 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:46.211505    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:34:46.211725 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:46.211643    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtkl" podUID="8eb2abc0-b610-4f3b-b6dc-02131ad29e35"
Jul 08 16:34:46.212041 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:46.212023    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:34:46.212163 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:46.212126    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkqjt" podUID="c98cc144-3322-4f83-b571-9772b5a63d49"
Jul 08 16:34:46.212220 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:46.212176    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:34:46.212268 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:46.212256    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z82nm" podUID="2796ef11-ffb4-4bff-b73b-38d727eb45f2"
Jul 08 16:34:48.212098 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:48.211553    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:34:48.212098 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:48.211593    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:34:48.212098 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:48.211697    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtkl" podUID="8eb2abc0-b610-4f3b-b6dc-02131ad29e35"
Jul 08 16:34:48.212098 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:48.211763    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z82nm" podUID="2796ef11-ffb4-4bff-b73b-38d727eb45f2"
Jul 08 16:34:48.212098 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:48.211823    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:34:48.212098 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:48.211887    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkqjt" podUID="c98cc144-3322-4f83-b571-9772b5a63d49"
Jul 08 16:34:49.718267 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:49.718225    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-original-pull-secret\") pod \"global-pull-secret-syncer-4vtkl\" (UID: \"8eb2abc0-b610-4f3b-b6dc-02131ad29e35\") " pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:34:49.718756 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:49.718326    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2796ef11-ffb4-4bff-b73b-38d727eb45f2-metrics-certs\") pod \"network-metrics-daemon-z82nm\" (UID: \"2796ef11-ffb4-4bff-b73b-38d727eb45f2\") " pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:34:49.718756 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:49.718481    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 16:34:49.718756 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:49.718544    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2796ef11-ffb4-4bff-b73b-38d727eb45f2-metrics-certs podName:2796ef11-ffb4-4bff-b73b-38d727eb45f2 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:57.718524755 +0000 UTC m=+17.059489824 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2796ef11-ffb4-4bff-b73b-38d727eb45f2-metrics-certs") pod "network-metrics-daemon-z82nm" (UID: "2796ef11-ffb4-4bff-b73b-38d727eb45f2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 16:34:49.719026 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:49.718944    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 16:34:49.719026 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:49.718992    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-original-pull-secret podName:8eb2abc0-b610-4f3b-b6dc-02131ad29e35 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:57.718976943 +0000 UTC m=+17.059942007 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-original-pull-secret") pod "global-pull-secret-syncer-4vtkl" (UID: "8eb2abc0-b610-4f3b-b6dc-02131ad29e35") : object "kube-system"/"original-pull-secret" not registered
Jul 08 16:34:49.819432 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:49.819393    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqv9w\" (UniqueName: \"kubernetes.io/projected/c98cc144-3322-4f83-b571-9772b5a63d49-kube-api-access-kqv9w\") pod \"network-check-target-bkqjt\" (UID: \"c98cc144-3322-4f83-b571-9772b5a63d49\") " pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:34:49.819613 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:49.819579    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 16:34:49.819613 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:49.819596    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 16:34:49.819613 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:49.819605    2582 projected.go:196] Error preparing data for projected volume kube-api-access-kqv9w for pod openshift-network-diagnostics/network-check-target-bkqjt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 16:34:49.819776 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:49.819661    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c98cc144-3322-4f83-b571-9772b5a63d49-kube-api-access-kqv9w podName:c98cc144-3322-4f83-b571-9772b5a63d49 nodeName:}" failed. No retries permitted until 2026-07-08 16:34:57.819643379 +0000 UTC m=+17.160608438 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kqv9w" (UniqueName: "kubernetes.io/projected/c98cc144-3322-4f83-b571-9772b5a63d49-kube-api-access-kqv9w") pod "network-check-target-bkqjt" (UID: "c98cc144-3322-4f83-b571-9772b5a63d49") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 16:34:50.212347 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:50.212241    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:34:50.212347 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:50.212263    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:34:50.212569 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:50.212393    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkqjt" podUID="c98cc144-3322-4f83-b571-9772b5a63d49"
Jul 08 16:34:50.212792 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:50.212743    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtkl" podUID="8eb2abc0-b610-4f3b-b6dc-02131ad29e35"
Jul 08 16:34:50.212792 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:50.212779    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:34:50.212963 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:50.212901    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z82nm" podUID="2796ef11-ffb4-4bff-b73b-38d727eb45f2"
Jul 08 16:34:52.211450 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:52.211422    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:34:52.211833 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:52.211453    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:34:52.211833 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:52.211453    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:34:52.211833 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:52.211550    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtkl" podUID="8eb2abc0-b610-4f3b-b6dc-02131ad29e35"
Jul 08 16:34:52.211833 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:52.211642    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z82nm" podUID="2796ef11-ffb4-4bff-b73b-38d727eb45f2"
Jul 08 16:34:52.211833 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:52.211699    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkqjt" podUID="c98cc144-3322-4f83-b571-9772b5a63d49"
Jul 08 16:34:54.211506 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:54.211467    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:34:54.211951 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:54.211573    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:34:54.211951 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:54.211582    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtkl" podUID="8eb2abc0-b610-4f3b-b6dc-02131ad29e35"
Jul 08 16:34:54.211951 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:54.211608    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:34:54.211951 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:54.211709    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkqjt" podUID="c98cc144-3322-4f83-b571-9772b5a63d49"
Jul 08 16:34:54.211951 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:54.211811    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z82nm" podUID="2796ef11-ffb4-4bff-b73b-38d727eb45f2"
Jul 08 16:34:56.211805 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:56.211759    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:34:56.212294 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:56.211884    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtkl" podUID="8eb2abc0-b610-4f3b-b6dc-02131ad29e35"
Jul 08 16:34:56.212294 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:56.212278    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:34:56.212465 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:56.212386    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z82nm" podUID="2796ef11-ffb4-4bff-b73b-38d727eb45f2"
Jul 08 16:34:56.212465 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:56.212442    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:34:56.212574 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:56.212503    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkqjt" podUID="c98cc144-3322-4f83-b571-9772b5a63d49"
Jul 08 16:34:57.782014 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:57.781972    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-original-pull-secret\") pod \"global-pull-secret-syncer-4vtkl\" (UID: \"8eb2abc0-b610-4f3b-b6dc-02131ad29e35\") " pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:34:57.782461 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:57.782044    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2796ef11-ffb4-4bff-b73b-38d727eb45f2-metrics-certs\") pod \"network-metrics-daemon-z82nm\" (UID: \"2796ef11-ffb4-4bff-b73b-38d727eb45f2\") " pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:34:57.782461 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:57.782141    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 16:34:57.782461 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:57.782159    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 16:34:57.782461 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:57.782213    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-original-pull-secret podName:8eb2abc0-b610-4f3b-b6dc-02131ad29e35 nodeName:}" failed. No retries permitted until 2026-07-08 16:35:13.782191538 +0000 UTC m=+33.123156600 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-original-pull-secret") pod "global-pull-secret-syncer-4vtkl" (UID: "8eb2abc0-b610-4f3b-b6dc-02131ad29e35") : object "kube-system"/"original-pull-secret" not registered
Jul 08 16:34:57.782461 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:57.782233    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2796ef11-ffb4-4bff-b73b-38d727eb45f2-metrics-certs podName:2796ef11-ffb4-4bff-b73b-38d727eb45f2 nodeName:}" failed. No retries permitted until 2026-07-08 16:35:13.782222884 +0000 UTC m=+33.123187944 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2796ef11-ffb4-4bff-b73b-38d727eb45f2-metrics-certs") pod "network-metrics-daemon-z82nm" (UID: "2796ef11-ffb4-4bff-b73b-38d727eb45f2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 16:34:57.882949 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:57.882909    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqv9w\" (UniqueName: \"kubernetes.io/projected/c98cc144-3322-4f83-b571-9772b5a63d49-kube-api-access-kqv9w\") pod \"network-check-target-bkqjt\" (UID: \"c98cc144-3322-4f83-b571-9772b5a63d49\") " pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:34:57.883136 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:57.883064    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 16:34:57.883136 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:57.883091    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 16:34:57.883136 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:57.883104    2582 projected.go:196] Error preparing data for projected volume kube-api-access-kqv9w for pod openshift-network-diagnostics/network-check-target-bkqjt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 16:34:57.883283 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:57.883171    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c98cc144-3322-4f83-b571-9772b5a63d49-kube-api-access-kqv9w podName:c98cc144-3322-4f83-b571-9772b5a63d49 nodeName:}" failed. No retries permitted until 2026-07-08 16:35:13.8831525 +0000 UTC m=+33.224117557 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kqv9w" (UniqueName: "kubernetes.io/projected/c98cc144-3322-4f83-b571-9772b5a63d49-kube-api-access-kqv9w") pod "network-check-target-bkqjt" (UID: "c98cc144-3322-4f83-b571-9772b5a63d49") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 16:34:58.212017 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:58.211835    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:34:58.212193 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:58.211883    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:34:58.212193 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:58.212112    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkqjt" podUID="c98cc144-3322-4f83-b571-9772b5a63d49"
Jul 08 16:34:58.212193 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:34:58.211889    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:34:58.212193 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:58.212163    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtkl" podUID="8eb2abc0-b610-4f3b-b6dc-02131ad29e35"
Jul 08 16:34:58.212422 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:34:58.212245    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z82nm" podUID="2796ef11-ffb4-4bff-b73b-38d727eb45f2"
Jul 08 16:35:00.211957 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:00.211923    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:35:00.212485 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:00.211928    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:35:00.212485 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:00.212063    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:35:00.212485 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:00.212058    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z82nm" podUID="2796ef11-ffb4-4bff-b73b-38d727eb45f2"
Jul 08 16:35:00.212485 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:00.212141    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtkl" podUID="8eb2abc0-b610-4f3b-b6dc-02131ad29e35"
Jul 08 16:35:00.212485 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:00.212204    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkqjt" podUID="c98cc144-3322-4f83-b571-9772b5a63d49"
Jul 08 16:35:02.212000 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:02.211957    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:35:02.212000 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:02.211980    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:35:02.212615 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:02.211957    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:35:02.212615 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:02.212108    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z82nm" podUID="2796ef11-ffb4-4bff-b73b-38d727eb45f2"
Jul 08 16:35:02.212615 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:02.212206    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtkl" podUID="8eb2abc0-b610-4f3b-b6dc-02131ad29e35"
Jul 08 16:35:02.212615 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:02.212292    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkqjt" podUID="c98cc144-3322-4f83-b571-9772b5a63d49"
Jul 08 16:35:03.392214 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:03.392174    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jgp2d" event={"ID":"a2b65945-693a-42f8-9955-f484a3f96d10","Type":"ContainerStarted","Data":"8f2219d8a68ad7e3383c5a965301cb852e0c3ca502c1fb1b651c1846d3a23dc1"}
Jul 08 16:35:03.394259 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:03.394228    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-237.ec2.internal" event={"ID":"81aa7add2915dbb7aae72d3a67b06d72","Type":"ContainerStarted","Data":"b89687236e35047b8ec9b1335b619e57c7e6ea47b0c966d4ae790b4e17dfa78b"}
Jul 08 16:35:03.395944 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:03.395913    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-p46xh" event={"ID":"782cdc3e-e8c4-4d37-a22e-832fac199082","Type":"ContainerStarted","Data":"1adf76c232592450bd76db001f23469dedc32124919e5f1d175b29decf2096e8"}
Jul 08 16:35:03.397596 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:03.397570    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr" event={"ID":"19df7751-4b62-4ae5-92a6-0bbda11fec85","Type":"ContainerStarted","Data":"313a57bb0f94c04781e727983f80236b7a2c3701f95dbdfc11e2655e4c5a0e73"}
Jul 08 16:35:03.411866 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:03.411808    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jgp2d" podStartSLOduration=2.198116765 podStartE2EDuration="22.411791441s" totalImagesPullingTime="20.213674676s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:03.411130003 +0000 UTC m=+22.752095081" watchObservedRunningTime="2026-07-08 16:35:03.411791441 +0000 UTC m=+22.752756519"
Jul 08 16:35:03.449916 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:03.449784    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-237.ec2.internal" podStartSLOduration=21.449765817 podStartE2EDuration="21.449765817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:34:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:03.431415759 +0000 UTC m=+22.772380923" watchObservedRunningTime="2026-07-08 16:35:03.449765817 +0000 UTC m=+22.790730938"
Jul 08 16:35:03.450080 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:03.449912    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-p46xh" podStartSLOduration=3.084432448 podStartE2EDuration="22.449907607s" totalImagesPullingTime="19.365475159s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:03.449424478 +0000 UTC m=+22.790389559" watchObservedRunningTime="2026-07-08 16:35:03.449907607 +0000 UTC m=+22.790872684"
Jul 08 16:35:04.211682 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:04.211505    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:35:04.211842 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:04.211555    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:35:04.211842 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:04.211752    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtkl" podUID="8eb2abc0-b610-4f3b-b6dc-02131ad29e35"
Jul 08 16:35:04.211955 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:04.211555    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:35:04.211955 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:04.211925    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkqjt" podUID="c98cc144-3322-4f83-b571-9772b5a63d49"
Jul 08 16:35:04.212027 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:04.211959    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z82nm" podUID="2796ef11-ffb4-4bff-b73b-38d727eb45f2"
Jul 08 16:35:04.402601 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:04.402528    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kvhpr_19df7751-4b62-4ae5-92a6-0bbda11fec85/ovn-acl-logging/0.log"
Jul 08 16:35:04.403406 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:04.402875    2582 generic.go:356] "Generic (PLEG): container finished" podID="19df7751-4b62-4ae5-92a6-0bbda11fec85" containerID="dba2715ae7807f9e045d61b3944997eb6172d58aee710610b7990d898bdec563" exitCode=1
Jul 08 16:35:04.403406 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:04.402942    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr" event={"ID":"19df7751-4b62-4ae5-92a6-0bbda11fec85","Type":"ContainerStarted","Data":"9e6f1ab3e52069423e5408c8eeb8affb7efc0a3dda4e80922351bcb3ba66962e"}
Jul 08 16:35:04.403406 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:04.403230    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr" event={"ID":"19df7751-4b62-4ae5-92a6-0bbda11fec85","Type":"ContainerStarted","Data":"b816a0d74086b80b9603a250c32675e7fdcef924e313d4300917632a574c8ec7"}
Jul 08 16:35:04.403406 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:04.403280    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr" event={"ID":"19df7751-4b62-4ae5-92a6-0bbda11fec85","Type":"ContainerStarted","Data":"da57ff0890733a5b761748ce6dde7b4474d5fdd7b857140265c83143fd3b1aa1"}
Jul 08 16:35:04.403406 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:04.403354    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr" event={"ID":"19df7751-4b62-4ae5-92a6-0bbda11fec85","Type":"ContainerStarted","Data":"72a11f364246815faf4a95ff56cb51f26a6a5698c40ff5f4c767141af3c34727"}
Jul 08 16:35:04.403406 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:04.403398    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr" event={"ID":"19df7751-4b62-4ae5-92a6-0bbda11fec85","Type":"ContainerDied","Data":"dba2715ae7807f9e045d61b3944997eb6172d58aee710610b7990d898bdec563"}
Jul 08 16:35:04.404508 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:04.404476    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8hh26" event={"ID":"a12a9aeb-4a05-41b1-9f81-dc6dc476fda8","Type":"ContainerStarted","Data":"3992934fa31fdae78304bbd7fc17e65f7656742df7724c4ad7a06cefb687c399"}
Jul 08 16:35:04.405991 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:04.405966    2582 generic.go:356] "Generic (PLEG): container finished" podID="d9ddf360-c5ae-4ef7-a947-59405bf4e8db" containerID="6fd8f8c5a76d17939778764332edbdaac6e95885d061d87d730409c9e69673df" exitCode=0
Jul 08 16:35:04.406074 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:04.406043    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mw6h7" event={"ID":"d9ddf360-c5ae-4ef7-a947-59405bf4e8db","Type":"ContainerDied","Data":"6fd8f8c5a76d17939778764332edbdaac6e95885d061d87d730409c9e69673df"}
Jul 08 16:35:04.407766 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:04.407579    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bpx78" event={"ID":"d45b9a7f-3c63-4f52-863f-5d128fd07d93","Type":"ContainerStarted","Data":"a524ad57ffb30838f66c3ac30e0bd1870ae573cffe4ad6985ece14ccb96e37ce"}
Jul 08 16:35:04.408906 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:04.408886    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5pm" event={"ID":"6c79a3b7-b6b3-4264-8c73-36fc32701b6e","Type":"ContainerStarted","Data":"cec20d8f61970c978e85b84fac5ea0e050c1c50c88438fcd983d44e3fa3b4185"}
Jul 08 16:35:04.414038 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:04.414012    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pq4sj" event={"ID":"ecc650f2-27aa-4300-b61a-8328a41d599d","Type":"ContainerStarted","Data":"c7b4c2a71c96301948aa59cd4dcf0e88c2bfdd8d0cdff84748c9dca8b4eb5715"}
Jul 08 16:35:04.415355 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:04.415301    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-957gg" event={"ID":"f877b6b1-282c-4c50-95d6-530371d41a15","Type":"ContainerStarted","Data":"35231676d62da53753b79debc9f5eb169091faed6d4a09d94332a3f4e7a8d73c"}
Jul 08 16:35:04.416714 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:04.416693    2582 generic.go:356] "Generic (PLEG): container finished" podID="9d4667db5036e322bfdaf5880d0290f1" containerID="824e3a37c5e3a213939d66c85e18921f5832fac1f7da9bbe615f14581692acae" exitCode=0
Jul 08 16:35:04.416815 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:04.416793    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-237.ec2.internal" event={"ID":"9d4667db5036e322bfdaf5880d0290f1","Type":"ContainerDied","Data":"824e3a37c5e3a213939d66c85e18921f5832fac1f7da9bbe615f14581692acae"}
Jul 08 16:35:04.420206 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:04.420171    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8hh26" podStartSLOduration=3.302607753 podStartE2EDuration="23.420159552s" totalImagesPullingTime="20.117551799s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:04.419725326 +0000 UTC m=+23.760690405" watchObservedRunningTime="2026-07-08 16:35:04.420159552 +0000 UTC m=+23.761124629"
Jul 08 16:35:04.471086 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:04.471029    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bpx78" podStartSLOduration=3.318278056 podStartE2EDuration="23.471010444s" totalImagesPullingTime="20.152732388s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:04.470427191 +0000 UTC m=+23.811392269" watchObservedRunningTime="2026-07-08 16:35:04.471010444 +0000 UTC m=+23.811975521"
Jul 08 16:35:04.508378 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:04.508332    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-957gg" podStartSLOduration=3.3726681530000002 podStartE2EDuration="23.508296058s" totalImagesPullingTime="20.135627905s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:04.489157454 +0000 UTC m=+23.830122533" watchObservedRunningTime="2026-07-08 16:35:04.508296058 +0000 UTC m=+23.849261136"
Jul 08 16:35:05.359955 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:05.359925    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-957gg"
Jul 08 16:35:05.360576 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:05.360555    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-957gg"
Jul 08 16:35:05.379301 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:05.379189    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-pq4sj" podStartSLOduration=4.258012174 podStartE2EDuration="24.379167521s" totalImagesPullingTime="20.121155347s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:04.508579233 +0000 UTC m=+23.849544325" watchObservedRunningTime="2026-07-08 16:35:05.379167521 +0000 UTC m=+24.720132599"
Jul 08 16:35:05.416306 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:05.416280    2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 08 16:35:05.421518 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:05.421482    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-237.ec2.internal" event={"ID":"9d4667db5036e322bfdaf5880d0290f1","Type":"ContainerStarted","Data":"dd2d2ced76ccc1f432c0497076ef23a1889c18f1447c3883621b81e5f4ad2602"}
Jul 08 16:35:05.423767 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:05.423366    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5pm" event={"ID":"6c79a3b7-b6b3-4264-8c73-36fc32701b6e","Type":"ContainerStarted","Data":"ba2ea1fb82efb2cd7463ca772d947879a14d130d404061304c6b746577e11365"}
Jul 08 16:35:05.423767 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:05.423734    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-957gg"
Jul 08 16:35:05.424201 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:05.424181    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-957gg"
Jul 08 16:35:05.446732 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:05.446679    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-237.ec2.internal" podStartSLOduration=23.446660021 podStartE2EDuration="23.446660021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:34:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:05.446095641 +0000 UTC m=+24.787060720" watchObservedRunningTime="2026-07-08 16:35:05.446660021 +0000 UTC m=+24.787625100"
Jul 08 16:35:06.109613 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:06.109251    2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T16:35:05.416302936Z","UUID":"d4f8460d-8748-4929-9b3c-59d6c4fd06a4","Handler":null,"Name":"","Endpoint":""}
Jul 08 16:35:06.111102 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:06.111080    2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 08 16:35:06.111213 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:06.111123    2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 08 16:35:06.211866 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:06.211831    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:35:06.212037 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:06.211850    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:35:06.212037 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:06.211964    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z82nm" podUID="2796ef11-ffb4-4bff-b73b-38d727eb45f2"
Jul 08 16:35:06.212037 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:06.212030    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:35:06.212176 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:06.212031    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkqjt" podUID="c98cc144-3322-4f83-b571-9772b5a63d49"
Jul 08 16:35:06.212176 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:06.212099    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtkl" podUID="8eb2abc0-b610-4f3b-b6dc-02131ad29e35"
Jul 08 16:35:06.429732 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:06.429663    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kvhpr_19df7751-4b62-4ae5-92a6-0bbda11fec85/ovn-acl-logging/0.log"
Jul 08 16:35:06.430109 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:06.430066    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr" event={"ID":"19df7751-4b62-4ae5-92a6-0bbda11fec85","Type":"ContainerStarted","Data":"8bcfe9f1602dadcfba0fa62f89c4d56880f0daa14bc737096127afaec21752a3"}
Jul 08 16:35:07.436649 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:07.436610    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5pm" event={"ID":"6c79a3b7-b6b3-4264-8c73-36fc32701b6e","Type":"ContainerStarted","Data":"5ef07fbe47eaa9804ab9b8e21f7594da2df9a1fc91ba4210f220152b2b8f1cc5"}
Jul 08 16:35:07.457090 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:07.457038    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ns5pm" podStartSLOduration=4.247279143 podStartE2EDuration="26.457021694s" totalImagesPullingTime="22.209742551s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:34:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:07.456950564 +0000 UTC m=+26.797915642" watchObservedRunningTime="2026-07-08 16:35:07.457021694 +0000 UTC m=+26.797986771"
Jul 08 16:35:08.211655 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:08.211613    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:35:08.211820 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:08.211744    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtkl" podUID="8eb2abc0-b610-4f3b-b6dc-02131ad29e35"
Jul 08 16:35:08.212175 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:08.212148    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:35:08.212300 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:08.212261    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z82nm" podUID="2796ef11-ffb4-4bff-b73b-38d727eb45f2"
Jul 08 16:35:08.212382 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:08.212344    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:35:08.212435 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:08.212407    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkqjt" podUID="c98cc144-3322-4f83-b571-9772b5a63d49"
Jul 08 16:35:09.445588 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:09.445391    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kvhpr_19df7751-4b62-4ae5-92a6-0bbda11fec85/ovn-acl-logging/0.log"
Jul 08 16:35:09.446232 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:09.445898    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr" event={"ID":"19df7751-4b62-4ae5-92a6-0bbda11fec85","Type":"ContainerStarted","Data":"cd0c24223a51cd01b178fddcb8615eb0b920e31002a9ee656920b2522598ff7b"}
Jul 08 16:35:09.446232 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:09.446169    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:35:09.446232 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:09.446229    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:35:09.446496 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:09.446372    2582 scope.go:117] "RemoveContainer" containerID="dba2715ae7807f9e045d61b3944997eb6172d58aee710610b7990d898bdec563"
Jul 08 16:35:09.463621 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:09.463598    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:35:10.211679 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:10.211654    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:35:10.211809 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:10.211654    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:35:10.211809 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:10.211757    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtkl" podUID="8eb2abc0-b610-4f3b-b6dc-02131ad29e35"
Jul 08 16:35:10.211876 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:10.211656    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:35:10.211876 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:10.211849    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z82nm" podUID="2796ef11-ffb4-4bff-b73b-38d727eb45f2"
Jul 08 16:35:10.211940 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:10.211904    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkqjt" podUID="c98cc144-3322-4f83-b571-9772b5a63d49"
Jul 08 16:35:10.458002 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:10.457917    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kvhpr_19df7751-4b62-4ae5-92a6-0bbda11fec85/ovn-acl-logging/0.log"
Jul 08 16:35:10.458635 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:10.458445    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr" event={"ID":"19df7751-4b62-4ae5-92a6-0bbda11fec85","Type":"ContainerStarted","Data":"d110e36ac8d60d73bb39398eb8113c154ce17d582962c637597e159f31e636a8"}
Jul 08 16:35:10.458902 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:10.458874    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:35:10.460616 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:10.460588    2582 generic.go:356] "Generic (PLEG): container finished" podID="d9ddf360-c5ae-4ef7-a947-59405bf4e8db" containerID="8156f9ef77d730932a5318fe1af365ff4f18c50f132386dce446f97f47dba45c" exitCode=0
Jul 08 16:35:10.460737 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:10.460627    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mw6h7" event={"ID":"d9ddf360-c5ae-4ef7-a947-59405bf4e8db","Type":"ContainerDied","Data":"8156f9ef77d730932a5318fe1af365ff4f18c50f132386dce446f97f47dba45c"}
Jul 08 16:35:10.479101 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:10.479070    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:35:10.498620 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:10.498558    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr" podStartSLOduration=8.791556141000001 podStartE2EDuration="29.498540586s" totalImagesPullingTime="20.706984445s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:10.498511101 +0000 UTC m=+29.839476180" watchObservedRunningTime="2026-07-08 16:35:10.498540586 +0000 UTC m=+29.839505668"
Jul 08 16:35:11.471502 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:11.471273    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4vtkl"]
Jul 08 16:35:11.471925 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:11.471603    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:35:11.471925 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:11.471745    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtkl" podUID="8eb2abc0-b610-4f3b-b6dc-02131ad29e35"
Jul 08 16:35:11.472135 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:11.472111    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bkqjt"]
Jul 08 16:35:11.472224 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:11.472217    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:35:11.472412 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:11.472355    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkqjt" podUID="c98cc144-3322-4f83-b571-9772b5a63d49"
Jul 08 16:35:11.472884 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:11.472860    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-z82nm"]
Jul 08 16:35:11.472978 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:11.472963    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:35:11.473079 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:11.473059    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z82nm" podUID="2796ef11-ffb4-4bff-b73b-38d727eb45f2"
Jul 08 16:35:12.466861 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:12.466823    2582 generic.go:356] "Generic (PLEG): container finished" podID="d9ddf360-c5ae-4ef7-a947-59405bf4e8db" containerID="e94d716e9df155d7c8fc1317c4df7738adf7f10a2653695c0d26c8da0cbaba44" exitCode=0
Jul 08 16:35:12.467041 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:12.466918    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mw6h7" event={"ID":"d9ddf360-c5ae-4ef7-a947-59405bf4e8db","Type":"ContainerDied","Data":"e94d716e9df155d7c8fc1317c4df7738adf7f10a2653695c0d26c8da0cbaba44"}
Jul 08 16:35:13.211591 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:13.211556    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:35:13.212038 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:13.211602    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:35:13.212038 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:13.211700    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtkl" podUID="8eb2abc0-b610-4f3b-b6dc-02131ad29e35"
Jul 08 16:35:13.212038 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:13.211768    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:35:13.212038 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:13.211773    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z82nm" podUID="2796ef11-ffb4-4bff-b73b-38d727eb45f2"
Jul 08 16:35:13.212038 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:13.211834    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkqjt" podUID="c98cc144-3322-4f83-b571-9772b5a63d49"
Jul 08 16:35:13.471460 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:13.471428    2582 generic.go:356] "Generic (PLEG): container finished" podID="d9ddf360-c5ae-4ef7-a947-59405bf4e8db" containerID="ab11abb89c3c6eb3477796c513c054d9817e80c496707866dc72e567a0d26548" exitCode=0
Jul 08 16:35:13.471609 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:13.471481    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mw6h7" event={"ID":"d9ddf360-c5ae-4ef7-a947-59405bf4e8db","Type":"ContainerDied","Data":"ab11abb89c3c6eb3477796c513c054d9817e80c496707866dc72e567a0d26548"}
Jul 08 16:35:13.556351 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:13.556250    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pq4sj_ecc650f2-27aa-4300-b61a-8328a41d599d/dns-node-resolver/0.log"
Jul 08 16:35:13.804753 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:13.804721    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-original-pull-secret\") pod \"global-pull-secret-syncer-4vtkl\" (UID: \"8eb2abc0-b610-4f3b-b6dc-02131ad29e35\") " pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:35:13.804918 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:13.804769    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2796ef11-ffb4-4bff-b73b-38d727eb45f2-metrics-certs\") pod \"network-metrics-daemon-z82nm\" (UID: \"2796ef11-ffb4-4bff-b73b-38d727eb45f2\") " pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:35:13.804918 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:13.804884    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 16:35:13.804918 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:13.804884    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 16:35:13.805013 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:13.804936    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2796ef11-ffb4-4bff-b73b-38d727eb45f2-metrics-certs podName:2796ef11-ffb4-4bff-b73b-38d727eb45f2 nodeName:}" failed. No retries permitted until 2026-07-08 16:35:45.804923207 +0000 UTC m=+65.145888267 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2796ef11-ffb4-4bff-b73b-38d727eb45f2-metrics-certs") pod "network-metrics-daemon-z82nm" (UID: "2796ef11-ffb4-4bff-b73b-38d727eb45f2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 16:35:13.805013 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:13.804948    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-original-pull-secret podName:8eb2abc0-b610-4f3b-b6dc-02131ad29e35 nodeName:}" failed. No retries permitted until 2026-07-08 16:35:45.804942933 +0000 UTC m=+65.145907989 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-original-pull-secret") pod "global-pull-secret-syncer-4vtkl" (UID: "8eb2abc0-b610-4f3b-b6dc-02131ad29e35") : object "kube-system"/"original-pull-secret" not registered
Jul 08 16:35:13.905657 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:13.905580    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqv9w\" (UniqueName: \"kubernetes.io/projected/c98cc144-3322-4f83-b571-9772b5a63d49-kube-api-access-kqv9w\") pod \"network-check-target-bkqjt\" (UID: \"c98cc144-3322-4f83-b571-9772b5a63d49\") " pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:35:13.905797 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:13.905736    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 16:35:13.905797 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:13.905755    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 16:35:13.905797 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:13.905763    2582 projected.go:196] Error preparing data for projected volume kube-api-access-kqv9w for pod openshift-network-diagnostics/network-check-target-bkqjt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 16:35:13.905891 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:13.905813    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c98cc144-3322-4f83-b571-9772b5a63d49-kube-api-access-kqv9w podName:c98cc144-3322-4f83-b571-9772b5a63d49 nodeName:}" failed. No retries permitted until 2026-07-08 16:35:45.905800245 +0000 UTC m=+65.246765301 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kqv9w" (UniqueName: "kubernetes.io/projected/c98cc144-3322-4f83-b571-9772b5a63d49-kube-api-access-kqv9w") pod "network-check-target-bkqjt" (UID: "c98cc144-3322-4f83-b571-9772b5a63d49") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 16:35:14.733160 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:14.732989    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bpx78_d45b9a7f-3c63-4f52-863f-5d128fd07d93/node-ca/0.log"
Jul 08 16:35:15.211334 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:15.211277    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:35:15.211524 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:15.211277    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:35:15.211524 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:15.211442    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:35:15.211524 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:15.211430    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bkqjt" podUID="c98cc144-3322-4f83-b571-9772b5a63d49"
Jul 08 16:35:15.211683 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:15.211572    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z82nm" podUID="2796ef11-ffb4-4bff-b73b-38d727eb45f2"
Jul 08 16:35:15.211683 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:15.211663    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtkl" podUID="8eb2abc0-b610-4f3b-b6dc-02131ad29e35"
Jul 08 16:35:16.917413 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:16.917380    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-237.ec2.internal" event="NodeReady"
Jul 08 16:35:16.917863 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:16.917562    2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 08 16:35:17.215455 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:17.215381    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:35:17.215605 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:17.215397    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:35:17.215605 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:17.215555    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:35:17.218722 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:17.218699    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 08 16:35:17.219558 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:17.219538    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 08 16:35:17.219784 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:17.219770    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kw94b\""
Jul 08 16:35:17.219856 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:17.219793    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 08 16:35:17.219856 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:17.219832    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 08 16:35:17.220014 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:17.219957    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pp6cc\""
Jul 08 16:35:18.486412 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:18.486375    2582 generic.go:356] "Generic (PLEG): container finished" podID="d9ddf360-c5ae-4ef7-a947-59405bf4e8db" containerID="b1e000cd04f6fcfd172ad44ebb65a01acec8bc56ff06d574c07e1343228fa56b" exitCode=0
Jul 08 16:35:18.486945 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:18.486429    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mw6h7" event={"ID":"d9ddf360-c5ae-4ef7-a947-59405bf4e8db","Type":"ContainerDied","Data":"b1e000cd04f6fcfd172ad44ebb65a01acec8bc56ff06d574c07e1343228fa56b"}
Jul 08 16:35:19.491528 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:19.491496    2582 generic.go:356] "Generic (PLEG): container finished" podID="d9ddf360-c5ae-4ef7-a947-59405bf4e8db" containerID="4cb212f5b6e74af619eb312a1e62f492d4d8afd70f44c9ea513bcfd0787da08b" exitCode=0
Jul 08 16:35:19.491528 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:19.491534    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mw6h7" event={"ID":"d9ddf360-c5ae-4ef7-a947-59405bf4e8db","Type":"ContainerDied","Data":"4cb212f5b6e74af619eb312a1e62f492d4d8afd70f44c9ea513bcfd0787da08b"}
Jul 08 16:35:20.497289 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:20.497256    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mw6h7" event={"ID":"d9ddf360-c5ae-4ef7-a947-59405bf4e8db","Type":"ContainerStarted","Data":"0c36c57f6ac6aaacfbc4d7daee786e9d4af820ccc9be0fde4093579403abafe2"}
Jul 08 16:35:20.525507 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:20.525449    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mw6h7" podStartSLOduration=7.753975463 podStartE2EDuration="39.525433265s" totalImagesPullingTime="31.771457802s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:34:41 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:20.524135411 +0000 UTC m=+39.865100490" watchObservedRunningTime="2026-07-08 16:35:20.525433265 +0000 UTC m=+39.866398343"
Jul 08 16:35:37.296129 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.296089    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-w69bj"]
Jul 08 16:35:37.298100 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.298080    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-w69bj"
Jul 08 16:35:37.303435 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.303410    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 08 16:35:37.303570 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.303418    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 08 16:35:37.304000 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.303975    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2m42c\""
Jul 08 16:35:37.304460 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.303977    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 08 16:35:37.304861 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.304833    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 08 16:35:37.305237 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.305221    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tpllq"]
Jul 08 16:35:37.306993 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.306977    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tpllq"
Jul 08 16:35:37.310165 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.310140    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ztnq7\""
Jul 08 16:35:37.310681 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.310662    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-w69bj"]
Jul 08 16:35:37.310766 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.310705    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 08 16:35:37.310766 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.310705    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 08 16:35:37.323564 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.323537    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tpllq"]
Jul 08 16:35:37.399474 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.399441    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wcgpv"]
Jul 08 16:35:37.401172 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.401158    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wcgpv"
Jul 08 16:35:37.403786 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.403765    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 08 16:35:37.406523 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.406500    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 08 16:35:37.406653 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.406610    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 08 16:35:37.406653 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.406649    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-6mtpn\""
Jul 08 16:35:37.415565 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.415544    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wcgpv"]
Jul 08 16:35:37.478022 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.477989    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f96e3712-16d4-4a1d-9d73-1b3214dda98c-crio-socket\") pod \"insights-runtime-extractor-w69bj\" (UID: \"f96e3712-16d4-4a1d-9d73-1b3214dda98c\") " pod="openshift-insights/insights-runtime-extractor-w69bj"
Jul 08 16:35:37.478224 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.478035    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9c7d9d9-4854-413e-8eec-d3608f5333f1-tmp-dir\") pod \"dns-default-tpllq\" (UID: \"d9c7d9d9-4854-413e-8eec-d3608f5333f1\") " pod="openshift-dns/dns-default-tpllq"
Jul 08 16:35:37.478224 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.478058    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f96e3712-16d4-4a1d-9d73-1b3214dda98c-data-volume\") pod \"insights-runtime-extractor-w69bj\" (UID: \"f96e3712-16d4-4a1d-9d73-1b3214dda98c\") " pod="openshift-insights/insights-runtime-extractor-w69bj"
Jul 08 16:35:37.478224 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.478076    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f96e3712-16d4-4a1d-9d73-1b3214dda98c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w69bj\" (UID: \"f96e3712-16d4-4a1d-9d73-1b3214dda98c\") " pod="openshift-insights/insights-runtime-extractor-w69bj"
Jul 08 16:35:37.478224 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.478130    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhvz7\" (UniqueName: \"kubernetes.io/projected/f96e3712-16d4-4a1d-9d73-1b3214dda98c-kube-api-access-zhvz7\") pod \"insights-runtime-extractor-w69bj\" (UID: \"f96e3712-16d4-4a1d-9d73-1b3214dda98c\") " pod="openshift-insights/insights-runtime-extractor-w69bj"
Jul 08 16:35:37.478224 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.478202    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f96e3712-16d4-4a1d-9d73-1b3214dda98c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w69bj\" (UID: \"f96e3712-16d4-4a1d-9d73-1b3214dda98c\") " pod="openshift-insights/insights-runtime-extractor-w69bj"
Jul 08 16:35:37.478470 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.478238    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d9c7d9d9-4854-413e-8eec-d3608f5333f1-metrics-tls\") pod \"dns-default-tpllq\" (UID: \"d9c7d9d9-4854-413e-8eec-d3608f5333f1\") " pod="openshift-dns/dns-default-tpllq"
Jul 08 16:35:37.478470 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.478306    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d9c7d9d9-4854-413e-8eec-d3608f5333f1-config-volume\") pod \"dns-default-tpllq\" (UID: \"d9c7d9d9-4854-413e-8eec-d3608f5333f1\") " pod="openshift-dns/dns-default-tpllq"
Jul 08 16:35:37.478470 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.478343    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b42b9\" (UniqueName: \"kubernetes.io/projected/d9c7d9d9-4854-413e-8eec-d3608f5333f1-kube-api-access-b42b9\") pod \"dns-default-tpllq\" (UID: \"d9c7d9d9-4854-413e-8eec-d3608f5333f1\") " pod="openshift-dns/dns-default-tpllq"
Jul 08 16:35:37.579117 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.579016    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d9c7d9d9-4854-413e-8eec-d3608f5333f1-metrics-tls\") pod \"dns-default-tpllq\" (UID: \"d9c7d9d9-4854-413e-8eec-d3608f5333f1\") " pod="openshift-dns/dns-default-tpllq"
Jul 08 16:35:37.579117 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.579065    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d9c7d9d9-4854-413e-8eec-d3608f5333f1-config-volume\") pod \"dns-default-tpllq\" (UID: \"d9c7d9d9-4854-413e-8eec-d3608f5333f1\") " pod="openshift-dns/dns-default-tpllq"
Jul 08 16:35:37.579117 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.579081    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b42b9\" (UniqueName: \"kubernetes.io/projected/d9c7d9d9-4854-413e-8eec-d3608f5333f1-kube-api-access-b42b9\") pod \"dns-default-tpllq\" (UID: \"d9c7d9d9-4854-413e-8eec-d3608f5333f1\") " pod="openshift-dns/dns-default-tpllq"
Jul 08 16:35:37.579117 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.579107    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f96e3712-16d4-4a1d-9d73-1b3214dda98c-crio-socket\") pod \"insights-runtime-extractor-w69bj\" (UID: \"f96e3712-16d4-4a1d-9d73-1b3214dda98c\") " pod="openshift-insights/insights-runtime-extractor-w69bj"
Jul 08 16:35:37.579475 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.579153    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9c7d9d9-4854-413e-8eec-d3608f5333f1-tmp-dir\") pod \"dns-default-tpllq\" (UID: \"d9c7d9d9-4854-413e-8eec-d3608f5333f1\") " pod="openshift-dns/dns-default-tpllq"
Jul 08 16:35:37.579475 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.579187    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f96e3712-16d4-4a1d-9d73-1b3214dda98c-data-volume\") pod \"insights-runtime-extractor-w69bj\" (UID: \"f96e3712-16d4-4a1d-9d73-1b3214dda98c\") " pod="openshift-insights/insights-runtime-extractor-w69bj"
Jul 08 16:35:37.579475 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.579210    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f68ab1b-b463-44bb-87f3-1341b827b5ec-cert\") pod \"ingress-canary-wcgpv\" (UID: \"9f68ab1b-b463-44bb-87f3-1341b827b5ec\") " pod="openshift-ingress-canary/ingress-canary-wcgpv"
Jul 08 16:35:37.579475 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.579236    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f96e3712-16d4-4a1d-9d73-1b3214dda98c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w69bj\" (UID: \"f96e3712-16d4-4a1d-9d73-1b3214dda98c\") " pod="openshift-insights/insights-runtime-extractor-w69bj"
Jul 08 16:35:37.579475 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.579259    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhvz7\" (UniqueName: \"kubernetes.io/projected/f96e3712-16d4-4a1d-9d73-1b3214dda98c-kube-api-access-zhvz7\") pod \"insights-runtime-extractor-w69bj\" (UID: \"f96e3712-16d4-4a1d-9d73-1b3214dda98c\") " pod="openshift-insights/insights-runtime-extractor-w69bj"
Jul 08 16:35:37.579475 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.579259    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f96e3712-16d4-4a1d-9d73-1b3214dda98c-crio-socket\") pod \"insights-runtime-extractor-w69bj\" (UID: \"f96e3712-16d4-4a1d-9d73-1b3214dda98c\") " pod="openshift-insights/insights-runtime-extractor-w69bj"
Jul 08 16:35:37.579774 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.579497    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f96e3712-16d4-4a1d-9d73-1b3214dda98c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w69bj\" (UID: \"f96e3712-16d4-4a1d-9d73-1b3214dda98c\") " pod="openshift-insights/insights-runtime-extractor-w69bj"
Jul 08 16:35:37.579774 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.579535    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7kw6\" (UniqueName: \"kubernetes.io/projected/9f68ab1b-b463-44bb-87f3-1341b827b5ec-kube-api-access-k7kw6\") pod \"ingress-canary-wcgpv\" (UID: \"9f68ab1b-b463-44bb-87f3-1341b827b5ec\") " pod="openshift-ingress-canary/ingress-canary-wcgpv"
Jul 08 16:35:37.579774 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.579548    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f96e3712-16d4-4a1d-9d73-1b3214dda98c-data-volume\") pod \"insights-runtime-extractor-w69bj\" (UID: \"f96e3712-16d4-4a1d-9d73-1b3214dda98c\") " pod="openshift-insights/insights-runtime-extractor-w69bj"
Jul 08 16:35:37.579774 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.579635    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9c7d9d9-4854-413e-8eec-d3608f5333f1-tmp-dir\") pod \"dns-default-tpllq\" (UID: \"d9c7d9d9-4854-413e-8eec-d3608f5333f1\") " pod="openshift-dns/dns-default-tpllq"
Jul 08 16:35:37.579910 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.579775    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d9c7d9d9-4854-413e-8eec-d3608f5333f1-config-volume\") pod \"dns-default-tpllq\" (UID: \"d9c7d9d9-4854-413e-8eec-d3608f5333f1\") " pod="openshift-dns/dns-default-tpllq"
Jul 08 16:35:37.579972 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.579955    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f96e3712-16d4-4a1d-9d73-1b3214dda98c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w69bj\" (UID: \"f96e3712-16d4-4a1d-9d73-1b3214dda98c\") " pod="openshift-insights/insights-runtime-extractor-w69bj"
Jul 08 16:35:37.583416 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.583395    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d9c7d9d9-4854-413e-8eec-d3608f5333f1-metrics-tls\") pod \"dns-default-tpllq\" (UID: \"d9c7d9d9-4854-413e-8eec-d3608f5333f1\") " pod="openshift-dns/dns-default-tpllq"
Jul 08 16:35:37.583516 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.583395    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f96e3712-16d4-4a1d-9d73-1b3214dda98c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w69bj\" (UID: \"f96e3712-16d4-4a1d-9d73-1b3214dda98c\") " pod="openshift-insights/insights-runtime-extractor-w69bj"
Jul 08 16:35:37.590205 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.590170    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b42b9\" (UniqueName: \"kubernetes.io/projected/d9c7d9d9-4854-413e-8eec-d3608f5333f1-kube-api-access-b42b9\") pod \"dns-default-tpllq\" (UID: \"d9c7d9d9-4854-413e-8eec-d3608f5333f1\") " pod="openshift-dns/dns-default-tpllq"
Jul 08 16:35:37.590205 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.590194    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhvz7\" (UniqueName: \"kubernetes.io/projected/f96e3712-16d4-4a1d-9d73-1b3214dda98c-kube-api-access-zhvz7\") pod \"insights-runtime-extractor-w69bj\" (UID: \"f96e3712-16d4-4a1d-9d73-1b3214dda98c\") " pod="openshift-insights/insights-runtime-extractor-w69bj"
Jul 08 16:35:37.606587 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.606563    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-w69bj"
Jul 08 16:35:37.616454 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.616434    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tpllq"
Jul 08 16:35:37.680162 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.680102    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f68ab1b-b463-44bb-87f3-1341b827b5ec-cert\") pod \"ingress-canary-wcgpv\" (UID: \"9f68ab1b-b463-44bb-87f3-1341b827b5ec\") " pod="openshift-ingress-canary/ingress-canary-wcgpv"
Jul 08 16:35:37.680359 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.680184    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7kw6\" (UniqueName: \"kubernetes.io/projected/9f68ab1b-b463-44bb-87f3-1341b827b5ec-kube-api-access-k7kw6\") pod \"ingress-canary-wcgpv\" (UID: \"9f68ab1b-b463-44bb-87f3-1341b827b5ec\") " pod="openshift-ingress-canary/ingress-canary-wcgpv"
Jul 08 16:35:37.683668 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.683609    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f68ab1b-b463-44bb-87f3-1341b827b5ec-cert\") pod \"ingress-canary-wcgpv\" (UID: \"9f68ab1b-b463-44bb-87f3-1341b827b5ec\") " pod="openshift-ingress-canary/ingress-canary-wcgpv"
Jul 08 16:35:37.690240 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.690176    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7kw6\" (UniqueName: \"kubernetes.io/projected/9f68ab1b-b463-44bb-87f3-1341b827b5ec-kube-api-access-k7kw6\") pod \"ingress-canary-wcgpv\" (UID: \"9f68ab1b-b463-44bb-87f3-1341b827b5ec\") " pod="openshift-ingress-canary/ingress-canary-wcgpv"
Jul 08 16:35:37.709155 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.709127    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wcgpv"
Jul 08 16:35:37.786519 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:35:37.786476    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf96e3712_16d4_4a1d_9d73_1b3214dda98c.slice/crio-d69a964ec232f5dc6cc581852c23ddcd5bce2902a3c8f1996901b5887935821e WatchSource:0}: Error finding container d69a964ec232f5dc6cc581852c23ddcd5bce2902a3c8f1996901b5887935821e: Status 404 returned error can't find the container with id d69a964ec232f5dc6cc581852c23ddcd5bce2902a3c8f1996901b5887935821e
Jul 08 16:35:37.794868 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.794766    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-w69bj"]
Jul 08 16:35:37.794868 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.794834    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tpllq"]
Jul 08 16:35:37.798091 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:35:37.798068    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9c7d9d9_4854_413e_8eec_d3608f5333f1.slice/crio-913ae448a5fbe90ef6c4d1b877f979ebb19fe523dc8074603bdbbfc6219ba6ac WatchSource:0}: Error finding container 913ae448a5fbe90ef6c4d1b877f979ebb19fe523dc8074603bdbbfc6219ba6ac: Status 404 returned error can't find the container with id 913ae448a5fbe90ef6c4d1b877f979ebb19fe523dc8074603bdbbfc6219ba6ac
Jul 08 16:35:37.846904 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:37.846828    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wcgpv"]
Jul 08 16:35:37.849611 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:35:37.849587    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f68ab1b_b463_44bb_87f3_1341b827b5ec.slice/crio-8894961d1280ea065ff46783b16d5742abc76f3319eb7b82ec08e43e37f6775f WatchSource:0}: Error finding container 8894961d1280ea065ff46783b16d5742abc76f3319eb7b82ec08e43e37f6775f: Status 404 returned error can't find the container with id 8894961d1280ea065ff46783b16d5742abc76f3319eb7b82ec08e43e37f6775f
Jul 08 16:35:38.541770 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:38.541731    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wcgpv" event={"ID":"9f68ab1b-b463-44bb-87f3-1341b827b5ec","Type":"ContainerStarted","Data":"8894961d1280ea065ff46783b16d5742abc76f3319eb7b82ec08e43e37f6775f"}
Jul 08 16:35:38.543575 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:38.543540    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tpllq" event={"ID":"d9c7d9d9-4854-413e-8eec-d3608f5333f1","Type":"ContainerStarted","Data":"913ae448a5fbe90ef6c4d1b877f979ebb19fe523dc8074603bdbbfc6219ba6ac"}
Jul 08 16:35:38.546553 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:38.546525    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w69bj" event={"ID":"f96e3712-16d4-4a1d-9d73-1b3214dda98c","Type":"ContainerStarted","Data":"d69a964ec232f5dc6cc581852c23ddcd5bce2902a3c8f1996901b5887935821e"}
Jul 08 16:35:42.307157 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.307112    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-559686bd8d-9qgwd"]
Jul 08 16:35:42.314076 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.314047    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-559686bd8d-9qgwd"
Jul 08 16:35:42.317227 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.317165    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 08 16:35:42.317227 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.317185    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 08 16:35:42.317449 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.317229    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 08 16:35:42.317550 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.317532    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 08 16:35:42.317666 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.317639    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2g5q\" (UniqueName: \"kubernetes.io/projected/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-kube-api-access-p2g5q\") pod \"console-559686bd8d-9qgwd\" (UID: \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\") " pod="openshift-console/console-559686bd8d-9qgwd"
Jul 08 16:35:42.317784 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.317677    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-console-config\") pod \"console-559686bd8d-9qgwd\" (UID: \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\") " pod="openshift-console/console-559686bd8d-9qgwd"
Jul 08 16:35:42.317784 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.317763    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-oauth-serving-cert\") pod \"console-559686bd8d-9qgwd\" (UID: \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\") " pod="openshift-console/console-559686bd8d-9qgwd"
Jul 08 16:35:42.317897 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.317806    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-console-oauth-config\") pod \"console-559686bd8d-9qgwd\" (UID: \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\") " pod="openshift-console/console-559686bd8d-9qgwd"
Jul 08 16:35:42.317897 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.317864    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-service-ca\") pod \"console-559686bd8d-9qgwd\" (UID: \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\") " pod="openshift-console/console-559686bd8d-9qgwd"
Jul 08 16:35:42.317998 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.317902    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-console-serving-cert\") pod \"console-559686bd8d-9qgwd\" (UID: \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\") " pod="openshift-console/console-559686bd8d-9qgwd"
Jul 08 16:35:42.318439 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.318419    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 08 16:35:42.318439 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.318436    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 08 16:35:42.318597 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.318517    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-lmmwj\""
Jul 08 16:35:42.318597 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.318551    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 08 16:35:42.320381 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.320360    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-559686bd8d-9qgwd"]
Jul 08 16:35:42.418344 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.418295    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-service-ca\") pod \"console-559686bd8d-9qgwd\" (UID: \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\") " pod="openshift-console/console-559686bd8d-9qgwd"
Jul 08 16:35:42.418552 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.418374    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-console-serving-cert\") pod \"console-559686bd8d-9qgwd\" (UID: \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\") " pod="openshift-console/console-559686bd8d-9qgwd"
Jul 08 16:35:42.418552 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.418409    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2g5q\" (UniqueName: \"kubernetes.io/projected/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-kube-api-access-p2g5q\") pod \"console-559686bd8d-9qgwd\" (UID: \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\") " pod="openshift-console/console-559686bd8d-9qgwd"
Jul 08 16:35:42.418552 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.418433    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-console-config\") pod \"console-559686bd8d-9qgwd\" (UID: \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\") " pod="openshift-console/console-559686bd8d-9qgwd"
Jul 08 16:35:42.418552 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.418482    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-oauth-serving-cert\") pod \"console-559686bd8d-9qgwd\" (UID: \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\") " pod="openshift-console/console-559686bd8d-9qgwd"
Jul 08 16:35:42.418552 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.418513    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-console-oauth-config\") pod \"console-559686bd8d-9qgwd\" (UID: \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\") " pod="openshift-console/console-559686bd8d-9qgwd"
Jul 08 16:35:42.419173 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.419143    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-service-ca\") pod \"console-559686bd8d-9qgwd\" (UID: \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\") " pod="openshift-console/console-559686bd8d-9qgwd"
Jul 08 16:35:42.419725 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.419704    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-console-config\") pod \"console-559686bd8d-9qgwd\" (UID: \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\") " pod="openshift-console/console-559686bd8d-9qgwd"
Jul 08 16:35:42.420219 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.420198    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-oauth-serving-cert\") pod \"console-559686bd8d-9qgwd\" (UID: \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\") " pod="openshift-console/console-559686bd8d-9qgwd"
Jul 08 16:35:42.421226 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.421196    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-console-serving-cert\") pod \"console-559686bd8d-9qgwd\" (UID: \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\") " pod="openshift-console/console-559686bd8d-9qgwd"
Jul 08 16:35:42.421360 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.421339    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-console-oauth-config\") pod \"console-559686bd8d-9qgwd\" (UID: \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\") " pod="openshift-console/console-559686bd8d-9qgwd"
Jul 08 16:35:42.428574 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.428536    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2g5q\" (UniqueName: \"kubernetes.io/projected/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-kube-api-access-p2g5q\") pod \"console-559686bd8d-9qgwd\" (UID: \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\") " pod="openshift-console/console-559686bd8d-9qgwd"
Jul 08 16:35:42.478352 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.478304    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kvhpr"
Jul 08 16:35:42.625862 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.625777    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-559686bd8d-9qgwd"
Jul 08 16:35:42.790482 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.790442    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-wlph5"]
Jul 08 16:35:42.793142 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.793112    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-wlph5"
Jul 08 16:35:42.796244 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.796019    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 08 16:35:42.796244 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.796078    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-7szj9\""
Jul 08 16:35:42.804537 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.804475    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-wlph5"]
Jul 08 16:35:42.820802 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.820767    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/cd61eaed-e188-4eaa-88b6-387b5333f6c9-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-wlph5\" (UID: \"cd61eaed-e188-4eaa-88b6-387b5333f6c9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-wlph5"
Jul 08 16:35:42.922132 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.922094    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/cd61eaed-e188-4eaa-88b6-387b5333f6c9-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-wlph5\" (UID: \"cd61eaed-e188-4eaa-88b6-387b5333f6c9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-wlph5"
Jul 08 16:35:42.925075 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:42.925044    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/cd61eaed-e188-4eaa-88b6-387b5333f6c9-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-wlph5\" (UID: \"cd61eaed-e188-4eaa-88b6-387b5333f6c9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-wlph5"
Jul 08 16:35:43.104495 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:43.104474    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-wlph5"
Jul 08 16:35:43.148658 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:43.148565    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-559686bd8d-9qgwd"]
Jul 08 16:35:43.272475 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:43.272442    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-wlph5"]
Jul 08 16:35:43.280358 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:35:43.280301    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd61eaed_e188_4eaa_88b6_387b5333f6c9.slice/crio-686760eb91d9e07e16dde04968efe4ef96d8199f5fd82fff024cfece524b7bc2 WatchSource:0}: Error finding container 686760eb91d9e07e16dde04968efe4ef96d8199f5fd82fff024cfece524b7bc2: Status 404 returned error can't find the container with id 686760eb91d9e07e16dde04968efe4ef96d8199f5fd82fff024cfece524b7bc2
Jul 08 16:35:43.561641 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:43.561547    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w69bj" event={"ID":"f96e3712-16d4-4a1d-9d73-1b3214dda98c","Type":"ContainerStarted","Data":"88ab4f77dcbb78794c7992255e9c96522f3168b94e64957404f203fb014de46b"}
Jul 08 16:35:43.562680 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:43.562654    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-wlph5" event={"ID":"cd61eaed-e188-4eaa-88b6-387b5333f6c9","Type":"ContainerStarted","Data":"686760eb91d9e07e16dde04968efe4ef96d8199f5fd82fff024cfece524b7bc2"}
Jul 08 16:35:43.563859 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:43.563825    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-559686bd8d-9qgwd" event={"ID":"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b","Type":"ContainerStarted","Data":"65331c46554c2f7184b8fc44c6a485c24a21d59e5b5ee89a6661765bbe608fb9"}
Jul 08 16:35:43.565407 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:43.565381    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wcgpv" event={"ID":"9f68ab1b-b463-44bb-87f3-1341b827b5ec","Type":"ContainerStarted","Data":"3a6b22094b5c522f1e5be3ff343c5f2de0516d19778d77b2e607e30ec623a419"}
Jul 08 16:35:43.567394 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:43.567371    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tpllq" event={"ID":"d9c7d9d9-4854-413e-8eec-d3608f5333f1","Type":"ContainerStarted","Data":"bcce04799378d2f5cd8d97e146b11a8732464e273e56a28e35e6a4188f30d5d8"}
Jul 08 16:35:43.567488 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:43.567406    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tpllq" event={"ID":"d9c7d9d9-4854-413e-8eec-d3608f5333f1","Type":"ContainerStarted","Data":"8dd8576c58fb846e5ac6f0e36aed715acc35875a41283f6555c89c9854f675a0"}
Jul 08 16:35:43.567560 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:43.567510    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tpllq"
Jul 08 16:35:43.601744 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:43.601681    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tpllq" podStartSLOduration=1.408084273 podStartE2EDuration="6.601659059s" totalImagesPullingTime="5.193574786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:35:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:43.601123441 +0000 UTC m=+62.942088521" watchObservedRunningTime="2026-07-08 16:35:43.601659059 +0000 UTC m=+62.942624137"
Jul 08 16:35:43.601920 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:43.601799    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wcgpv" podStartSLOduration=1.4573378639999999 podStartE2EDuration="6.60179175s" totalImagesPullingTime="5.144453886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:35:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:43.584029996 +0000 UTC m=+62.924995074" watchObservedRunningTime="2026-07-08 16:35:43.60179175 +0000 UTC m=+62.942756806"
Jul 08 16:35:44.572996 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:44.572962    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w69bj" event={"ID":"f96e3712-16d4-4a1d-9d73-1b3214dda98c","Type":"ContainerStarted","Data":"5d2457d8763833d938e5baae3de7a8d2527c6e7b00c0204bdcf01177f9d157ec"}
Jul 08 16:35:44.574702 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:44.574632    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-wlph5" event={"ID":"cd61eaed-e188-4eaa-88b6-387b5333f6c9","Type":"ContainerStarted","Data":"f828aa351ade1a7b0c034e0be0491a2eddb36f6bdc2eea7c1a2dbd7e61a9702d"}
Jul 08 16:35:44.575204 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:44.575183    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-wlph5"
Jul 08 16:35:44.580832 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:44.580804    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-wlph5"
Jul 08 16:35:44.594686 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:44.594637    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-wlph5" podStartSLOduration=1.439380184 podStartE2EDuration="2.594618963s" totalImagesPullingTime="1.155238779s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:35:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:44.594367112 +0000 UTC m=+63.935332190" watchObservedRunningTime="2026-07-08 16:35:44.594618963 +0000 UTC m=+63.935584036"
Jul 08 16:35:44.879839 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:44.879766    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-x927t"]
Jul 08 16:35:44.882050 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:44.882028    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x927t"
Jul 08 16:35:44.885043 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:44.885024    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 08 16:35:44.885278 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:44.885253    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 08 16:35:44.885494 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:44.885469    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 08 16:35:44.885699 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:44.885679    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 08 16:35:44.885889 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:44.885875    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-lxmzj\""
Jul 08 16:35:44.886195 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:44.886174    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 08 16:35:44.892042 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:44.892015    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5bb99c498f-vpzlq"]
Jul 08 16:35:44.895024 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:44.894440    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-x927t"]
Jul 08 16:35:44.895024 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:44.894564    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bb99c498f-vpzlq"
Jul 08 16:35:44.901961 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:44.901936    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 08 16:35:44.911964 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:44.911940    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bb99c498f-vpzlq"]
Jul 08 16:35:45.036837 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.036798    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdbxt\" (UniqueName: \"kubernetes.io/projected/6eae6020-c953-45de-a2bb-32e708038bcb-kube-api-access-jdbxt\") pod \"prometheus-operator-567f7b8b7c-x927t\" (UID: \"6eae6020-c953-45de-a2bb-32e708038bcb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x927t"
Jul 08 16:35:45.036837 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.036834    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-trusted-ca-bundle\") pod \"console-5bb99c498f-vpzlq\" (UID: \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\") " pod="openshift-console/console-5bb99c498f-vpzlq"
Jul 08 16:35:45.037086 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.036888    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-console-serving-cert\") pod \"console-5bb99c498f-vpzlq\" (UID: \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\") " pod="openshift-console/console-5bb99c498f-vpzlq"
Jul 08 16:35:45.037086 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.036909    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-console-config\") pod \"console-5bb99c498f-vpzlq\" (UID: \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\") " pod="openshift-console/console-5bb99c498f-vpzlq"
Jul 08 16:35:45.037086 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.036935    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-oauth-serving-cert\") pod \"console-5bb99c498f-vpzlq\" (UID: \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\") " pod="openshift-console/console-5bb99c498f-vpzlq"
Jul 08 16:35:45.037086 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.037028    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6eae6020-c953-45de-a2bb-32e708038bcb-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-x927t\" (UID: \"6eae6020-c953-45de-a2bb-32e708038bcb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x927t"
Jul 08 16:35:45.037290 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.037092    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6eae6020-c953-45de-a2bb-32e708038bcb-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-x927t\" (UID: \"6eae6020-c953-45de-a2bb-32e708038bcb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x927t"
Jul 08 16:35:45.037290 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.037122    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-service-ca\") pod \"console-5bb99c498f-vpzlq\" (UID: \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\") " pod="openshift-console/console-5bb99c498f-vpzlq"
Jul 08 16:35:45.037290 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.037165    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh87z\" (UniqueName: \"kubernetes.io/projected/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-kube-api-access-zh87z\") pod \"console-5bb99c498f-vpzlq\" (UID: \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\") " pod="openshift-console/console-5bb99c498f-vpzlq"
Jul 08 16:35:45.037290 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.037214    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6eae6020-c953-45de-a2bb-32e708038bcb-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-x927t\" (UID: \"6eae6020-c953-45de-a2bb-32e708038bcb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x927t"
Jul 08 16:35:45.037290 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.037250    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-console-oauth-config\") pod \"console-5bb99c498f-vpzlq\" (UID: \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\") " pod="openshift-console/console-5bb99c498f-vpzlq"
Jul 08 16:35:45.137860 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.137780    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdbxt\" (UniqueName: \"kubernetes.io/projected/6eae6020-c953-45de-a2bb-32e708038bcb-kube-api-access-jdbxt\") pod \"prometheus-operator-567f7b8b7c-x927t\" (UID: \"6eae6020-c953-45de-a2bb-32e708038bcb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x927t"
Jul 08 16:35:45.137860 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.137825    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-trusted-ca-bundle\") pod \"console-5bb99c498f-vpzlq\" (UID: \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\") " pod="openshift-console/console-5bb99c498f-vpzlq"
Jul 08 16:35:45.137860 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.137849    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-console-serving-cert\") pod \"console-5bb99c498f-vpzlq\" (UID: \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\") " pod="openshift-console/console-5bb99c498f-vpzlq"
Jul 08 16:35:45.138050 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.137868    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-console-config\") pod \"console-5bb99c498f-vpzlq\" (UID: \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\") " pod="openshift-console/console-5bb99c498f-vpzlq"
Jul 08 16:35:45.138050 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.137885    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-oauth-serving-cert\") pod \"console-5bb99c498f-vpzlq\" (UID: \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\") " pod="openshift-console/console-5bb99c498f-vpzlq"
Jul 08 16:35:45.138050 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.137936    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6eae6020-c953-45de-a2bb-32e708038bcb-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-x927t\" (UID: \"6eae6020-c953-45de-a2bb-32e708038bcb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x927t"
Jul 08 16:35:45.138050 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.137962    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6eae6020-c953-45de-a2bb-32e708038bcb-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-x927t\" (UID: \"6eae6020-c953-45de-a2bb-32e708038bcb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x927t"
Jul 08 16:35:45.138050 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.137980    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-service-ca\") pod \"console-5bb99c498f-vpzlq\" (UID: \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\") " pod="openshift-console/console-5bb99c498f-vpzlq"
Jul 08 16:35:45.138050 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.138011    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh87z\" (UniqueName: \"kubernetes.io/projected/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-kube-api-access-zh87z\") pod \"console-5bb99c498f-vpzlq\" (UID: \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\") " pod="openshift-console/console-5bb99c498f-vpzlq"
Jul 08 16:35:45.138050 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.138030    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6eae6020-c953-45de-a2bb-32e708038bcb-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-x927t\" (UID: \"6eae6020-c953-45de-a2bb-32e708038bcb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x927t"
Jul 08 16:35:45.138379 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.138063    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-console-oauth-config\") pod \"console-5bb99c498f-vpzlq\" (UID: \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\") " pod="openshift-console/console-5bb99c498f-vpzlq"
Jul 08 16:35:45.139495 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.139096    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-oauth-serving-cert\") pod \"console-5bb99c498f-vpzlq\" (UID: \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\") " pod="openshift-console/console-5bb99c498f-vpzlq"
Jul 08 16:35:45.139495 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.139131    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-service-ca\") pod \"console-5bb99c498f-vpzlq\" (UID: \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\") " pod="openshift-console/console-5bb99c498f-vpzlq"
Jul 08 16:35:45.140163 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.140134    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6eae6020-c953-45de-a2bb-32e708038bcb-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-x927t\" (UID: \"6eae6020-c953-45de-a2bb-32e708038bcb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x927t"
Jul 08 16:35:45.140901 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.140878    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-trusted-ca-bundle\") pod \"console-5bb99c498f-vpzlq\" (UID: \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\") " pod="openshift-console/console-5bb99c498f-vpzlq"
Jul 08 16:35:45.141860 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.141835    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6eae6020-c953-45de-a2bb-32e708038bcb-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-x927t\" (UID: \"6eae6020-c953-45de-a2bb-32e708038bcb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x927t"
Jul 08 16:35:45.146627 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.145012    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6eae6020-c953-45de-a2bb-32e708038bcb-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-x927t\" (UID: \"6eae6020-c953-45de-a2bb-32e708038bcb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x927t"
Jul 08 16:35:45.146627 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.145024    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-console-config\") pod \"console-5bb99c498f-vpzlq\" (UID: \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\") " pod="openshift-console/console-5bb99c498f-vpzlq"
Jul 08 16:35:45.146627 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.145400    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-console-oauth-config\") pod \"console-5bb99c498f-vpzlq\" (UID: \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\") " pod="openshift-console/console-5bb99c498f-vpzlq"
Jul 08 16:35:45.148179 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.148138    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-console-serving-cert\") pod \"console-5bb99c498f-vpzlq\" (UID: \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\") " pod="openshift-console/console-5bb99c498f-vpzlq"
Jul 08 16:35:45.148403 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.148385    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh87z\" (UniqueName: \"kubernetes.io/projected/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-kube-api-access-zh87z\") pod \"console-5bb99c498f-vpzlq\" (UID: \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\") " pod="openshift-console/console-5bb99c498f-vpzlq"
Jul 08 16:35:45.149994 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.149971    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdbxt\" (UniqueName: \"kubernetes.io/projected/6eae6020-c953-45de-a2bb-32e708038bcb-kube-api-access-jdbxt\") pod \"prometheus-operator-567f7b8b7c-x927t\" (UID: \"6eae6020-c953-45de-a2bb-32e708038bcb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x927t"
Jul 08 16:35:45.193590 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.193546    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x927t"
Jul 08 16:35:45.207050 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.206827    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bb99c498f-vpzlq"
Jul 08 16:35:45.843481 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.843440    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-original-pull-secret\") pod \"global-pull-secret-syncer-4vtkl\" (UID: \"8eb2abc0-b610-4f3b-b6dc-02131ad29e35\") " pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:35:45.843952 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.843527    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2796ef11-ffb4-4bff-b73b-38d727eb45f2-metrics-certs\") pod \"network-metrics-daemon-z82nm\" (UID: \"2796ef11-ffb4-4bff-b73b-38d727eb45f2\") " pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:35:45.848281 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.848255    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 08 16:35:45.848962 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.848943    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 08 16:35:45.856643 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.856622    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2796ef11-ffb4-4bff-b73b-38d727eb45f2-metrics-certs\") pod \"network-metrics-daemon-z82nm\" (UID: \"2796ef11-ffb4-4bff-b73b-38d727eb45f2\") " pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:35:45.857531 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.857511    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8eb2abc0-b610-4f3b-b6dc-02131ad29e35-original-pull-secret\") pod \"global-pull-secret-syncer-4vtkl\" (UID: \"8eb2abc0-b610-4f3b-b6dc-02131ad29e35\") " pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:35:45.943885 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.943842    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqv9w\" (UniqueName: \"kubernetes.io/projected/c98cc144-3322-4f83-b571-9772b5a63d49-kube-api-access-kqv9w\") pod \"network-check-target-bkqjt\" (UID: \"c98cc144-3322-4f83-b571-9772b5a63d49\") " pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:35:45.947034 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.947006    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 08 16:35:45.957486 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.957456    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 08 16:35:45.967779 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:45.967754    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqv9w\" (UniqueName: \"kubernetes.io/projected/c98cc144-3322-4f83-b571-9772b5a63d49-kube-api-access-kqv9w\") pod \"network-check-target-bkqjt\" (UID: \"c98cc144-3322-4f83-b571-9772b5a63d49\") " pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:35:46.027873 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:46.027832    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtkl"
Jul 08 16:35:46.036329 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:46.036289    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pp6cc\""
Jul 08 16:35:46.042389 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:46.042366    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kw94b\""
Jul 08 16:35:46.044476 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:46.044452    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z82nm"
Jul 08 16:35:46.050192 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:46.050172    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:35:47.562460 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:47.562378    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bkqjt"]
Jul 08 16:35:47.565961 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:35:47.565937    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc98cc144_3322_4f83_b571_9772b5a63d49.slice/crio-85bb9ecf0b5b399a4d3c5a2c1ad58777fa0e06c2c0fdf9ceecc0ea7289b4cafe WatchSource:0}: Error finding container 85bb9ecf0b5b399a4d3c5a2c1ad58777fa0e06c2c0fdf9ceecc0ea7289b4cafe: Status 404 returned error can't find the container with id 85bb9ecf0b5b399a4d3c5a2c1ad58777fa0e06c2c0fdf9ceecc0ea7289b4cafe
Jul 08 16:35:47.588430 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:47.588399    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w69bj" event={"ID":"f96e3712-16d4-4a1d-9d73-1b3214dda98c","Type":"ContainerStarted","Data":"1e95e7b107167a3db56cc9f943f21896aa76106b845e086c6e2a193ad25fe75b"}
Jul 08 16:35:47.589940 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:47.589914    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-559686bd8d-9qgwd" event={"ID":"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b","Type":"ContainerStarted","Data":"5f5fa63622c4c44de793d80e8a120c3a6c1fea0256e9ed441dd1972a9189ad5e"}
Jul 08 16:35:47.591028 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:47.591007    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bkqjt" event={"ID":"c98cc144-3322-4f83-b571-9772b5a63d49","Type":"ContainerStarted","Data":"85bb9ecf0b5b399a4d3c5a2c1ad58777fa0e06c2c0fdf9ceecc0ea7289b4cafe"}
Jul 08 16:35:47.606859 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:47.606812    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-w69bj" podStartSLOduration=1.233370157 podStartE2EDuration="10.606795377s" totalImagesPullingTime="9.37342522s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:35:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:47.605485065 +0000 UTC m=+66.946450135" watchObservedRunningTime="2026-07-08 16:35:47.606795377 +0000 UTC m=+66.947760454"
Jul 08 16:35:47.622464 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:47.622339    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-559686bd8d-9qgwd" podStartSLOduration=1.469193983 podStartE2EDuration="5.622299387s" totalImagesPullingTime="4.153105404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:35:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:47.622099078 +0000 UTC m=+66.963064163" watchObservedRunningTime="2026-07-08 16:35:47.622299387 +0000 UTC m=+66.963264466"
Jul 08 16:35:47.706228 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:47.706195    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4vtkl"]
Jul 08 16:35:47.709477 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:35:47.709435    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eb2abc0_b610_4f3b_b6dc_02131ad29e35.slice/crio-5038b36296b1269875a1c0dcb2f4e88d14143b326b61ca299f35a6dd7d9c9bb4 WatchSource:0}: Error finding container 5038b36296b1269875a1c0dcb2f4e88d14143b326b61ca299f35a6dd7d9c9bb4: Status 404 returned error can't find the container with id 5038b36296b1269875a1c0dcb2f4e88d14143b326b61ca299f35a6dd7d9c9bb4
Jul 08 16:35:47.710181 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:47.710163    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bb99c498f-vpzlq"]
Jul 08 16:35:47.713499 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:35:47.713398    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb85d7a12_14dc_44f9_8cc8_cc68bfd56125.slice/crio-f0c29988c10b4d92493db01a964642e7128deb99005a6c21adffd641e8412cae WatchSource:0}: Error finding container f0c29988c10b4d92493db01a964642e7128deb99005a6c21adffd641e8412cae: Status 404 returned error can't find the container with id f0c29988c10b4d92493db01a964642e7128deb99005a6c21adffd641e8412cae
Jul 08 16:35:47.741919 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:47.741897    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-z82nm"]
Jul 08 16:35:47.744648 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:47.744627    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-x927t"]
Jul 08 16:35:47.749598 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:35:47.749566    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2796ef11_ffb4_4bff_b73b_38d727eb45f2.slice/crio-b6ac0e37ed1df340ed066c1f76430c85bf848bc666428a1a3fcf17c4c48c4666 WatchSource:0}: Error finding container b6ac0e37ed1df340ed066c1f76430c85bf848bc666428a1a3fcf17c4c48c4666: Status 404 returned error can't find the container with id b6ac0e37ed1df340ed066c1f76430c85bf848bc666428a1a3fcf17c4c48c4666
Jul 08 16:35:47.749837 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:35:47.749807    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6eae6020_c953_45de_a2bb_32e708038bcb.slice/crio-0db782cdb67fe0e2be21a4bfa596ad987f386c048ad732683198fb2b1c0f601c WatchSource:0}: Error finding container 0db782cdb67fe0e2be21a4bfa596ad987f386c048ad732683198fb2b1c0f601c: Status 404 returned error can't find the container with id 0db782cdb67fe0e2be21a4bfa596ad987f386c048ad732683198fb2b1c0f601c
Jul 08 16:35:48.598768 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:48.598720    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4vtkl" event={"ID":"8eb2abc0-b610-4f3b-b6dc-02131ad29e35","Type":"ContainerStarted","Data":"5038b36296b1269875a1c0dcb2f4e88d14143b326b61ca299f35a6dd7d9c9bb4"}
Jul 08 16:35:48.600880 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:48.600846    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x927t" event={"ID":"6eae6020-c953-45de-a2bb-32e708038bcb","Type":"ContainerStarted","Data":"0db782cdb67fe0e2be21a4bfa596ad987f386c048ad732683198fb2b1c0f601c"}
Jul 08 16:35:48.604418 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:48.603646    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bb99c498f-vpzlq" event={"ID":"b85d7a12-14dc-44f9-8cc8-cc68bfd56125","Type":"ContainerStarted","Data":"3274a7f42296057ea85a4bc2f450a177a58b0c46c423c6cccc052414afb0ac66"}
Jul 08 16:35:48.604418 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:48.603690    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bb99c498f-vpzlq" event={"ID":"b85d7a12-14dc-44f9-8cc8-cc68bfd56125","Type":"ContainerStarted","Data":"f0c29988c10b4d92493db01a964642e7128deb99005a6c21adffd641e8412cae"}
Jul 08 16:35:48.607449 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:48.606775    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-z82nm" event={"ID":"2796ef11-ffb4-4bff-b73b-38d727eb45f2","Type":"ContainerStarted","Data":"b6ac0e37ed1df340ed066c1f76430c85bf848bc666428a1a3fcf17c4c48c4666"}
Jul 08 16:35:48.623332 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:48.623268    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5bb99c498f-vpzlq" podStartSLOduration=4.623249596 podStartE2EDuration="4.623249596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:35:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:48.622004749 +0000 UTC m=+67.962969829" watchObservedRunningTime="2026-07-08 16:35:48.623249596 +0000 UTC m=+67.964214675"
Jul 08 16:35:49.615687 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:49.615648    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-z82nm" event={"ID":"2796ef11-ffb4-4bff-b73b-38d727eb45f2","Type":"ContainerStarted","Data":"9ab3891f56694b75daef5365b3096400c149132803d9f17fbac9f6f4768c30e4"}
Jul 08 16:35:49.619485 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:49.619382    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x927t" event={"ID":"6eae6020-c953-45de-a2bb-32e708038bcb","Type":"ContainerStarted","Data":"c2540f58f40fb6727a9f22d72264879c5a08b4069e3d3833d948442e90f76e73"}
Jul 08 16:35:49.619485 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:49.619445    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x927t" event={"ID":"6eae6020-c953-45de-a2bb-32e708038bcb","Type":"ContainerStarted","Data":"936bfd702907d761d30b797b241241a2fec777c421f201dfaad9efdeecdcda6f"}
Jul 08 16:35:50.625464 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:50.625422    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-z82nm" event={"ID":"2796ef11-ffb4-4bff-b73b-38d727eb45f2","Type":"ContainerStarted","Data":"5c4b09ca796480891441a0ed3e20d3f7a8387c7d2b84a98ae67f17bed4ecd38e"}
Jul 08 16:35:50.644822 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:50.644758    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-z82nm" podStartSLOduration=68.005319008 podStartE2EDuration="1m9.644741139s" totalImagesPullingTime="1.639422131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:50.643983939 +0000 UTC m=+69.984949019" watchObservedRunningTime="2026-07-08 16:35:50.644741139 +0000 UTC m=+69.985706216"
Jul 08 16:35:50.645013 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:50.644904    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x927t" podStartSLOduration=5.002242177 podStartE2EDuration="6.644897883s" totalImagesPullingTime="1.642655706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:35:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:49.646353318 +0000 UTC m=+68.987318396" watchObservedRunningTime="2026-07-08 16:35:50.644897883 +0000 UTC m=+69.985862960"
Jul 08 16:35:51.355113 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.355077    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-fsp5w"]
Jul 08 16:35:51.375396 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.375367    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fsp5w"
Jul 08 16:35:51.378697 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.378667    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 08 16:35:51.378856 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.378767    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-2685j\""
Jul 08 16:35:51.379115 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.379093    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 08 16:35:51.379235 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.379112    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-fsp5w"]
Jul 08 16:35:51.380125 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.380104    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ckxc5"]
Jul 08 16:35:51.387161 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.387141    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ed65374-16ab-405d-96b6-69d3318cc2e7-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-fsp5w\" (UID: \"3ed65374-16ab-405d-96b6-69d3318cc2e7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fsp5w"
Jul 08 16:35:51.387272 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.387171    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ed65374-16ab-405d-96b6-69d3318cc2e7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-fsp5w\" (UID: \"3ed65374-16ab-405d-96b6-69d3318cc2e7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fsp5w"
Jul 08 16:35:51.387272 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.387210    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb47n\" (UniqueName: \"kubernetes.io/projected/3ed65374-16ab-405d-96b6-69d3318cc2e7-kube-api-access-lb47n\") pod \"openshift-state-metrics-54cf96dfcf-fsp5w\" (UID: \"3ed65374-16ab-405d-96b6-69d3318cc2e7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fsp5w"
Jul 08 16:35:51.387272 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.387239    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ed65374-16ab-405d-96b6-69d3318cc2e7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-fsp5w\" (UID: \"3ed65374-16ab-405d-96b6-69d3318cc2e7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fsp5w"
Jul 08 16:35:51.399076 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.399044    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fd6f4"]
Jul 08 16:35:51.399271 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.399240    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ckxc5"
Jul 08 16:35:51.403395 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.403374    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 08 16:35:51.403854 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.403828    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 08 16:35:51.404705 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.404685    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-hkr4h\""
Jul 08 16:35:51.405302 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.405284    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 08 16:35:51.416979 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.416946    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ckxc5"]
Jul 08 16:35:51.417084 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.417066    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:51.420463 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.420441    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 08 16:35:51.421226 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.421206    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 08 16:35:51.421302 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.421262    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-w5pbb\""
Jul 08 16:35:51.421495 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.421479    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 08 16:35:51.487599 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.487567    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8f30902a-26b7-48d4-92f1-d0ca067dab66-node-exporter-tls\") pod \"node-exporter-fd6f4\" (UID: \"8f30902a-26b7-48d4-92f1-d0ca067dab66\") " pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:51.487788 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.487615    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ed65374-16ab-405d-96b6-69d3318cc2e7-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-fsp5w\" (UID: \"3ed65374-16ab-405d-96b6-69d3318cc2e7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fsp5w"
Jul 08 16:35:51.487788 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.487648    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ed65374-16ab-405d-96b6-69d3318cc2e7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-fsp5w\" (UID: \"3ed65374-16ab-405d-96b6-69d3318cc2e7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fsp5w"
Jul 08 16:35:51.487788 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.487678    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ade5e891-8f50-4385-9388-6ebfad50927d-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-ckxc5\" (UID: \"ade5e891-8f50-4385-9388-6ebfad50927d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ckxc5"
Jul 08 16:35:51.487788 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.487704    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8f30902a-26b7-48d4-92f1-d0ca067dab66-node-exporter-textfile\") pod \"node-exporter-fd6f4\" (UID: \"8f30902a-26b7-48d4-92f1-d0ca067dab66\") " pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:51.487788 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.487728    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ade5e891-8f50-4385-9388-6ebfad50927d-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-ckxc5\" (UID: \"ade5e891-8f50-4385-9388-6ebfad50927d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ckxc5"
Jul 08 16:35:51.487788 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.487780    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f30902a-26b7-48d4-92f1-d0ca067dab66-sys\") pod \"node-exporter-fd6f4\" (UID: \"8f30902a-26b7-48d4-92f1-d0ca067dab66\") " pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:51.488150 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.487900    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8f30902a-26b7-48d4-92f1-d0ca067dab66-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fd6f4\" (UID: \"8f30902a-26b7-48d4-92f1-d0ca067dab66\") " pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:51.488150 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.487937    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8f30902a-26b7-48d4-92f1-d0ca067dab66-root\") pod \"node-exporter-fd6f4\" (UID: \"8f30902a-26b7-48d4-92f1-d0ca067dab66\") " pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:51.488150 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.487970    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb47n\" (UniqueName: \"kubernetes.io/projected/3ed65374-16ab-405d-96b6-69d3318cc2e7-kube-api-access-lb47n\") pod \"openshift-state-metrics-54cf96dfcf-fsp5w\" (UID: \"3ed65374-16ab-405d-96b6-69d3318cc2e7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fsp5w"
Jul 08 16:35:51.488150 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.488001    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8f30902a-26b7-48d4-92f1-d0ca067dab66-node-exporter-accelerators-collector-config\") pod \"node-exporter-fd6f4\" (UID: \"8f30902a-26b7-48d4-92f1-d0ca067dab66\") " pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:51.488150 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.488029    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ade5e891-8f50-4385-9388-6ebfad50927d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-ckxc5\" (UID: \"ade5e891-8f50-4385-9388-6ebfad50927d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ckxc5"
Jul 08 16:35:51.488150 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.488060    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f30902a-26b7-48d4-92f1-d0ca067dab66-metrics-client-ca\") pod \"node-exporter-fd6f4\" (UID: \"8f30902a-26b7-48d4-92f1-d0ca067dab66\") " pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:51.488150 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.488083    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkdkr\" (UniqueName: \"kubernetes.io/projected/ade5e891-8f50-4385-9388-6ebfad50927d-kube-api-access-dkdkr\") pod \"kube-state-metrics-7c9dbb8b7d-ckxc5\" (UID: \"ade5e891-8f50-4385-9388-6ebfad50927d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ckxc5"
Jul 08 16:35:51.488150 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.488112    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ed65374-16ab-405d-96b6-69d3318cc2e7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-fsp5w\" (UID: \"3ed65374-16ab-405d-96b6-69d3318cc2e7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fsp5w"
Jul 08 16:35:51.488472 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.488159    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8f30902a-26b7-48d4-92f1-d0ca067dab66-node-exporter-wtmp\") pod \"node-exporter-fd6f4\" (UID: \"8f30902a-26b7-48d4-92f1-d0ca067dab66\") " pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:51.488472 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.488241    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ed65374-16ab-405d-96b6-69d3318cc2e7-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-fsp5w\" (UID: \"3ed65374-16ab-405d-96b6-69d3318cc2e7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fsp5w"
Jul 08 16:35:51.488472 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.488254    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-869sp\" (UniqueName: \"kubernetes.io/projected/8f30902a-26b7-48d4-92f1-d0ca067dab66-kube-api-access-869sp\") pod \"node-exporter-fd6f4\" (UID: \"8f30902a-26b7-48d4-92f1-d0ca067dab66\") " pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:51.488472 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.488294    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ade5e891-8f50-4385-9388-6ebfad50927d-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-ckxc5\" (UID: \"ade5e891-8f50-4385-9388-6ebfad50927d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ckxc5"
Jul 08 16:35:51.488472 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.488350    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ade5e891-8f50-4385-9388-6ebfad50927d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-ckxc5\" (UID: \"ade5e891-8f50-4385-9388-6ebfad50927d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ckxc5"
Jul 08 16:35:51.490347 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.490299    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ed65374-16ab-405d-96b6-69d3318cc2e7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-fsp5w\" (UID: \"3ed65374-16ab-405d-96b6-69d3318cc2e7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fsp5w"
Jul 08 16:35:51.490458 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.490443    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ed65374-16ab-405d-96b6-69d3318cc2e7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-fsp5w\" (UID: \"3ed65374-16ab-405d-96b6-69d3318cc2e7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fsp5w"
Jul 08 16:35:51.501076 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.501046    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb47n\" (UniqueName: \"kubernetes.io/projected/3ed65374-16ab-405d-96b6-69d3318cc2e7-kube-api-access-lb47n\") pod \"openshift-state-metrics-54cf96dfcf-fsp5w\" (UID: \"3ed65374-16ab-405d-96b6-69d3318cc2e7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fsp5w"
Jul 08 16:35:51.588984 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.588949    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8f30902a-26b7-48d4-92f1-d0ca067dab66-node-exporter-textfile\") pod \"node-exporter-fd6f4\" (UID: \"8f30902a-26b7-48d4-92f1-d0ca067dab66\") " pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:51.588984 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.588988    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ade5e891-8f50-4385-9388-6ebfad50927d-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-ckxc5\" (UID: \"ade5e891-8f50-4385-9388-6ebfad50927d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ckxc5"
Jul 08 16:35:51.589193 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.589006    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f30902a-26b7-48d4-92f1-d0ca067dab66-sys\") pod \"node-exporter-fd6f4\" (UID: \"8f30902a-26b7-48d4-92f1-d0ca067dab66\") " pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:51.589193 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.589038    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8f30902a-26b7-48d4-92f1-d0ca067dab66-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fd6f4\" (UID: \"8f30902a-26b7-48d4-92f1-d0ca067dab66\") " pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:51.589193 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.589106    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8f30902a-26b7-48d4-92f1-d0ca067dab66-root\") pod \"node-exporter-fd6f4\" (UID: \"8f30902a-26b7-48d4-92f1-d0ca067dab66\") " pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:51.589193 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.589113    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f30902a-26b7-48d4-92f1-d0ca067dab66-sys\") pod \"node-exporter-fd6f4\" (UID: \"8f30902a-26b7-48d4-92f1-d0ca067dab66\") " pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:51.589193 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.589140    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8f30902a-26b7-48d4-92f1-d0ca067dab66-node-exporter-accelerators-collector-config\") pod \"node-exporter-fd6f4\" (UID: \"8f30902a-26b7-48d4-92f1-d0ca067dab66\") " pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:51.589193 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.589168    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ade5e891-8f50-4385-9388-6ebfad50927d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-ckxc5\" (UID: \"ade5e891-8f50-4385-9388-6ebfad50927d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ckxc5"
Jul 08 16:35:51.589193 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.589195    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f30902a-26b7-48d4-92f1-d0ca067dab66-metrics-client-ca\") pod \"node-exporter-fd6f4\" (UID: \"8f30902a-26b7-48d4-92f1-d0ca067dab66\") " pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:51.589465 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.589219    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkdkr\" (UniqueName: \"kubernetes.io/projected/ade5e891-8f50-4385-9388-6ebfad50927d-kube-api-access-dkdkr\") pod \"kube-state-metrics-7c9dbb8b7d-ckxc5\" (UID: \"ade5e891-8f50-4385-9388-6ebfad50927d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ckxc5"
Jul 08 16:35:51.589465 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.589254    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8f30902a-26b7-48d4-92f1-d0ca067dab66-node-exporter-wtmp\") pod \"node-exporter-fd6f4\" (UID: \"8f30902a-26b7-48d4-92f1-d0ca067dab66\") " pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:51.589465 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.589280    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-869sp\" (UniqueName: \"kubernetes.io/projected/8f30902a-26b7-48d4-92f1-d0ca067dab66-kube-api-access-869sp\") pod \"node-exporter-fd6f4\" (UID: \"8f30902a-26b7-48d4-92f1-d0ca067dab66\") " pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:51.589465 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.589331    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ade5e891-8f50-4385-9388-6ebfad50927d-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-ckxc5\" (UID: \"ade5e891-8f50-4385-9388-6ebfad50927d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ckxc5"
Jul 08 16:35:51.589465 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.589363    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ade5e891-8f50-4385-9388-6ebfad50927d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-ckxc5\" (UID: \"ade5e891-8f50-4385-9388-6ebfad50927d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ckxc5"
Jul 08 16:35:51.589465 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.589393    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8f30902a-26b7-48d4-92f1-d0ca067dab66-node-exporter-tls\") pod \"node-exporter-fd6f4\" (UID: \"8f30902a-26b7-48d4-92f1-d0ca067dab66\") " pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:51.589465 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.589410    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8f30902a-26b7-48d4-92f1-d0ca067dab66-node-exporter-textfile\") pod \"node-exporter-fd6f4\" (UID: \"8f30902a-26b7-48d4-92f1-d0ca067dab66\") " pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:51.589465 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.589440    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ade5e891-8f50-4385-9388-6ebfad50927d-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-ckxc5\" (UID: \"ade5e891-8f50-4385-9388-6ebfad50927d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ckxc5"
Jul 08 16:35:51.589760 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.589483    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8f30902a-26b7-48d4-92f1-d0ca067dab66-root\") pod \"node-exporter-fd6f4\" (UID: \"8f30902a-26b7-48d4-92f1-d0ca067dab66\") " pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:51.589760 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:51.589550    2582 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 08 16:35:51.589760 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.589606    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8f30902a-26b7-48d4-92f1-d0ca067dab66-node-exporter-wtmp\") pod \"node-exporter-fd6f4\" (UID: \"8f30902a-26b7-48d4-92f1-d0ca067dab66\") " pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:51.589760 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:51.589627    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ade5e891-8f50-4385-9388-6ebfad50927d-kube-state-metrics-tls podName:ade5e891-8f50-4385-9388-6ebfad50927d nodeName:}" failed. No retries permitted until 2026-07-08 16:35:52.089603453 +0000 UTC m=+71.430568510 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/ade5e891-8f50-4385-9388-6ebfad50927d-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-ckxc5" (UID: "ade5e891-8f50-4385-9388-6ebfad50927d") : secret "kube-state-metrics-tls" not found
Jul 08 16:35:51.589760 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.589753    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8f30902a-26b7-48d4-92f1-d0ca067dab66-node-exporter-accelerators-collector-config\") pod \"node-exporter-fd6f4\" (UID: \"8f30902a-26b7-48d4-92f1-d0ca067dab66\") " pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:51.590173 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.589778    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ade5e891-8f50-4385-9388-6ebfad50927d-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-ckxc5\" (UID: \"ade5e891-8f50-4385-9388-6ebfad50927d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ckxc5"
Jul 08 16:35:51.590173 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.589940    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ade5e891-8f50-4385-9388-6ebfad50927d-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-ckxc5\" (UID: \"ade5e891-8f50-4385-9388-6ebfad50927d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ckxc5"
Jul 08 16:35:51.590173 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:51.589971    2582 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 08 16:35:51.590173 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:35:51.590024    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f30902a-26b7-48d4-92f1-d0ca067dab66-node-exporter-tls podName:8f30902a-26b7-48d4-92f1-d0ca067dab66 nodeName:}" failed. No retries permitted until 2026-07-08 16:35:52.090006826 +0000 UTC m=+71.430971902 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/8f30902a-26b7-48d4-92f1-d0ca067dab66-node-exporter-tls") pod "node-exporter-fd6f4" (UID: "8f30902a-26b7-48d4-92f1-d0ca067dab66") : secret "node-exporter-tls" not found
Jul 08 16:35:51.590404 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.590226    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f30902a-26b7-48d4-92f1-d0ca067dab66-metrics-client-ca\") pod \"node-exporter-fd6f4\" (UID: \"8f30902a-26b7-48d4-92f1-d0ca067dab66\") " pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:51.590462 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.590409    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ade5e891-8f50-4385-9388-6ebfad50927d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-ckxc5\" (UID: \"ade5e891-8f50-4385-9388-6ebfad50927d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ckxc5"
Jul 08 16:35:51.592081 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.591927    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8f30902a-26b7-48d4-92f1-d0ca067dab66-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fd6f4\" (UID: \"8f30902a-26b7-48d4-92f1-d0ca067dab66\") " pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:51.592688 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.592664    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ade5e891-8f50-4385-9388-6ebfad50927d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-ckxc5\" (UID: \"ade5e891-8f50-4385-9388-6ebfad50927d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ckxc5"
Jul 08 16:35:51.610112 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.610052    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkdkr\" (UniqueName: \"kubernetes.io/projected/ade5e891-8f50-4385-9388-6ebfad50927d-kube-api-access-dkdkr\") pod \"kube-state-metrics-7c9dbb8b7d-ckxc5\" (UID: \"ade5e891-8f50-4385-9388-6ebfad50927d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ckxc5"
Jul 08 16:35:51.610112 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.610073    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-869sp\" (UniqueName: \"kubernetes.io/projected/8f30902a-26b7-48d4-92f1-d0ca067dab66-kube-api-access-869sp\") pod \"node-exporter-fd6f4\" (UID: \"8f30902a-26b7-48d4-92f1-d0ca067dab66\") " pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:51.686487 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:51.686243    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fsp5w"
Jul 08 16:35:52.094978 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.094947    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8f30902a-26b7-48d4-92f1-d0ca067dab66-node-exporter-tls\") pod \"node-exporter-fd6f4\" (UID: \"8f30902a-26b7-48d4-92f1-d0ca067dab66\") " pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:52.095196 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.094994    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ade5e891-8f50-4385-9388-6ebfad50927d-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-ckxc5\" (UID: \"ade5e891-8f50-4385-9388-6ebfad50927d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ckxc5"
Jul 08 16:35:52.098024 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.097993    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8f30902a-26b7-48d4-92f1-d0ca067dab66-node-exporter-tls\") pod \"node-exporter-fd6f4\" (UID: \"8f30902a-26b7-48d4-92f1-d0ca067dab66\") " pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:52.098135 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.098044    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ade5e891-8f50-4385-9388-6ebfad50927d-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-ckxc5\" (UID: \"ade5e891-8f50-4385-9388-6ebfad50927d\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ckxc5"
Jul 08 16:35:52.311695 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.311653    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ckxc5"
Jul 08 16:35:52.326406 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.326379    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fd6f4"
Jul 08 16:35:52.416149 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.416116    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 08 16:35:52.436712 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.436674    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 08 16:35:52.436882 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.436823    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.439639 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.439606    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 08 16:35:52.439639 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.439629    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 08 16:35:52.439851 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.439605    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 08 16:35:52.439851 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.439606    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 08 16:35:52.439959 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.439897    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 08 16:35:52.440343 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.440033    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 08 16:35:52.440343 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.440081    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 08 16:35:52.440343 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.440035    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 08 16:35:52.440343 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.440120    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 08 16:35:52.440343 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.440346    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-pg7md\""
Jul 08 16:35:52.498366 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.498335    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/45190ca2-bc75-41c0-8d71-d01d5728f856-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.498519 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.498381    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbz54\" (UniqueName: \"kubernetes.io/projected/45190ca2-bc75-41c0-8d71-d01d5728f856-kube-api-access-rbz54\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.498519 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.498419    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/45190ca2-bc75-41c0-8d71-d01d5728f856-config-volume\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.498519 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.498448    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/45190ca2-bc75-41c0-8d71-d01d5728f856-tls-assets\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.498519 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.498501    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/45190ca2-bc75-41c0-8d71-d01d5728f856-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.498726 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.498526    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45190ca2-bc75-41c0-8d71-d01d5728f856-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.498726 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.498604    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/45190ca2-bc75-41c0-8d71-d01d5728f856-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.498726 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.498646    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/45190ca2-bc75-41c0-8d71-d01d5728f856-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.498726 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.498686    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/45190ca2-bc75-41c0-8d71-d01d5728f856-web-config\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.498922 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.498728    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/45190ca2-bc75-41c0-8d71-d01d5728f856-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.498922 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.498752    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/45190ca2-bc75-41c0-8d71-d01d5728f856-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.498922 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.498824    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45190ca2-bc75-41c0-8d71-d01d5728f856-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.498922 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.498901    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/45190ca2-bc75-41c0-8d71-d01d5728f856-config-out\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.600248 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.600201    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/45190ca2-bc75-41c0-8d71-d01d5728f856-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.600425 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.600269    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/45190ca2-bc75-41c0-8d71-d01d5728f856-web-config\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.600425 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.600351    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/45190ca2-bc75-41c0-8d71-d01d5728f856-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.600425 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.600383    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/45190ca2-bc75-41c0-8d71-d01d5728f856-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.600552 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.600423    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45190ca2-bc75-41c0-8d71-d01d5728f856-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.600552 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.600474    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/45190ca2-bc75-41c0-8d71-d01d5728f856-config-out\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.600552 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.600526    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/45190ca2-bc75-41c0-8d71-d01d5728f856-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.600651 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.600561    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbz54\" (UniqueName: \"kubernetes.io/projected/45190ca2-bc75-41c0-8d71-d01d5728f856-kube-api-access-rbz54\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.600651 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.600606    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/45190ca2-bc75-41c0-8d71-d01d5728f856-config-volume\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.600651 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.600639    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/45190ca2-bc75-41c0-8d71-d01d5728f856-tls-assets\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.600756 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.600663    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/45190ca2-bc75-41c0-8d71-d01d5728f856-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.600756 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.600694    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45190ca2-bc75-41c0-8d71-d01d5728f856-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.600756 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.600743    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/45190ca2-bc75-41c0-8d71-d01d5728f856-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.606004 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.605974    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/45190ca2-bc75-41c0-8d71-d01d5728f856-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.606154 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.606111    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45190ca2-bc75-41c0-8d71-d01d5728f856-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.607655 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.607137    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45190ca2-bc75-41c0-8d71-d01d5728f856-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.608648 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.608137    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/45190ca2-bc75-41c0-8d71-d01d5728f856-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.608648 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.608605    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/45190ca2-bc75-41c0-8d71-d01d5728f856-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.609351 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.608978    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/45190ca2-bc75-41c0-8d71-d01d5728f856-config-out\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.609351 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.609224    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/45190ca2-bc75-41c0-8d71-d01d5728f856-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.609351 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.609332    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/45190ca2-bc75-41c0-8d71-d01d5728f856-web-config\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.609570 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.609542    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/45190ca2-bc75-41c0-8d71-d01d5728f856-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.609676 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.609612    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/45190ca2-bc75-41c0-8d71-d01d5728f856-tls-assets\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.609884 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.609860    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/45190ca2-bc75-41c0-8d71-d01d5728f856-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.610024 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.610003    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/45190ca2-bc75-41c0-8d71-d01d5728f856-config-volume\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.617781 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.617753    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbz54\" (UniqueName: \"kubernetes.io/projected/45190ca2-bc75-41c0-8d71-d01d5728f856-kube-api-access-rbz54\") pod \"alertmanager-main-0\" (UID: \"45190ca2-bc75-41c0-8d71-d01d5728f856\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:52.626458 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.626436    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-559686bd8d-9qgwd"
Jul 08 16:35:52.626573 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.626542    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-559686bd8d-9qgwd"
Jul 08 16:35:52.632579 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.632553    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-559686bd8d-9qgwd"
Jul 08 16:35:52.639064 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.639038    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-559686bd8d-9qgwd"
Jul 08 16:35:52.750668 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:52.750581    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 08 16:35:53.249271 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:53.249235    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 08 16:35:53.253670 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:35:53.253636    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45190ca2_bc75_41c0_8d71_d01d5728f856.slice/crio-3b267bf831abb237fb1b2beda580ee8dd55b2807bd39a7b81e0caae129f57ffd WatchSource:0}: Error finding container 3b267bf831abb237fb1b2beda580ee8dd55b2807bd39a7b81e0caae129f57ffd: Status 404 returned error can't find the container with id 3b267bf831abb237fb1b2beda580ee8dd55b2807bd39a7b81e0caae129f57ffd
Jul 08 16:35:53.256692 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:53.256584    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-fsp5w"]
Jul 08 16:35:53.258546 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:53.258514    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ckxc5"]
Jul 08 16:35:53.260925 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:35:53.260896    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podade5e891_8f50_4385_9388_6ebfad50927d.slice/crio-ad4242677055e476a89bb27dddf0e44a86b0a9ee506bffc53a348f571c0e67ec WatchSource:0}: Error finding container ad4242677055e476a89bb27dddf0e44a86b0a9ee506bffc53a348f571c0e67ec: Status 404 returned error can't find the container with id ad4242677055e476a89bb27dddf0e44a86b0a9ee506bffc53a348f571c0e67ec
Jul 08 16:35:53.261230 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:35:53.261209    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ed65374_16ab_405d_96b6_69d3318cc2e7.slice/crio-8639dd90fa610cc7c3eef1e8ff6c771c035cc5895b6714f2210358db8cd88f44 WatchSource:0}: Error finding container 8639dd90fa610cc7c3eef1e8ff6c771c035cc5895b6714f2210358db8cd88f44: Status 404 returned error can't find the container with id 8639dd90fa610cc7c3eef1e8ff6c771c035cc5895b6714f2210358db8cd88f44
Jul 08 16:35:53.576809 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:53.576712    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tpllq"
Jul 08 16:35:53.641019 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:53.640895    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45190ca2-bc75-41c0-8d71-d01d5728f856","Type":"ContainerStarted","Data":"3b267bf831abb237fb1b2beda580ee8dd55b2807bd39a7b81e0caae129f57ffd"}
Jul 08 16:35:53.643066 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:53.643025    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bkqjt" event={"ID":"c98cc144-3322-4f83-b571-9772b5a63d49","Type":"ContainerStarted","Data":"635b95eee1f35a28b7784e1ca82264a8d50e7b5727b1c4c919b09a97f4083746"}
Jul 08 16:35:53.643263 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:53.643247    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:35:53.646166 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:53.646138    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fsp5w" event={"ID":"3ed65374-16ab-405d-96b6-69d3318cc2e7","Type":"ContainerStarted","Data":"4d8113b334d6cb3300b6f15891985b015d2c9c6096e92b40c3c5e433e45d5b81"}
Jul 08 16:35:53.646361 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:53.646197    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fsp5w" event={"ID":"3ed65374-16ab-405d-96b6-69d3318cc2e7","Type":"ContainerStarted","Data":"b4cb45a7f01f164d7b6353021f20dbe946a430e337f3e6c2e8a52e81fcfb09be"}
Jul 08 16:35:53.646361 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:53.646224    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fsp5w" event={"ID":"3ed65374-16ab-405d-96b6-69d3318cc2e7","Type":"ContainerStarted","Data":"8639dd90fa610cc7c3eef1e8ff6c771c035cc5895b6714f2210358db8cd88f44"}
Jul 08 16:35:53.647890 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:53.647867    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4vtkl" event={"ID":"8eb2abc0-b610-4f3b-b6dc-02131ad29e35","Type":"ContainerStarted","Data":"2222168b3df2b385ded3f8507925f0f23228e77f77e5d7c835fe9a14f55df8b1"}
Jul 08 16:35:53.649715 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:53.649686    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fd6f4" event={"ID":"8f30902a-26b7-48d4-92f1-d0ca067dab66","Type":"ContainerStarted","Data":"6342244e51ca4431c00210dfadc70e849fc09887b550b75d24fc7dbb0ff209f7"}
Jul 08 16:35:53.651269 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:53.651238    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ckxc5" event={"ID":"ade5e891-8f50-4385-9388-6ebfad50927d","Type":"ContainerStarted","Data":"ad4242677055e476a89bb27dddf0e44a86b0a9ee506bffc53a348f571c0e67ec"}
Jul 08 16:35:53.678601 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:53.678546    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bkqjt" podStartSLOduration=67.188978496 podStartE2EDuration="1m12.678524928s" totalImagesPullingTime="5.489546432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:53.661519213 +0000 UTC m=+73.002484291" watchObservedRunningTime="2026-07-08 16:35:53.678524928 +0000 UTC m=+73.019490005"
Jul 08 16:35:53.679277 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:53.679246    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4vtkl" podStartSLOduration=67.320796942 podStartE2EDuration="1m12.679234139s" totalImagesPullingTime="5.358437197s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 16:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:53.677606332 +0000 UTC m=+73.018571413" watchObservedRunningTime="2026-07-08 16:35:53.679234139 +0000 UTC m=+73.020199221"
Jul 08 16:35:54.419892 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.419860    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"]
Jul 08 16:35:54.423532 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.423509    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"
Jul 08 16:35:54.426119 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.426098    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 08 16:35:54.426521 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.426428    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-vln7g\""
Jul 08 16:35:54.426521 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.426484    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 08 16:35:54.426819 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.426553    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 08 16:35:54.426819 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.426571    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 08 16:35:54.426819 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.426602    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-1b40n3sfb6mb8\""
Jul 08 16:35:54.429994 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.426476    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 08 16:35:54.436369 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.436303    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"]
Jul 08 16:35:54.520586 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.520528    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7cdd4d5f-4ac1-4dc0-8839-47f474392834-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-89b9f5c4c-4nd2s\" (UID: \"7cdd4d5f-4ac1-4dc0-8839-47f474392834\") " pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"
Jul 08 16:35:54.520761 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.520625    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7cdd4d5f-4ac1-4dc0-8839-47f474392834-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-89b9f5c4c-4nd2s\" (UID: \"7cdd4d5f-4ac1-4dc0-8839-47f474392834\") " pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"
Jul 08 16:35:54.520761 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.520693    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7cdd4d5f-4ac1-4dc0-8839-47f474392834-secret-thanos-querier-tls\") pod \"thanos-querier-89b9f5c4c-4nd2s\" (UID: \"7cdd4d5f-4ac1-4dc0-8839-47f474392834\") " pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"
Jul 08 16:35:54.520761 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.520734    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrv7v\" (UniqueName: \"kubernetes.io/projected/7cdd4d5f-4ac1-4dc0-8839-47f474392834-kube-api-access-wrv7v\") pod \"thanos-querier-89b9f5c4c-4nd2s\" (UID: \"7cdd4d5f-4ac1-4dc0-8839-47f474392834\") " pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"
Jul 08 16:35:54.520761 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.520761    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7cdd4d5f-4ac1-4dc0-8839-47f474392834-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-89b9f5c4c-4nd2s\" (UID: \"7cdd4d5f-4ac1-4dc0-8839-47f474392834\") " pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"
Jul 08 16:35:54.520973 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.520787    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7cdd4d5f-4ac1-4dc0-8839-47f474392834-metrics-client-ca\") pod \"thanos-querier-89b9f5c4c-4nd2s\" (UID: \"7cdd4d5f-4ac1-4dc0-8839-47f474392834\") " pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"
Jul 08 16:35:54.520973 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.520828    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7cdd4d5f-4ac1-4dc0-8839-47f474392834-secret-grpc-tls\") pod \"thanos-querier-89b9f5c4c-4nd2s\" (UID: \"7cdd4d5f-4ac1-4dc0-8839-47f474392834\") " pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"
Jul 08 16:35:54.521034 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.521006    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7cdd4d5f-4ac1-4dc0-8839-47f474392834-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-89b9f5c4c-4nd2s\" (UID: \"7cdd4d5f-4ac1-4dc0-8839-47f474392834\") " pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"
Jul 08 16:35:54.622578 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.621828    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7cdd4d5f-4ac1-4dc0-8839-47f474392834-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-89b9f5c4c-4nd2s\" (UID: \"7cdd4d5f-4ac1-4dc0-8839-47f474392834\") " pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"
Jul 08 16:35:54.622578 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.621884    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7cdd4d5f-4ac1-4dc0-8839-47f474392834-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-89b9f5c4c-4nd2s\" (UID: \"7cdd4d5f-4ac1-4dc0-8839-47f474392834\") " pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"
Jul 08 16:35:54.622578 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.621930    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7cdd4d5f-4ac1-4dc0-8839-47f474392834-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-89b9f5c4c-4nd2s\" (UID: \"7cdd4d5f-4ac1-4dc0-8839-47f474392834\") " pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"
Jul 08 16:35:54.622578 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.621976    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7cdd4d5f-4ac1-4dc0-8839-47f474392834-secret-thanos-querier-tls\") pod \"thanos-querier-89b9f5c4c-4nd2s\" (UID: \"7cdd4d5f-4ac1-4dc0-8839-47f474392834\") " pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"
Jul 08 16:35:54.622578 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.622010    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrv7v\" (UniqueName: \"kubernetes.io/projected/7cdd4d5f-4ac1-4dc0-8839-47f474392834-kube-api-access-wrv7v\") pod \"thanos-querier-89b9f5c4c-4nd2s\" (UID: \"7cdd4d5f-4ac1-4dc0-8839-47f474392834\") " pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"
Jul 08 16:35:54.622578 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.622037    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7cdd4d5f-4ac1-4dc0-8839-47f474392834-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-89b9f5c4c-4nd2s\" (UID: \"7cdd4d5f-4ac1-4dc0-8839-47f474392834\") " pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"
Jul 08 16:35:54.622578 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.622060    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7cdd4d5f-4ac1-4dc0-8839-47f474392834-metrics-client-ca\") pod \"thanos-querier-89b9f5c4c-4nd2s\" (UID: \"7cdd4d5f-4ac1-4dc0-8839-47f474392834\") " pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"
Jul 08 16:35:54.622578 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.622089    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7cdd4d5f-4ac1-4dc0-8839-47f474392834-secret-grpc-tls\") pod \"thanos-querier-89b9f5c4c-4nd2s\" (UID: \"7cdd4d5f-4ac1-4dc0-8839-47f474392834\") " pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"
Jul 08 16:35:54.638035 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.638003    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7cdd4d5f-4ac1-4dc0-8839-47f474392834-metrics-client-ca\") pod \"thanos-querier-89b9f5c4c-4nd2s\" (UID: \"7cdd4d5f-4ac1-4dc0-8839-47f474392834\") " pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"
Jul 08 16:35:54.640765 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.640705    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7cdd4d5f-4ac1-4dc0-8839-47f474392834-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-89b9f5c4c-4nd2s\" (UID: \"7cdd4d5f-4ac1-4dc0-8839-47f474392834\") " pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"
Jul 08 16:35:54.641154 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.641128    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7cdd4d5f-4ac1-4dc0-8839-47f474392834-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-89b9f5c4c-4nd2s\" (UID: \"7cdd4d5f-4ac1-4dc0-8839-47f474392834\") " pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"
Jul 08 16:35:54.644403 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.644284    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7cdd4d5f-4ac1-4dc0-8839-47f474392834-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-89b9f5c4c-4nd2s\" (UID: \"7cdd4d5f-4ac1-4dc0-8839-47f474392834\") " pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"
Jul 08 16:35:54.646808 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.646742    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7cdd4d5f-4ac1-4dc0-8839-47f474392834-secret-grpc-tls\") pod \"thanos-querier-89b9f5c4c-4nd2s\" (UID: \"7cdd4d5f-4ac1-4dc0-8839-47f474392834\") " pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"
Jul 08 16:35:54.647774 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.647737    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7cdd4d5f-4ac1-4dc0-8839-47f474392834-secret-thanos-querier-tls\") pod \"thanos-querier-89b9f5c4c-4nd2s\" (UID: \"7cdd4d5f-4ac1-4dc0-8839-47f474392834\") " pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"
Jul 08 16:35:54.648073 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.648047    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7cdd4d5f-4ac1-4dc0-8839-47f474392834-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-89b9f5c4c-4nd2s\" (UID: \"7cdd4d5f-4ac1-4dc0-8839-47f474392834\") " pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"
Jul 08 16:35:54.649874 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.649846    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrv7v\" (UniqueName: \"kubernetes.io/projected/7cdd4d5f-4ac1-4dc0-8839-47f474392834-kube-api-access-wrv7v\") pod \"thanos-querier-89b9f5c4c-4nd2s\" (UID: \"7cdd4d5f-4ac1-4dc0-8839-47f474392834\") " pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"
Jul 08 16:35:54.741734 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.741715    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"
Jul 08 16:35:54.925621 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:54.925592    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"]
Jul 08 16:35:55.207344 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.207286    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5bb99c498f-vpzlq"
Jul 08 16:35:55.207604 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.207580    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5bb99c498f-vpzlq"
Jul 08 16:35:55.217722 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.217695    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5bb99c498f-vpzlq"
Jul 08 16:35:55.268396 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:35:55.268364    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cdd4d5f_4ac1_4dc0_8839_47f474392834.slice/crio-6db584e4f5b425a0e49f164eb72e2225f87396a4ae1e148cafb72b5f7fd41d67 WatchSource:0}: Error finding container 6db584e4f5b425a0e49f164eb72e2225f87396a4ae1e148cafb72b5f7fd41d67: Status 404 returned error can't find the container with id 6db584e4f5b425a0e49f164eb72e2225f87396a4ae1e148cafb72b5f7fd41d67
Jul 08 16:35:55.662604 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.662464    2582 generic.go:356] "Generic (PLEG): container finished" podID="8f30902a-26b7-48d4-92f1-d0ca067dab66" containerID="3f0eec0e58e0f663c4d5fbbd741eb30c3040778f2ee420d68d773e643c3e9f3e" exitCode=0
Jul 08 16:35:55.662604 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.662546    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fd6f4" event={"ID":"8f30902a-26b7-48d4-92f1-d0ca067dab66","Type":"ContainerDied","Data":"3f0eec0e58e0f663c4d5fbbd741eb30c3040778f2ee420d68d773e643c3e9f3e"}
Jul 08 16:35:55.665364 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.665215    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ckxc5" event={"ID":"ade5e891-8f50-4385-9388-6ebfad50927d","Type":"ContainerStarted","Data":"b88a044c6acd9fe63cd452ae30f043ada5df097534a7877c7d3f824418f56c47"}
Jul 08 16:35:55.665364 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.665268    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ckxc5" event={"ID":"ade5e891-8f50-4385-9388-6ebfad50927d","Type":"ContainerStarted","Data":"42bb9f8001b15ab03b352bf1987cdd261c6d70e66f1a9702e7ec34f724f8d8b7"}
Jul 08 16:35:55.665364 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.665299    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ckxc5" event={"ID":"ade5e891-8f50-4385-9388-6ebfad50927d","Type":"ContainerStarted","Data":"dce70c3a4f09c4f28ed5e29a671583c4361348c3831e6b5156ac0c3d2f3ab3c4"}
Jul 08 16:35:55.667556 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.667529    2582 generic.go:356] "Generic (PLEG): container finished" podID="45190ca2-bc75-41c0-8d71-d01d5728f856" containerID="acc24e0bb35cf2d57eb8eefe58f3268268b9b947370698a5b342f1d9e5be1232" exitCode=0
Jul 08 16:35:55.667637 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.667608    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45190ca2-bc75-41c0-8d71-d01d5728f856","Type":"ContainerDied","Data":"acc24e0bb35cf2d57eb8eefe58f3268268b9b947370698a5b342f1d9e5be1232"}
Jul 08 16:35:55.671167 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.671143    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fsp5w" event={"ID":"3ed65374-16ab-405d-96b6-69d3318cc2e7","Type":"ContainerStarted","Data":"aaf5b97e4ef648386d4df9a473bbee67a13ba672b09ac24a0fee26ab811d0ec9"}
Jul 08 16:35:55.672891 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.672861    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s" event={"ID":"7cdd4d5f-4ac1-4dc0-8839-47f474392834","Type":"ContainerStarted","Data":"6db584e4f5b425a0e49f164eb72e2225f87396a4ae1e148cafb72b5f7fd41d67"}
Jul 08 16:35:55.678170 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.677848    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5bb99c498f-vpzlq"
Jul 08 16:35:55.722762 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.722668    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ckxc5" podStartSLOduration=2.673169567 podStartE2EDuration="4.722648032s" totalImagesPullingTime="2.049478465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:35:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:55.721368622 +0000 UTC m=+75.062333700" watchObservedRunningTime="2026-07-08 16:35:55.722648032 +0000 UTC m=+75.063613112"
Jul 08 16:35:55.742759 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.742528    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-559686bd8d-9qgwd"]
Jul 08 16:35:55.751996 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.751801    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fsp5w" podStartSLOduration=3.399094947 podStartE2EDuration="4.751780475s" totalImagesPullingTime="1.352685528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:35:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:55.749365454 +0000 UTC m=+75.090330533" watchObservedRunningTime="2026-07-08 16:35:55.751780475 +0000 UTC m=+75.092745554"
Jul 08 16:35:55.798882 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.798850    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6f8c55cbdf-96278"]
Jul 08 16:35:55.803197 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.803173    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278"
Jul 08 16:35:55.807489 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.807411    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 08 16:35:55.807489 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.807394    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 08 16:35:55.807900 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.807877    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 08 16:35:55.808072 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.808000    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-22lsu8qqrkpc0\""
Jul 08 16:35:55.808681 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.808502    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 08 16:35:55.808681 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.808594    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-7hd4q\""
Jul 08 16:35:55.817119 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.817084    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6f8c55cbdf-96278"]
Jul 08 16:35:55.845568 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.845403    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a77c65f3-98f6-49b6-bbb9-b64e0906f14e-secret-metrics-server-tls\") pod \"metrics-server-6f8c55cbdf-96278\" (UID: \"a77c65f3-98f6-49b6-bbb9-b64e0906f14e\") " pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278"
Jul 08 16:35:55.845568 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.845450    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a77c65f3-98f6-49b6-bbb9-b64e0906f14e-audit-log\") pod \"metrics-server-6f8c55cbdf-96278\" (UID: \"a77c65f3-98f6-49b6-bbb9-b64e0906f14e\") " pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278"
Jul 08 16:35:55.845568 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.845487    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a77c65f3-98f6-49b6-bbb9-b64e0906f14e-secret-metrics-server-client-certs\") pod \"metrics-server-6f8c55cbdf-96278\" (UID: \"a77c65f3-98f6-49b6-bbb9-b64e0906f14e\") " pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278"
Jul 08 16:35:55.845786 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.845573    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a77c65f3-98f6-49b6-bbb9-b64e0906f14e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6f8c55cbdf-96278\" (UID: \"a77c65f3-98f6-49b6-bbb9-b64e0906f14e\") " pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278"
Jul 08 16:35:55.845786 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.845600    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a77c65f3-98f6-49b6-bbb9-b64e0906f14e-client-ca-bundle\") pod \"metrics-server-6f8c55cbdf-96278\" (UID: \"a77c65f3-98f6-49b6-bbb9-b64e0906f14e\") " pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278"
Jul 08 16:35:55.845786 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.845653    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhnnz\" (UniqueName: \"kubernetes.io/projected/a77c65f3-98f6-49b6-bbb9-b64e0906f14e-kube-api-access-xhnnz\") pod \"metrics-server-6f8c55cbdf-96278\" (UID: \"a77c65f3-98f6-49b6-bbb9-b64e0906f14e\") " pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278"
Jul 08 16:35:55.845786 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.845703    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a77c65f3-98f6-49b6-bbb9-b64e0906f14e-metrics-server-audit-profiles\") pod \"metrics-server-6f8c55cbdf-96278\" (UID: \"a77c65f3-98f6-49b6-bbb9-b64e0906f14e\") " pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278"
Jul 08 16:35:55.946877 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.946843    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a77c65f3-98f6-49b6-bbb9-b64e0906f14e-secret-metrics-server-tls\") pod \"metrics-server-6f8c55cbdf-96278\" (UID: \"a77c65f3-98f6-49b6-bbb9-b64e0906f14e\") " pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278"
Jul 08 16:35:55.946877 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.946889    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a77c65f3-98f6-49b6-bbb9-b64e0906f14e-audit-log\") pod \"metrics-server-6f8c55cbdf-96278\" (UID: \"a77c65f3-98f6-49b6-bbb9-b64e0906f14e\") " pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278"
Jul 08 16:35:55.947103 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.946922    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a77c65f3-98f6-49b6-bbb9-b64e0906f14e-secret-metrics-server-client-certs\") pod \"metrics-server-6f8c55cbdf-96278\" (UID: \"a77c65f3-98f6-49b6-bbb9-b64e0906f14e\") " pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278"
Jul 08 16:35:55.947103 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.946983    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a77c65f3-98f6-49b6-bbb9-b64e0906f14e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6f8c55cbdf-96278\" (UID: \"a77c65f3-98f6-49b6-bbb9-b64e0906f14e\") " pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278"
Jul 08 16:35:55.947103 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.947005    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a77c65f3-98f6-49b6-bbb9-b64e0906f14e-client-ca-bundle\") pod \"metrics-server-6f8c55cbdf-96278\" (UID: \"a77c65f3-98f6-49b6-bbb9-b64e0906f14e\") " pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278"
Jul 08 16:35:55.947103 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.947031    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhnnz\" (UniqueName: \"kubernetes.io/projected/a77c65f3-98f6-49b6-bbb9-b64e0906f14e-kube-api-access-xhnnz\") pod \"metrics-server-6f8c55cbdf-96278\" (UID: \"a77c65f3-98f6-49b6-bbb9-b64e0906f14e\") " pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278"
Jul 08 16:35:55.947103 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.947076    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a77c65f3-98f6-49b6-bbb9-b64e0906f14e-metrics-server-audit-profiles\") pod \"metrics-server-6f8c55cbdf-96278\" (UID: \"a77c65f3-98f6-49b6-bbb9-b64e0906f14e\") " pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278"
Jul 08 16:35:55.947748 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.947716    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a77c65f3-98f6-49b6-bbb9-b64e0906f14e-audit-log\") pod \"metrics-server-6f8c55cbdf-96278\" (UID: \"a77c65f3-98f6-49b6-bbb9-b64e0906f14e\") " pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278"
Jul 08 16:35:55.947997 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.947977    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a77c65f3-98f6-49b6-bbb9-b64e0906f14e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6f8c55cbdf-96278\" (UID: \"a77c65f3-98f6-49b6-bbb9-b64e0906f14e\") " pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278"
Jul 08 16:35:55.948121 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.948100    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a77c65f3-98f6-49b6-bbb9-b64e0906f14e-metrics-server-audit-profiles\") pod \"metrics-server-6f8c55cbdf-96278\" (UID: \"a77c65f3-98f6-49b6-bbb9-b64e0906f14e\") " pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278"
Jul 08 16:35:55.950529 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.950485    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a77c65f3-98f6-49b6-bbb9-b64e0906f14e-secret-metrics-server-tls\") pod \"metrics-server-6f8c55cbdf-96278\" (UID: \"a77c65f3-98f6-49b6-bbb9-b64e0906f14e\") " pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278"
Jul 08 16:35:55.950645 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.950619    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a77c65f3-98f6-49b6-bbb9-b64e0906f14e-client-ca-bundle\") pod \"metrics-server-6f8c55cbdf-96278\" (UID: \"a77c65f3-98f6-49b6-bbb9-b64e0906f14e\") " pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278"
Jul 08 16:35:55.950703 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.950643    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a77c65f3-98f6-49b6-bbb9-b64e0906f14e-secret-metrics-server-client-certs\") pod \"metrics-server-6f8c55cbdf-96278\" (UID: \"a77c65f3-98f6-49b6-bbb9-b64e0906f14e\") " pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278"
Jul 08 16:35:55.958343 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:55.958300    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhnnz\" (UniqueName: \"kubernetes.io/projected/a77c65f3-98f6-49b6-bbb9-b64e0906f14e-kube-api-access-xhnnz\") pod \"metrics-server-6f8c55cbdf-96278\" (UID: \"a77c65f3-98f6-49b6-bbb9-b64e0906f14e\") " pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278"
Jul 08 16:35:56.120979 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:56.120894    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-btd5p"]
Jul 08 16:35:56.125657 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:56.125631    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-btd5p"
Jul 08 16:35:56.128075 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:56.128049    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 08 16:35:56.128350 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:56.128306    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-j8fzm\""
Jul 08 16:35:56.131726 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:56.131524    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278"
Jul 08 16:35:56.133284 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:56.133221    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-btd5p"]
Jul 08 16:35:56.250043 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:56.250006    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzr2m\" (UniqueName: \"kubernetes.io/projected/3f03d25a-b3a9-4497-9b40-22e97ae5fcf6-kube-api-access-kzr2m\") pod \"monitoring-plugin-d5dcff7cf-btd5p\" (UID: \"3f03d25a-b3a9-4497-9b40-22e97ae5fcf6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-btd5p"
Jul 08 16:35:56.250239 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:56.250084    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3f03d25a-b3a9-4497-9b40-22e97ae5fcf6-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-btd5p\" (UID: \"3f03d25a-b3a9-4497-9b40-22e97ae5fcf6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-btd5p"
Jul 08 16:35:56.290201 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:56.290169    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6f8c55cbdf-96278"]
Jul 08 16:35:56.293573 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:35:56.293548    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda77c65f3_98f6_49b6_bbb9_b64e0906f14e.slice/crio-2bb730470a572d42df40bcaa434265e5b203890cbac7d0d8eb9cde18b73844e4 WatchSource:0}: Error finding container 2bb730470a572d42df40bcaa434265e5b203890cbac7d0d8eb9cde18b73844e4: Status 404 returned error can't find the container with id 2bb730470a572d42df40bcaa434265e5b203890cbac7d0d8eb9cde18b73844e4
Jul 08 16:35:56.351038 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:56.350996    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzr2m\" (UniqueName: \"kubernetes.io/projected/3f03d25a-b3a9-4497-9b40-22e97ae5fcf6-kube-api-access-kzr2m\") pod \"monitoring-plugin-d5dcff7cf-btd5p\" (UID: \"3f03d25a-b3a9-4497-9b40-22e97ae5fcf6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-btd5p"
Jul 08 16:35:56.351214 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:56.351054    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3f03d25a-b3a9-4497-9b40-22e97ae5fcf6-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-btd5p\" (UID: \"3f03d25a-b3a9-4497-9b40-22e97ae5fcf6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-btd5p"
Jul 08 16:35:56.353619 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:56.353579    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3f03d25a-b3a9-4497-9b40-22e97ae5fcf6-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-btd5p\" (UID: \"3f03d25a-b3a9-4497-9b40-22e97ae5fcf6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-btd5p"
Jul 08 16:35:56.366514 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:56.366485    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzr2m\" (UniqueName: \"kubernetes.io/projected/3f03d25a-b3a9-4497-9b40-22e97ae5fcf6-kube-api-access-kzr2m\") pod \"monitoring-plugin-d5dcff7cf-btd5p\" (UID: \"3f03d25a-b3a9-4497-9b40-22e97ae5fcf6\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-btd5p"
Jul 08 16:35:56.440456 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:56.440423    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-btd5p"
Jul 08 16:35:56.681235 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:56.681022    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fd6f4" event={"ID":"8f30902a-26b7-48d4-92f1-d0ca067dab66","Type":"ContainerStarted","Data":"18019b07f3314d4ecd2ec07999d1bed497f212ee3b91a34920b89b102545398f"}
Jul 08 16:35:56.681235 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:56.681095    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fd6f4" event={"ID":"8f30902a-26b7-48d4-92f1-d0ca067dab66","Type":"ContainerStarted","Data":"f464b282804a5de8db970233c89f26d52e61e34d8109909c09b0877f906ed118"}
Jul 08 16:35:56.682872 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:56.682841    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278" event={"ID":"a77c65f3-98f6-49b6-bbb9-b64e0906f14e","Type":"ContainerStarted","Data":"2bb730470a572d42df40bcaa434265e5b203890cbac7d0d8eb9cde18b73844e4"}
Jul 08 16:35:56.708428 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:56.707952    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fd6f4" podStartSLOduration=3.946118013 podStartE2EDuration="5.707931426s" totalImagesPullingTime="1.667001947s" totalInitContainerRuntime="94.811466ms" isStatefulPod=true podCreationTimestamp="2026-07-08 16:35:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:56.705187162 +0000 UTC m=+76.046152242" watchObservedRunningTime="2026-07-08 16:35:56.707931426 +0000 UTC m=+76.048896505"
Jul 08 16:35:57.532868 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.532833    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-btd5p"]
Jul 08 16:35:57.541973 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:35:57.541906    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f03d25a_b3a9_4497_9b40_22e97ae5fcf6.slice/crio-467f1733b0944f8d9b2d1b34ec90fbd8a08e2ff5e03d9caf63d8c765a837d915 WatchSource:0}: Error finding container 467f1733b0944f8d9b2d1b34ec90fbd8a08e2ff5e03d9caf63d8c765a837d915: Status 404 returned error can't find the container with id 467f1733b0944f8d9b2d1b34ec90fbd8a08e2ff5e03d9caf63d8c765a837d915
Jul 08 16:35:57.691198 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.691165    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45190ca2-bc75-41c0-8d71-d01d5728f856","Type":"ContainerStarted","Data":"8b8b1ab382fcd939e420e0dc223154b0fe57ac9730b1337a895596a748b10c39"}
Jul 08 16:35:57.691716 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.691247    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45190ca2-bc75-41c0-8d71-d01d5728f856","Type":"ContainerStarted","Data":"497809d6f6e1407d80fcb69c8c9cdc10449be56d59fa1dde041dd62d87de02d8"}
Jul 08 16:35:57.693458 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.693393    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-btd5p" event={"ID":"3f03d25a-b3a9-4497-9b40-22e97ae5fcf6","Type":"ContainerStarted","Data":"467f1733b0944f8d9b2d1b34ec90fbd8a08e2ff5e03d9caf63d8c765a837d915"}
Jul 08 16:35:57.695775 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.695663    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 16:35:57.700590 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.700555    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s" event={"ID":"7cdd4d5f-4ac1-4dc0-8839-47f474392834","Type":"ContainerStarted","Data":"48d2773c7e1f68dd07b66a83fea12f6a7c998f774b0818d9f55a9cfa5d73d86a"}
Jul 08 16:35:57.700818 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.700794    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s" event={"ID":"7cdd4d5f-4ac1-4dc0-8839-47f474392834","Type":"ContainerStarted","Data":"8d53296757f7b411c1676d134f8b5c7312ec224d2cad51af139272ee085e2945"}
Jul 08 16:35:57.700993 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.700694    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.705543 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.704844    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 08 16:35:57.705543 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.705124    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 08 16:35:57.705903 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.705693    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 08 16:35:57.705903 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.705855    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 08 16:35:57.706345 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.706286    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-p8ghx\""
Jul 08 16:35:57.706345 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.706345    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-71dl88jtg2l0t\""
Jul 08 16:35:57.706595 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.706568    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 08 16:35:57.706665 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.706628    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 08 16:35:57.706866 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.706849    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 08 16:35:57.707421 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.707388    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 08 16:35:57.707421 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.707415    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 08 16:35:57.707943 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.707925    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 08 16:35:57.708241 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.708194    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 08 16:35:57.710593 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.709400    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 08 16:35:57.710593 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.710429    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 08 16:35:57.718238 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.718185    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 08 16:35:57.721732 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.721706    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 16:35:57.766793 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.766757    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bd303057-abe0-43bc-a865-5e735e00ba8d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.767059 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.767034    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bd303057-abe0-43bc-a865-5e735e00ba8d-config\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.767201 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.767079    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd303057-abe0-43bc-a865-5e735e00ba8d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.767201 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.767116    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd303057-abe0-43bc-a865-5e735e00ba8d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.767201 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.767141    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bd303057-abe0-43bc-a865-5e735e00ba8d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.767201 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.767181    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bd303057-abe0-43bc-a865-5e735e00ba8d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.767426 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.767206    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd303057-abe0-43bc-a865-5e735e00ba8d-web-config\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.767426 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.767408    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd303057-abe0-43bc-a865-5e735e00ba8d-config-out\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.767489 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.767452    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bd303057-abe0-43bc-a865-5e735e00ba8d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.767543 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.767487    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd303057-abe0-43bc-a865-5e735e00ba8d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.767584 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.767567    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd303057-abe0-43bc-a865-5e735e00ba8d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.767636 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.767618    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd303057-abe0-43bc-a865-5e735e00ba8d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.767690 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.767641    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bd303057-abe0-43bc-a865-5e735e00ba8d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.767690 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.767673    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc5lf\" (UniqueName: \"kubernetes.io/projected/bd303057-abe0-43bc-a865-5e735e00ba8d-kube-api-access-qc5lf\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.767789 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.767702    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bd303057-abe0-43bc-a865-5e735e00ba8d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.767789 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.767745    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd303057-abe0-43bc-a865-5e735e00ba8d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.767789 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.767768    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bd303057-abe0-43bc-a865-5e735e00ba8d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.768110 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.768090    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bd303057-abe0-43bc-a865-5e735e00ba8d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.768197 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.768141    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd303057-abe0-43bc-a865-5e735e00ba8d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.768197 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.768176    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bd303057-abe0-43bc-a865-5e735e00ba8d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.869689 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.869655    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bd303057-abe0-43bc-a865-5e735e00ba8d-config\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.869877 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.869721    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd303057-abe0-43bc-a865-5e735e00ba8d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.869877 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.869755    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd303057-abe0-43bc-a865-5e735e00ba8d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.869877 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.869781    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bd303057-abe0-43bc-a865-5e735e00ba8d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.870016 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.869845    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bd303057-abe0-43bc-a865-5e735e00ba8d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.870069 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.870041    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd303057-abe0-43bc-a865-5e735e00ba8d-web-config\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.870132 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.870101    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd303057-abe0-43bc-a865-5e735e00ba8d-config-out\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.870285 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.870129    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bd303057-abe0-43bc-a865-5e735e00ba8d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.870285 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.870151    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd303057-abe0-43bc-a865-5e735e00ba8d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.870285 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.870177    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd303057-abe0-43bc-a865-5e735e00ba8d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.870285 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.870214    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd303057-abe0-43bc-a865-5e735e00ba8d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.870285 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.870231    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bd303057-abe0-43bc-a865-5e735e00ba8d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.870285 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.870260    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qc5lf\" (UniqueName: \"kubernetes.io/projected/bd303057-abe0-43bc-a865-5e735e00ba8d-kube-api-access-qc5lf\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.870635 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.870290    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bd303057-abe0-43bc-a865-5e735e00ba8d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.870635 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.870333    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd303057-abe0-43bc-a865-5e735e00ba8d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.870635 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.870359    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bd303057-abe0-43bc-a865-5e735e00ba8d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.871896 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.871039    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bd303057-abe0-43bc-a865-5e735e00ba8d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.871896 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.871128    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd303057-abe0-43bc-a865-5e735e00ba8d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.871896 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.871156    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bd303057-abe0-43bc-a865-5e735e00ba8d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.871896 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.871203    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bd303057-abe0-43bc-a865-5e735e00ba8d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.873366 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.873289    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd303057-abe0-43bc-a865-5e735e00ba8d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.873478 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.873403    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bd303057-abe0-43bc-a865-5e735e00ba8d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.873746 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.873715    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bd303057-abe0-43bc-a865-5e735e00ba8d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.873859 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.873839    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd303057-abe0-43bc-a865-5e735e00ba8d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.874154 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.874126    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bd303057-abe0-43bc-a865-5e735e00ba8d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.874280 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.874250    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd303057-abe0-43bc-a865-5e735e00ba8d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.874724 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.874633    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/bd303057-abe0-43bc-a865-5e735e00ba8d-config\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.875252 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.875229    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd303057-abe0-43bc-a865-5e735e00ba8d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.875643 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.875610    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bd303057-abe0-43bc-a865-5e735e00ba8d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.876028 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.876003    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bd303057-abe0-43bc-a865-5e735e00ba8d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.876140 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.876115    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd303057-abe0-43bc-a865-5e735e00ba8d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.876435 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.876380    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd303057-abe0-43bc-a865-5e735e00ba8d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.876515 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.876453    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd303057-abe0-43bc-a865-5e735e00ba8d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.878620 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.878600    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bd303057-abe0-43bc-a865-5e735e00ba8d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.880302 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.880281    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd303057-abe0-43bc-a865-5e735e00ba8d-config-out\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.880446 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.880358    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bd303057-abe0-43bc-a865-5e735e00ba8d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.880446 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.880423    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd303057-abe0-43bc-a865-5e735e00ba8d-web-config\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.880937 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.880467    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bd303057-abe0-43bc-a865-5e735e00ba8d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.880937 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.880757    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bd303057-abe0-43bc-a865-5e735e00ba8d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:57.884147 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:57.884128    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qc5lf\" (UniqueName: \"kubernetes.io/projected/bd303057-abe0-43bc-a865-5e735e00ba8d-kube-api-access-qc5lf\") pod \"prometheus-k8s-0\" (UID: \"bd303057-abe0-43bc-a865-5e735e00ba8d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:58.019036 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:58.018948    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:35:58.258086 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:58.258042    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 16:35:58.262700 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:35:58.262510    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd303057_abe0_43bc_a865_5e735e00ba8d.slice/crio-8fb53b32fa6ee6c4f77c42ac1f028de3edc0f6f3d1e16c035adc09a863496426 WatchSource:0}: Error finding container 8fb53b32fa6ee6c4f77c42ac1f028de3edc0f6f3d1e16c035adc09a863496426: Status 404 returned error can't find the container with id 8fb53b32fa6ee6c4f77c42ac1f028de3edc0f6f3d1e16c035adc09a863496426
Jul 08 16:35:58.704580 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:58.704538    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s" event={"ID":"7cdd4d5f-4ac1-4dc0-8839-47f474392834","Type":"ContainerStarted","Data":"13a90749f20955284ac5bf4b059bbda228c52ce58c438d4c07bddcbc1645237f"}
Jul 08 16:35:58.706172 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:58.706145    2582 generic.go:356] "Generic (PLEG): container finished" podID="bd303057-abe0-43bc-a865-5e735e00ba8d" containerID="807231f250cb8ac3ee98be9229ab2df62b5aaa5e77c54db386798a08a15e11f6" exitCode=0
Jul 08 16:35:58.706299 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:58.706229    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd303057-abe0-43bc-a865-5e735e00ba8d","Type":"ContainerDied","Data":"807231f250cb8ac3ee98be9229ab2df62b5aaa5e77c54db386798a08a15e11f6"}
Jul 08 16:35:58.706369 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:58.706303    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd303057-abe0-43bc-a865-5e735e00ba8d","Type":"ContainerStarted","Data":"8fb53b32fa6ee6c4f77c42ac1f028de3edc0f6f3d1e16c035adc09a863496426"}
Jul 08 16:35:58.710087 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:58.709926    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45190ca2-bc75-41c0-8d71-d01d5728f856","Type":"ContainerStarted","Data":"75ca91f6171a43c7a2f96a8c946e354adf8365f1fac0bd2c4f32b8db62e81d13"}
Jul 08 16:35:58.710087 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:58.710009    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45190ca2-bc75-41c0-8d71-d01d5728f856","Type":"ContainerStarted","Data":"9267d476a69e1207a6852e4e2731355731190db2216b105629788246f68a145e"}
Jul 08 16:35:58.710087 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:58.710049    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45190ca2-bc75-41c0-8d71-d01d5728f856","Type":"ContainerStarted","Data":"97fe71b3204a4df85bc8805f92c639f565dca81e6da1e693df0c6eef07343be2"}
Jul 08 16:35:58.711848 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:58.711814    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278" event={"ID":"a77c65f3-98f6-49b6-bbb9-b64e0906f14e","Type":"ContainerStarted","Data":"1c74780701ce8135d19d928b4addd0c81751927787b62df6cbb33e7fd8ce0f82"}
Jul 08 16:35:59.719128 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:59.719083    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-btd5p" event={"ID":"3f03d25a-b3a9-4497-9b40-22e97ae5fcf6","Type":"ContainerStarted","Data":"91503c26d37902024badc0e59a2ca7bbb73b1d757848d8ff06b9a37a658bc5ce"}
Jul 08 16:35:59.719618 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:59.719283    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-btd5p"
Jul 08 16:35:59.725966 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:59.725937    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-btd5p"
Jul 08 16:35:59.728434 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:59.726883    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s" event={"ID":"7cdd4d5f-4ac1-4dc0-8839-47f474392834","Type":"ContainerStarted","Data":"7391f8882f0367d8d14d968a8857a6dee700ea89ac8a6faeabcfba95f4057f1e"}
Jul 08 16:35:59.728434 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:59.726971    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s" event={"ID":"7cdd4d5f-4ac1-4dc0-8839-47f474392834","Type":"ContainerStarted","Data":"3509b3a11b7b83a517558b6601829cda7b600f29d9158f265d2c162d0a521768"}
Jul 08 16:35:59.728434 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:59.727027    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s" event={"ID":"7cdd4d5f-4ac1-4dc0-8839-47f474392834","Type":"ContainerStarted","Data":"81b751f869a4f7b80624546df469c882e6347a23c80ccf777ce4018d9e0137c8"}
Jul 08 16:35:59.728434 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:59.728131    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"
Jul 08 16:35:59.732991 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:59.732954    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45190ca2-bc75-41c0-8d71-d01d5728f856","Type":"ContainerStarted","Data":"a19cb93aef00cdf02c50274df38446db2b5dc0f9acd66b8ed7831d8e1af2d519"}
Jul 08 16:35:59.738343 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:59.737556    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278" podStartSLOduration=2.895793189 podStartE2EDuration="4.737539363s" totalImagesPullingTime="1.841746174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:35:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:58.756512426 +0000 UTC m=+78.097477505" watchObservedRunningTime="2026-07-08 16:35:59.737539363 +0000 UTC m=+79.078504441"
Jul 08 16:35:59.739116 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:59.739079    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-btd5p" podStartSLOduration=2.03332902 podStartE2EDuration="3.739067275s" totalImagesPullingTime="1.705738255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:35:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:59.737537701 +0000 UTC m=+79.078502780" watchObservedRunningTime="2026-07-08 16:35:59.739067275 +0000 UTC m=+79.080032353"
Jul 08 16:35:59.763484 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:59.763376    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.634336787 podStartE2EDuration="7.763355089s" totalImagesPullingTime="4.106633105s" totalInitContainerRuntime="22.385197ms" isStatefulPod=false podCreationTimestamp="2026-07-08 16:35:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:59.762404233 +0000 UTC m=+79.103369327" watchObservedRunningTime="2026-07-08 16:35:59.763355089 +0000 UTC m=+79.104320169"
Jul 08 16:35:59.787507 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:35:59.787442    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s" podStartSLOduration=2.5966753110000003 podStartE2EDuration="5.787404453s" totalImagesPullingTime="3.190729142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:35:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:35:59.786815555 +0000 UTC m=+79.127780635" watchObservedRunningTime="2026-07-08 16:35:59.787404453 +0000 UTC m=+79.128369533"
Jul 08 16:36:02.752754 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:02.752717    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd303057-abe0-43bc-a865-5e735e00ba8d","Type":"ContainerStarted","Data":"f3d0b082f45eefac061a8fa5c7891ef2e167c94986a1b423cb1cdf63a6a87f54"}
Jul 08 16:36:02.753216 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:02.752835    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd303057-abe0-43bc-a865-5e735e00ba8d","Type":"ContainerStarted","Data":"03093eb4576e327b0f303067c39c67c40979ef93665a61722563842a7b1d0367"}
Jul 08 16:36:02.753216 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:02.752880    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd303057-abe0-43bc-a865-5e735e00ba8d","Type":"ContainerStarted","Data":"e5994e1a8e5f523a99a697098fcac9970d3e802cd42ec27eafcd43788bef5ca0"}
Jul 08 16:36:02.753216 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:02.752924    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd303057-abe0-43bc-a865-5e735e00ba8d","Type":"ContainerStarted","Data":"6dfb36481d7cb6299ec2b722f449fa6176f398739e8d9cf745fab47f26d14947"}
Jul 08 16:36:02.753216 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:02.752963    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd303057-abe0-43bc-a865-5e735e00ba8d","Type":"ContainerStarted","Data":"383beb3cb5867f0a03f99036c5159d2fc782aedab45199565d6ecfef0e059823"}
Jul 08 16:36:02.753216 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:02.753001    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd303057-abe0-43bc-a865-5e735e00ba8d","Type":"ContainerStarted","Data":"8498f7648bfa1f332a8e664ff790d1b7a7deb45b41f8f8d94804ba9b671b9704"}
Jul 08 16:36:02.786172 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:02.786096    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.7686052979999998 podStartE2EDuration="5.786068903s" totalImagesPullingTime="2.997110817s" totalInitContainerRuntime="20.352788ms" isStatefulPod=false podCreationTimestamp="2026-07-08 16:35:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:36:02.785386193 +0000 UTC m=+82.126351272" watchObservedRunningTime="2026-07-08 16:36:02.786068903 +0000 UTC m=+82.127033982"
Jul 08 16:36:03.019443 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:03.019351    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:36:06.751976 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:06.751946    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-89b9f5c4c-4nd2s"
Jul 08 16:36:16.131694 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:16.131661    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278"
Jul 08 16:36:16.132284 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:16.131719    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278"
Jul 08 16:36:16.636932 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:16.636897    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5bb99c498f-vpzlq"]
Jul 08 16:36:20.782247 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:20.782191    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-559686bd8d-9qgwd" podUID="9ada8a6d-1368-4b5a-8cc0-f9032315ad4b" containerName="console" containerID="cri-o://5f5fa63622c4c44de793d80e8a120c3a6c1fea0256e9ed441dd1972a9189ad5e" gracePeriod=15
Jul 08 16:36:21.566599 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:21.566573    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-559686bd8d-9qgwd"
Jul 08 16:36:21.691915 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:21.691817    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-console-serving-cert\") pod \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\" (UID: \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\") "
Jul 08 16:36:21.691915 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:21.691875    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-console-config\") pod \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\" (UID: \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\") "
Jul 08 16:36:21.691915 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:21.691893    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-console-oauth-config\") pod \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\" (UID: \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\") "
Jul 08 16:36:21.692178 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:21.691940    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2g5q\" (UniqueName: \"kubernetes.io/projected/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-kube-api-access-p2g5q\") pod \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\" (UID: \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\") "
Jul 08 16:36:21.692178 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:21.691955    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-service-ca\") pod \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\" (UID: \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\") "
Jul 08 16:36:21.692178 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:21.692009    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-oauth-serving-cert\") pod \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\" (UID: \"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b\") "
Jul 08 16:36:21.692454 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:21.692425    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-console-config" (OuterVolumeSpecName: "console-config") pod "9ada8a6d-1368-4b5a-8cc0-f9032315ad4b" (UID: "9ada8a6d-1368-4b5a-8cc0-f9032315ad4b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:36:21.692519 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:21.692448    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9ada8a6d-1368-4b5a-8cc0-f9032315ad4b" (UID: "9ada8a6d-1368-4b5a-8cc0-f9032315ad4b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:36:21.692519 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:21.692427    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-service-ca" (OuterVolumeSpecName: "service-ca") pod "9ada8a6d-1368-4b5a-8cc0-f9032315ad4b" (UID: "9ada8a6d-1368-4b5a-8cc0-f9032315ad4b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:36:21.694485 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:21.694451    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9ada8a6d-1368-4b5a-8cc0-f9032315ad4b" (UID: "9ada8a6d-1368-4b5a-8cc0-f9032315ad4b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 16:36:21.694619 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:21.694459    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-kube-api-access-p2g5q" (OuterVolumeSpecName: "kube-api-access-p2g5q") pod "9ada8a6d-1368-4b5a-8cc0-f9032315ad4b" (UID: "9ada8a6d-1368-4b5a-8cc0-f9032315ad4b"). InnerVolumeSpecName "kube-api-access-p2g5q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 16:36:21.694619 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:21.694499    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9ada8a6d-1368-4b5a-8cc0-f9032315ad4b" (UID: "9ada8a6d-1368-4b5a-8cc0-f9032315ad4b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 16:36:21.792787 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:21.792750    2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-console-serving-cert\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:36:21.792787 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:21.792781    2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-console-config\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:36:21.792787 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:21.792790    2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-console-oauth-config\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:36:21.793231 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:21.792799    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2g5q\" (UniqueName: \"kubernetes.io/projected/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-kube-api-access-p2g5q\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:36:21.793231 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:21.792809    2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-service-ca\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:36:21.793231 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:21.792816    2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b-oauth-serving-cert\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:36:21.837405 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:21.837370    2582 generic.go:356] "Generic (PLEG): container finished" podID="9ada8a6d-1368-4b5a-8cc0-f9032315ad4b" containerID="5f5fa63622c4c44de793d80e8a120c3a6c1fea0256e9ed441dd1972a9189ad5e" exitCode=0
Jul 08 16:36:21.837575 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:21.837427    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-559686bd8d-9qgwd" event={"ID":"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b","Type":"ContainerDied","Data":"5f5fa63622c4c44de793d80e8a120c3a6c1fea0256e9ed441dd1972a9189ad5e"}
Jul 08 16:36:21.837575 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:21.837487    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-559686bd8d-9qgwd" event={"ID":"9ada8a6d-1368-4b5a-8cc0-f9032315ad4b","Type":"ContainerDied","Data":"65331c46554c2f7184b8fc44c6a485c24a21d59e5b5ee89a6661765bbe608fb9"}
Jul 08 16:36:21.837575 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:21.837505    2582 scope.go:117] "RemoveContainer" containerID="5f5fa63622c4c44de793d80e8a120c3a6c1fea0256e9ed441dd1972a9189ad5e"
Jul 08 16:36:21.837575 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:21.837430    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-559686bd8d-9qgwd"
Jul 08 16:36:21.848950 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:21.848925    2582 scope.go:117] "RemoveContainer" containerID="5f5fa63622c4c44de793d80e8a120c3a6c1fea0256e9ed441dd1972a9189ad5e"
Jul 08 16:36:21.849254 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:36:21.849231    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f5fa63622c4c44de793d80e8a120c3a6c1fea0256e9ed441dd1972a9189ad5e\": container with ID starting with 5f5fa63622c4c44de793d80e8a120c3a6c1fea0256e9ed441dd1972a9189ad5e not found: ID does not exist" containerID="5f5fa63622c4c44de793d80e8a120c3a6c1fea0256e9ed441dd1972a9189ad5e"
Jul 08 16:36:21.849361 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:21.849264    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f5fa63622c4c44de793d80e8a120c3a6c1fea0256e9ed441dd1972a9189ad5e"} err="failed to get container status \"5f5fa63622c4c44de793d80e8a120c3a6c1fea0256e9ed441dd1972a9189ad5e\": rpc error: code = NotFound desc = could not find container \"5f5fa63622c4c44de793d80e8a120c3a6c1fea0256e9ed441dd1972a9189ad5e\": container with ID starting with 5f5fa63622c4c44de793d80e8a120c3a6c1fea0256e9ed441dd1972a9189ad5e not found: ID does not exist"
Jul 08 16:36:21.859878 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:21.859850    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-559686bd8d-9qgwd"]
Jul 08 16:36:21.863968 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:21.863945    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-559686bd8d-9qgwd"]
Jul 08 16:36:23.216659 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:23.216627    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ada8a6d-1368-4b5a-8cc0-f9032315ad4b" path="/var/lib/kubelet/pods/9ada8a6d-1368-4b5a-8cc0-f9032315ad4b/volumes"
Jul 08 16:36:24.658807 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:24.658780    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bkqjt"
Jul 08 16:36:36.137931 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:36.137894    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278"
Jul 08 16:36:36.141903 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:36.141881    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6f8c55cbdf-96278"
Jul 08 16:36:41.664194 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:41.664140    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5bb99c498f-vpzlq" podUID="b85d7a12-14dc-44f9-8cc8-cc68bfd56125" containerName="console" containerID="cri-o://3274a7f42296057ea85a4bc2f450a177a58b0c46c423c6cccc052414afb0ac66" gracePeriod=15
Jul 08 16:36:41.909588 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:41.909561    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bb99c498f-vpzlq"
Jul 08 16:36:41.920742 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:41.920715    2582 generic.go:356] "Generic (PLEG): container finished" podID="b85d7a12-14dc-44f9-8cc8-cc68bfd56125" containerID="3274a7f42296057ea85a4bc2f450a177a58b0c46c423c6cccc052414afb0ac66" exitCode=0
Jul 08 16:36:41.920902 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:41.920765    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bb99c498f-vpzlq"
Jul 08 16:36:41.920902 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:41.920794    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bb99c498f-vpzlq" event={"ID":"b85d7a12-14dc-44f9-8cc8-cc68bfd56125","Type":"ContainerDied","Data":"3274a7f42296057ea85a4bc2f450a177a58b0c46c423c6cccc052414afb0ac66"}
Jul 08 16:36:41.920902 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:41.920841    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bb99c498f-vpzlq" event={"ID":"b85d7a12-14dc-44f9-8cc8-cc68bfd56125","Type":"ContainerDied","Data":"f0c29988c10b4d92493db01a964642e7128deb99005a6c21adffd641e8412cae"}
Jul 08 16:36:41.920902 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:41.920866    2582 scope.go:117] "RemoveContainer" containerID="3274a7f42296057ea85a4bc2f450a177a58b0c46c423c6cccc052414afb0ac66"
Jul 08 16:36:41.929794 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:41.929716    2582 scope.go:117] "RemoveContainer" containerID="3274a7f42296057ea85a4bc2f450a177a58b0c46c423c6cccc052414afb0ac66"
Jul 08 16:36:41.930216 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:36:41.930187    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3274a7f42296057ea85a4bc2f450a177a58b0c46c423c6cccc052414afb0ac66\": container with ID starting with 3274a7f42296057ea85a4bc2f450a177a58b0c46c423c6cccc052414afb0ac66 not found: ID does not exist" containerID="3274a7f42296057ea85a4bc2f450a177a58b0c46c423c6cccc052414afb0ac66"
Jul 08 16:36:41.930349 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:41.930226    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3274a7f42296057ea85a4bc2f450a177a58b0c46c423c6cccc052414afb0ac66"} err="failed to get container status \"3274a7f42296057ea85a4bc2f450a177a58b0c46c423c6cccc052414afb0ac66\": rpc error: code = NotFound desc = could not find container \"3274a7f42296057ea85a4bc2f450a177a58b0c46c423c6cccc052414afb0ac66\": container with ID starting with 3274a7f42296057ea85a4bc2f450a177a58b0c46c423c6cccc052414afb0ac66 not found: ID does not exist"
Jul 08 16:36:41.964086 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:41.964053    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-oauth-serving-cert\") pod \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\" (UID: \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\") "
Jul 08 16:36:41.964086 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:41.964089    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-service-ca\") pod \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\" (UID: \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\") "
Jul 08 16:36:41.964341 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:41.964127    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zh87z\" (UniqueName: \"kubernetes.io/projected/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-kube-api-access-zh87z\") pod \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\" (UID: \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\") "
Jul 08 16:36:41.964341 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:41.964175    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-trusted-ca-bundle\") pod \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\" (UID: \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\") "
Jul 08 16:36:41.964341 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:41.964196    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-console-config\") pod \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\" (UID: \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\") "
Jul 08 16:36:41.964341 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:41.964228    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-console-oauth-config\") pod \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\" (UID: \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\") "
Jul 08 16:36:41.964341 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:41.964252    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-console-serving-cert\") pod \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\" (UID: \"b85d7a12-14dc-44f9-8cc8-cc68bfd56125\") "
Jul 08 16:36:41.964572 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:41.964499    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-service-ca" (OuterVolumeSpecName: "service-ca") pod "b85d7a12-14dc-44f9-8cc8-cc68bfd56125" (UID: "b85d7a12-14dc-44f9-8cc8-cc68bfd56125"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:36:41.964572 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:41.964555    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b85d7a12-14dc-44f9-8cc8-cc68bfd56125" (UID: "b85d7a12-14dc-44f9-8cc8-cc68bfd56125"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:36:41.964673 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:41.964647    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-console-config" (OuterVolumeSpecName: "console-config") pod "b85d7a12-14dc-44f9-8cc8-cc68bfd56125" (UID: "b85d7a12-14dc-44f9-8cc8-cc68bfd56125"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:36:41.964709 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:41.964666    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b85d7a12-14dc-44f9-8cc8-cc68bfd56125" (UID: "b85d7a12-14dc-44f9-8cc8-cc68bfd56125"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:36:41.966634 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:41.966607    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-kube-api-access-zh87z" (OuterVolumeSpecName: "kube-api-access-zh87z") pod "b85d7a12-14dc-44f9-8cc8-cc68bfd56125" (UID: "b85d7a12-14dc-44f9-8cc8-cc68bfd56125"). InnerVolumeSpecName "kube-api-access-zh87z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 16:36:41.966733 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:41.966616    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b85d7a12-14dc-44f9-8cc8-cc68bfd56125" (UID: "b85d7a12-14dc-44f9-8cc8-cc68bfd56125"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 16:36:41.966733 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:41.966672    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b85d7a12-14dc-44f9-8cc8-cc68bfd56125" (UID: "b85d7a12-14dc-44f9-8cc8-cc68bfd56125"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 16:36:42.064997 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:42.064958    2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-oauth-serving-cert\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:36:42.064997 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:42.064994    2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-service-ca\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:36:42.064997 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:42.065004    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zh87z\" (UniqueName: \"kubernetes.io/projected/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-kube-api-access-zh87z\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:36:42.065231 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:42.065013    2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-trusted-ca-bundle\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:36:42.065231 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:42.065021    2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-console-config\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:36:42.065231 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:42.065029    2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-console-oauth-config\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:36:42.065231 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:42.065038    2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b85d7a12-14dc-44f9-8cc8-cc68bfd56125-console-serving-cert\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:36:42.242574 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:42.242541    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5bb99c498f-vpzlq"]
Jul 08 16:36:42.246339 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:42.246299    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5bb99c498f-vpzlq"]
Jul 08 16:36:43.217714 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:43.217682    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b85d7a12-14dc-44f9-8cc8-cc68bfd56125" path="/var/lib/kubelet/pods/b85d7a12-14dc-44f9-8cc8-cc68bfd56125/volumes"
Jul 08 16:36:58.019650 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:58.019604    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:36:58.039568 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:58.039542    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:36:59.010962 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:36:59.010933    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 16:37:15.719885 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.719840    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-8df798f9d-kkfjw"]
Jul 08 16:37:15.720501 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.720234    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ada8a6d-1368-4b5a-8cc0-f9032315ad4b" containerName="console"
Jul 08 16:37:15.720501 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.720252    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ada8a6d-1368-4b5a-8cc0-f9032315ad4b" containerName="console"
Jul 08 16:37:15.720501 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.720273    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b85d7a12-14dc-44f9-8cc8-cc68bfd56125" containerName="console"
Jul 08 16:37:15.720501 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.720280    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b85d7a12-14dc-44f9-8cc8-cc68bfd56125" containerName="console"
Jul 08 16:37:15.720501 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.720359    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b85d7a12-14dc-44f9-8cc8-cc68bfd56125" containerName="console"
Jul 08 16:37:15.720501 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.720369    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ada8a6d-1368-4b5a-8cc0-f9032315ad4b" containerName="console"
Jul 08 16:37:15.723053 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.723031    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw"
Jul 08 16:37:15.726409 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.726370    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-mc8pj\""
Jul 08 16:37:15.726548 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.726426    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 08 16:37:15.726618 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.726560    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 08 16:37:15.726690 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.726613    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 08 16:37:15.726690 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.726561    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 08 16:37:15.726791 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.726720    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 08 16:37:15.733144 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.733097    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 08 16:37:15.736356 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.736335    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-8df798f9d-kkfjw"]
Jul 08 16:37:15.764852 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.764774    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9c773b06-163f-4e08-874e-a607cdf4cb3a-secret-telemeter-client\") pod \"telemeter-client-8df798f9d-kkfjw\" (UID: \"9c773b06-163f-4e08-874e-a607cdf4cb3a\") " pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw"
Jul 08 16:37:15.764852 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.764823    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c773b06-163f-4e08-874e-a607cdf4cb3a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8df798f9d-kkfjw\" (UID: \"9c773b06-163f-4e08-874e-a607cdf4cb3a\") " pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw"
Jul 08 16:37:15.765100 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.764887    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c773b06-163f-4e08-874e-a607cdf4cb3a-serving-certs-ca-bundle\") pod \"telemeter-client-8df798f9d-kkfjw\" (UID: \"9c773b06-163f-4e08-874e-a607cdf4cb3a\") " pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw"
Jul 08 16:37:15.765100 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.764944    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9c773b06-163f-4e08-874e-a607cdf4cb3a-federate-client-tls\") pod \"telemeter-client-8df798f9d-kkfjw\" (UID: \"9c773b06-163f-4e08-874e-a607cdf4cb3a\") " pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw"
Jul 08 16:37:15.765100 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.765004    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9c773b06-163f-4e08-874e-a607cdf4cb3a-telemeter-client-tls\") pod \"telemeter-client-8df798f9d-kkfjw\" (UID: \"9c773b06-163f-4e08-874e-a607cdf4cb3a\") " pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw"
Jul 08 16:37:15.765100 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.765039    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c773b06-163f-4e08-874e-a607cdf4cb3a-metrics-client-ca\") pod \"telemeter-client-8df798f9d-kkfjw\" (UID: \"9c773b06-163f-4e08-874e-a607cdf4cb3a\") " pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw"
Jul 08 16:37:15.765100 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.765079    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m9hr\" (UniqueName: \"kubernetes.io/projected/9c773b06-163f-4e08-874e-a607cdf4cb3a-kube-api-access-4m9hr\") pod \"telemeter-client-8df798f9d-kkfjw\" (UID: \"9c773b06-163f-4e08-874e-a607cdf4cb3a\") " pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw"
Jul 08 16:37:15.765369 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.765156    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9c773b06-163f-4e08-874e-a607cdf4cb3a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8df798f9d-kkfjw\" (UID: \"9c773b06-163f-4e08-874e-a607cdf4cb3a\") " pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw"
Jul 08 16:37:15.866370 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.866336    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c773b06-163f-4e08-874e-a607cdf4cb3a-metrics-client-ca\") pod \"telemeter-client-8df798f9d-kkfjw\" (UID: \"9c773b06-163f-4e08-874e-a607cdf4cb3a\") " pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw"
Jul 08 16:37:15.866564 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.866383    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4m9hr\" (UniqueName: \"kubernetes.io/projected/9c773b06-163f-4e08-874e-a607cdf4cb3a-kube-api-access-4m9hr\") pod \"telemeter-client-8df798f9d-kkfjw\" (UID: \"9c773b06-163f-4e08-874e-a607cdf4cb3a\") " pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw"
Jul 08 16:37:15.866564 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.866409    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9c773b06-163f-4e08-874e-a607cdf4cb3a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8df798f9d-kkfjw\" (UID: \"9c773b06-163f-4e08-874e-a607cdf4cb3a\") " pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw"
Jul 08 16:37:15.866564 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.866528    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9c773b06-163f-4e08-874e-a607cdf4cb3a-secret-telemeter-client\") pod \"telemeter-client-8df798f9d-kkfjw\" (UID: \"9c773b06-163f-4e08-874e-a607cdf4cb3a\") " pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw"
Jul 08 16:37:15.866732 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.866577    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c773b06-163f-4e08-874e-a607cdf4cb3a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8df798f9d-kkfjw\" (UID: \"9c773b06-163f-4e08-874e-a607cdf4cb3a\") " pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw"
Jul 08 16:37:15.866732 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.866617    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c773b06-163f-4e08-874e-a607cdf4cb3a-serving-certs-ca-bundle\") pod \"telemeter-client-8df798f9d-kkfjw\" (UID: \"9c773b06-163f-4e08-874e-a607cdf4cb3a\") " pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw"
Jul 08 16:37:15.866732 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.866673    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9c773b06-163f-4e08-874e-a607cdf4cb3a-federate-client-tls\") pod \"telemeter-client-8df798f9d-kkfjw\" (UID: \"9c773b06-163f-4e08-874e-a607cdf4cb3a\") " pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw"
Jul 08 16:37:15.866874 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.866747    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9c773b06-163f-4e08-874e-a607cdf4cb3a-telemeter-client-tls\") pod \"telemeter-client-8df798f9d-kkfjw\" (UID: \"9c773b06-163f-4e08-874e-a607cdf4cb3a\") " pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw"
Jul 08 16:37:15.867355 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.867302    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c773b06-163f-4e08-874e-a607cdf4cb3a-serving-certs-ca-bundle\") pod \"telemeter-client-8df798f9d-kkfjw\" (UID: \"9c773b06-163f-4e08-874e-a607cdf4cb3a\") " pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw"
Jul 08 16:37:15.867355 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.867303    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c773b06-163f-4e08-874e-a607cdf4cb3a-metrics-client-ca\") pod \"telemeter-client-8df798f9d-kkfjw\" (UID: \"9c773b06-163f-4e08-874e-a607cdf4cb3a\") " pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw"
Jul 08 16:37:15.867558 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.867506    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c773b06-163f-4e08-874e-a607cdf4cb3a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8df798f9d-kkfjw\" (UID: \"9c773b06-163f-4e08-874e-a607cdf4cb3a\") " pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw"
Jul 08 16:37:15.868982 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.868952    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9c773b06-163f-4e08-874e-a607cdf4cb3a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8df798f9d-kkfjw\" (UID: \"9c773b06-163f-4e08-874e-a607cdf4cb3a\") " pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw"
Jul 08 16:37:15.869138 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.869122    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9c773b06-163f-4e08-874e-a607cdf4cb3a-secret-telemeter-client\") pod \"telemeter-client-8df798f9d-kkfjw\" (UID: \"9c773b06-163f-4e08-874e-a607cdf4cb3a\") " pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw"
Jul 08 16:37:15.869707 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.869687    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9c773b06-163f-4e08-874e-a607cdf4cb3a-federate-client-tls\") pod \"telemeter-client-8df798f9d-kkfjw\" (UID: \"9c773b06-163f-4e08-874e-a607cdf4cb3a\") " pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw"
Jul 08 16:37:15.869789 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.869698    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9c773b06-163f-4e08-874e-a607cdf4cb3a-telemeter-client-tls\") pod \"telemeter-client-8df798f9d-kkfjw\" (UID: \"9c773b06-163f-4e08-874e-a607cdf4cb3a\") " pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw"
Jul 08 16:37:15.875395 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:15.875372    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m9hr\" (UniqueName: \"kubernetes.io/projected/9c773b06-163f-4e08-874e-a607cdf4cb3a-kube-api-access-4m9hr\") pod \"telemeter-client-8df798f9d-kkfjw\" (UID: \"9c773b06-163f-4e08-874e-a607cdf4cb3a\") " pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw"
Jul 08 16:37:16.036711 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:16.036624    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw"
Jul 08 16:37:16.178014 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:16.177938    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-8df798f9d-kkfjw"]
Jul 08 16:37:16.180433 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:37:16.180402    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c773b06_163f_4e08_874e_a607cdf4cb3a.slice/crio-67c530b1743fb78186b6fd6b19c5a8fc231eae3be26bf6a4f40f1f7fa8e3ee85 WatchSource:0}: Error finding container 67c530b1743fb78186b6fd6b19c5a8fc231eae3be26bf6a4f40f1f7fa8e3ee85: Status 404 returned error can't find the container with id 67c530b1743fb78186b6fd6b19c5a8fc231eae3be26bf6a4f40f1f7fa8e3ee85
Jul 08 16:37:17.067549 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:17.067508    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw" event={"ID":"9c773b06-163f-4e08-874e-a607cdf4cb3a","Type":"ContainerStarted","Data":"67c530b1743fb78186b6fd6b19c5a8fc231eae3be26bf6a4f40f1f7fa8e3ee85"}
Jul 08 16:37:18.073527 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:18.073439    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw" event={"ID":"9c773b06-163f-4e08-874e-a607cdf4cb3a","Type":"ContainerStarted","Data":"9a99e35206b22b15765ef5dc84c311e97f736c061e8adaef5644572db9dd584a"}
Jul 08 16:37:18.073527 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:18.073494    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw" event={"ID":"9c773b06-163f-4e08-874e-a607cdf4cb3a","Type":"ContainerStarted","Data":"9f49a27415156c9c642c0cae05ac8c146b687932a920637f3139d0bb379a9949"}
Jul 08 16:37:18.073527 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:18.073517    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw" event={"ID":"9c773b06-163f-4e08-874e-a607cdf4cb3a","Type":"ContainerStarted","Data":"3de9c3970cd8ba82d903c356c94329203e235ed3b96b623b920397e9b9f00596"}
Jul 08 16:37:18.098816 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:37:18.098763    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-8df798f9d-kkfjw" podStartSLOduration=1.5110907120000001 podStartE2EDuration="3.098746416s" totalImagesPullingTime="1.587655704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:37:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:37:18.097231268 +0000 UTC m=+157.438196346" watchObservedRunningTime="2026-07-08 16:37:18.098746416 +0000 UTC m=+157.439711493"
Jul 08 16:39:41.062047 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:39:41.062016    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kvhpr_19df7751-4b62-4ae5-92a6-0bbda11fec85/ovn-acl-logging/0.log"
Jul 08 16:39:41.062607 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:39:41.062274    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kvhpr_19df7751-4b62-4ae5-92a6-0bbda11fec85/ovn-acl-logging/0.log"
Jul 08 16:39:41.068658 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:39:41.068640    2582 kubelet.go:1648] "Image garbage collection succeeded"
Jul 08 16:41:14.310064 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:41:14.310034    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-64568bcd49-tn4bw"]
Jul 08 16:41:14.313252 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:41:14.313234    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-tn4bw"
Jul 08 16:41:14.315669 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:41:14.315646    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Jul 08 16:41:14.315789 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:41:14.315652    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 08 16:41:14.316522 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:41:14.316504    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 08 16:41:14.316522 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:41:14.316512    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-jm8n7\""
Jul 08 16:41:14.321418 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:41:14.321396    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-tn4bw"]
Jul 08 16:41:14.334039 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:41:14.334011    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj4xz\" (UniqueName: \"kubernetes.io/projected/61a8704c-1d52-4394-b894-22384e72201a-kube-api-access-xj4xz\") pod \"seaweedfs-64568bcd49-tn4bw\" (UID: \"61a8704c-1d52-4394-b894-22384e72201a\") " pod="kserve/seaweedfs-64568bcd49-tn4bw"
Jul 08 16:41:14.334185 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:41:14.334059    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/61a8704c-1d52-4394-b894-22384e72201a-data\") pod \"seaweedfs-64568bcd49-tn4bw\" (UID: \"61a8704c-1d52-4394-b894-22384e72201a\") " pod="kserve/seaweedfs-64568bcd49-tn4bw"
Jul 08 16:41:14.435101 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:41:14.435058    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj4xz\" (UniqueName: \"kubernetes.io/projected/61a8704c-1d52-4394-b894-22384e72201a-kube-api-access-xj4xz\") pod \"seaweedfs-64568bcd49-tn4bw\" (UID: \"61a8704c-1d52-4394-b894-22384e72201a\") " pod="kserve/seaweedfs-64568bcd49-tn4bw"
Jul 08 16:41:14.435304 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:41:14.435132    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/61a8704c-1d52-4394-b894-22384e72201a-data\") pod \"seaweedfs-64568bcd49-tn4bw\" (UID: \"61a8704c-1d52-4394-b894-22384e72201a\") " pod="kserve/seaweedfs-64568bcd49-tn4bw"
Jul 08 16:41:14.435587 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:41:14.435566    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/61a8704c-1d52-4394-b894-22384e72201a-data\") pod \"seaweedfs-64568bcd49-tn4bw\" (UID: \"61a8704c-1d52-4394-b894-22384e72201a\") " pod="kserve/seaweedfs-64568bcd49-tn4bw"
Jul 08 16:41:14.443984 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:41:14.443960    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj4xz\" (UniqueName: \"kubernetes.io/projected/61a8704c-1d52-4394-b894-22384e72201a-kube-api-access-xj4xz\") pod \"seaweedfs-64568bcd49-tn4bw\" (UID: \"61a8704c-1d52-4394-b894-22384e72201a\") " pod="kserve/seaweedfs-64568bcd49-tn4bw"
Jul 08 16:41:14.623445 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:41:14.623358    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-tn4bw"
Jul 08 16:41:14.761749 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:41:14.761719    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-tn4bw"]
Jul 08 16:41:14.765115 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:41:14.765083    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61a8704c_1d52_4394_b894_22384e72201a.slice/crio-fe1f7f46bbc02b725919180b6495acc1f8284a4f3d68432787b6769afd8ee81f WatchSource:0}: Error finding container fe1f7f46bbc02b725919180b6495acc1f8284a4f3d68432787b6769afd8ee81f: Status 404 returned error can't find the container with id fe1f7f46bbc02b725919180b6495acc1f8284a4f3d68432787b6769afd8ee81f
Jul 08 16:41:14.766394 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:41:14.766372    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 16:41:15.033872 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:41:15.033834    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-tn4bw" event={"ID":"61a8704c-1d52-4394-b894-22384e72201a","Type":"ContainerStarted","Data":"fe1f7f46bbc02b725919180b6495acc1f8284a4f3d68432787b6769afd8ee81f"}
Jul 08 16:41:19.058055 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:41:19.058016    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-tn4bw" event={"ID":"61a8704c-1d52-4394-b894-22384e72201a","Type":"ContainerStarted","Data":"1254df2920ef8a852f7ad0bfe5ba55effd4629dfb01614e3ce501f0160bc8dea"}
Jul 08 16:41:19.058507 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:41:19.058167    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-64568bcd49-tn4bw"
Jul 08 16:41:19.075303 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:41:19.075254    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-64568bcd49-tn4bw" podStartSLOduration=1.720446671 podStartE2EDuration="5.075237685s" totalImagesPullingTime="3.354791014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:41:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:41:19.074243082 +0000 UTC m=+398.415208180" watchObservedRunningTime="2026-07-08 16:41:19.075237685 +0000 UTC m=+398.416202763"
Jul 08 16:41:25.064057 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:41:25.064017    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-64568bcd49-tn4bw"
Jul 08 16:42:40.057986 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:42:40.055135    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-8jv9l"]
Jul 08 16:42:40.060035 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:42:40.060013    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-8jv9l"
Jul 08 16:42:40.066724 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:42:40.066695    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-8jv9l"]
Jul 08 16:42:40.169478 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:42:40.169439    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdcmh\" (UniqueName: \"kubernetes.io/projected/e805780f-a7fa-4a63-83b7-a6b60acaa1dd-kube-api-access-vdcmh\") pod \"s3-init-8jv9l\" (UID: \"e805780f-a7fa-4a63-83b7-a6b60acaa1dd\") " pod="kserve/s3-init-8jv9l"
Jul 08 16:42:40.169664 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:42:40.169530    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e805780f-a7fa-4a63-83b7-a6b60acaa1dd-model-cache\") pod \"s3-init-8jv9l\" (UID: \"e805780f-a7fa-4a63-83b7-a6b60acaa1dd\") " pod="kserve/s3-init-8jv9l"
Jul 08 16:42:40.270664 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:42:40.270628    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e805780f-a7fa-4a63-83b7-a6b60acaa1dd-model-cache\") pod \"s3-init-8jv9l\" (UID: \"e805780f-a7fa-4a63-83b7-a6b60acaa1dd\") " pod="kserve/s3-init-8jv9l"
Jul 08 16:42:40.270806 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:42:40.270689    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdcmh\" (UniqueName: \"kubernetes.io/projected/e805780f-a7fa-4a63-83b7-a6b60acaa1dd-kube-api-access-vdcmh\") pod \"s3-init-8jv9l\" (UID: \"e805780f-a7fa-4a63-83b7-a6b60acaa1dd\") " pod="kserve/s3-init-8jv9l"
Jul 08 16:42:40.270989 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:42:40.270969    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e805780f-a7fa-4a63-83b7-a6b60acaa1dd-model-cache\") pod \"s3-init-8jv9l\" (UID: \"e805780f-a7fa-4a63-83b7-a6b60acaa1dd\") " pod="kserve/s3-init-8jv9l"
Jul 08 16:42:40.279436 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:42:40.279413    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdcmh\" (UniqueName: \"kubernetes.io/projected/e805780f-a7fa-4a63-83b7-a6b60acaa1dd-kube-api-access-vdcmh\") pod \"s3-init-8jv9l\" (UID: \"e805780f-a7fa-4a63-83b7-a6b60acaa1dd\") " pod="kserve/s3-init-8jv9l"
Jul 08 16:42:40.380404 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:42:40.380288    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-8jv9l"
Jul 08 16:42:40.515824 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:42:40.515666    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-8jv9l"]
Jul 08 16:42:40.518521 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:42:40.518493    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode805780f_a7fa_4a63_83b7_a6b60acaa1dd.slice/crio-47f438d5563b51b3abd9056ef33b559de028dac2870742b6643d846ab5c1b874 WatchSource:0}: Error finding container 47f438d5563b51b3abd9056ef33b559de028dac2870742b6643d846ab5c1b874: Status 404 returned error can't find the container with id 47f438d5563b51b3abd9056ef33b559de028dac2870742b6643d846ab5c1b874
Jul 08 16:42:41.380468 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:42:41.380432    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-8jv9l" event={"ID":"e805780f-a7fa-4a63-83b7-a6b60acaa1dd","Type":"ContainerStarted","Data":"47f438d5563b51b3abd9056ef33b559de028dac2870742b6643d846ab5c1b874"}
Jul 08 16:42:46.408839 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:42:46.408749    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-8jv9l" event={"ID":"e805780f-a7fa-4a63-83b7-a6b60acaa1dd","Type":"ContainerStarted","Data":"c78f815fc27b81ec6acb6a3bd865613c8f57369725764e6a951db7973765b8e0"}
Jul 08 16:42:48.419006 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:42:48.418972    2582 generic.go:356] "Generic (PLEG): container finished" podID="e805780f-a7fa-4a63-83b7-a6b60acaa1dd" containerID="c78f815fc27b81ec6acb6a3bd865613c8f57369725764e6a951db7973765b8e0" exitCode=0
Jul 08 16:42:48.419403 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:42:48.419047    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-8jv9l" event={"ID":"e805780f-a7fa-4a63-83b7-a6b60acaa1dd","Type":"ContainerDied","Data":"c78f815fc27b81ec6acb6a3bd865613c8f57369725764e6a951db7973765b8e0"}
Jul 08 16:42:54.449042 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:42:54.449005    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-8jv9l" event={"ID":"e805780f-a7fa-4a63-83b7-a6b60acaa1dd","Type":"ContainerStarted","Data":"c8df1c2d271d3be3d32ea22d6e7938ee44f3744621627681fa2aa6bca06bec17"}
Jul 08 16:42:54.466156 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:42:54.466105    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-8jv9l" podStartSLOduration=1.618263372 podStartE2EDuration="14.466088274s" totalImagesPullingTime="10.805305444s" totalInitContainerRuntime="2.042519458s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:42:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:42:54.464505686 +0000 UTC m=+493.805470761" watchObservedRunningTime="2026-07-08 16:42:54.466088274 +0000 UTC m=+493.807053351"
Jul 08 16:43:59.736622 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:43:59.736583    2582 generic.go:356] "Generic (PLEG): container finished" podID="e805780f-a7fa-4a63-83b7-a6b60acaa1dd" containerID="c8df1c2d271d3be3d32ea22d6e7938ee44f3744621627681fa2aa6bca06bec17" exitCode=0
Jul 08 16:43:59.737064 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:43:59.736631    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-8jv9l" event={"ID":"e805780f-a7fa-4a63-83b7-a6b60acaa1dd","Type":"ContainerDied","Data":"c8df1c2d271d3be3d32ea22d6e7938ee44f3744621627681fa2aa6bca06bec17"}
Jul 08 16:44:00.865036 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:00.865013    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-8jv9l"
Jul 08 16:44:00.962998 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:00.962964    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e805780f-a7fa-4a63-83b7-a6b60acaa1dd-model-cache\") pod \"e805780f-a7fa-4a63-83b7-a6b60acaa1dd\" (UID: \"e805780f-a7fa-4a63-83b7-a6b60acaa1dd\") "
Jul 08 16:44:00.963185 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:00.963032    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vdcmh\" (UniqueName: \"kubernetes.io/projected/e805780f-a7fa-4a63-83b7-a6b60acaa1dd-kube-api-access-vdcmh\") pod \"e805780f-a7fa-4a63-83b7-a6b60acaa1dd\" (UID: \"e805780f-a7fa-4a63-83b7-a6b60acaa1dd\") "
Jul 08 16:44:00.965453 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:00.965425    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e805780f-a7fa-4a63-83b7-a6b60acaa1dd-kube-api-access-vdcmh" (OuterVolumeSpecName: "kube-api-access-vdcmh") pod "e805780f-a7fa-4a63-83b7-a6b60acaa1dd" (UID: "e805780f-a7fa-4a63-83b7-a6b60acaa1dd"). InnerVolumeSpecName "kube-api-access-vdcmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 16:44:00.982335 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:00.982287    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e805780f-a7fa-4a63-83b7-a6b60acaa1dd-model-cache" (OuterVolumeSpecName: "model-cache") pod "e805780f-a7fa-4a63-83b7-a6b60acaa1dd" (UID: "e805780f-a7fa-4a63-83b7-a6b60acaa1dd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 16:44:01.064380 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:01.064270    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e805780f-a7fa-4a63-83b7-a6b60acaa1dd-model-cache\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:44:01.064380 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:01.064298    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdcmh\" (UniqueName: \"kubernetes.io/projected/e805780f-a7fa-4a63-83b7-a6b60acaa1dd-kube-api-access-vdcmh\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:44:01.746685 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:01.746642    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-8jv9l" event={"ID":"e805780f-a7fa-4a63-83b7-a6b60acaa1dd","Type":"ContainerDied","Data":"47f438d5563b51b3abd9056ef33b559de028dac2870742b6643d846ab5c1b874"}
Jul 08 16:44:01.746685 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:01.746676    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-8jv9l"
Jul 08 16:44:01.746685 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:01.746688    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="47f438d5563b51b3abd9056ef33b559de028dac2870742b6643d846ab5c1b874"
Jul 08 16:44:02.501460 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:02.501413    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-hwd68"]
Jul 08 16:44:02.501959 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:02.501939    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e805780f-a7fa-4a63-83b7-a6b60acaa1dd" containerName="s3-init"
Jul 08 16:44:02.502034 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:02.501963    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e805780f-a7fa-4a63-83b7-a6b60acaa1dd" containerName="s3-init"
Jul 08 16:44:02.502034 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:02.501988    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e805780f-a7fa-4a63-83b7-a6b60acaa1dd" containerName="download-hf-model"
Jul 08 16:44:02.502034 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:02.501997    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e805780f-a7fa-4a63-83b7-a6b60acaa1dd" containerName="download-hf-model"
Jul 08 16:44:02.502177 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:02.502091    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e805780f-a7fa-4a63-83b7-a6b60acaa1dd" containerName="s3-init"
Jul 08 16:44:02.505174 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:02.505150    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-hwd68"
Jul 08 16:44:02.507841 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:02.507815    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom-artifact\""
Jul 08 16:44:02.514114 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:02.514087    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-hwd68"]
Jul 08 16:44:02.576363 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:02.576301    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0301a677-dd39-4272-ad0c-50dcbe108c13-data\") pod \"seaweedfs-tls-custom-6644bf5d6c-hwd68\" (UID: \"0301a677-dd39-4272-ad0c-50dcbe108c13\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-hwd68"
Jul 08 16:44:02.576538 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:02.576371    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnxp2\" (UniqueName: \"kubernetes.io/projected/0301a677-dd39-4272-ad0c-50dcbe108c13-kube-api-access-tnxp2\") pod \"seaweedfs-tls-custom-6644bf5d6c-hwd68\" (UID: \"0301a677-dd39-4272-ad0c-50dcbe108c13\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-hwd68"
Jul 08 16:44:02.677793 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:02.677754    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0301a677-dd39-4272-ad0c-50dcbe108c13-data\") pod \"seaweedfs-tls-custom-6644bf5d6c-hwd68\" (UID: \"0301a677-dd39-4272-ad0c-50dcbe108c13\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-hwd68"
Jul 08 16:44:02.677793 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:02.677794    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnxp2\" (UniqueName: \"kubernetes.io/projected/0301a677-dd39-4272-ad0c-50dcbe108c13-kube-api-access-tnxp2\") pod \"seaweedfs-tls-custom-6644bf5d6c-hwd68\" (UID: \"0301a677-dd39-4272-ad0c-50dcbe108c13\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-hwd68"
Jul 08 16:44:02.678123 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:02.678100    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0301a677-dd39-4272-ad0c-50dcbe108c13-data\") pod \"seaweedfs-tls-custom-6644bf5d6c-hwd68\" (UID: \"0301a677-dd39-4272-ad0c-50dcbe108c13\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-hwd68"
Jul 08 16:44:02.687830 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:02.687805    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnxp2\" (UniqueName: \"kubernetes.io/projected/0301a677-dd39-4272-ad0c-50dcbe108c13-kube-api-access-tnxp2\") pod \"seaweedfs-tls-custom-6644bf5d6c-hwd68\" (UID: \"0301a677-dd39-4272-ad0c-50dcbe108c13\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-hwd68"
Jul 08 16:44:02.814902 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:02.814803    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-hwd68"
Jul 08 16:44:02.955907 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:02.955872    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-hwd68"]
Jul 08 16:44:02.959547 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:44:02.959520    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0301a677_dd39_4272_ad0c_50dcbe108c13.slice/crio-3dfc8eb8dcff1b439f417494a8a06fcabe49e4174b1ba28d4b55d8834368e705 WatchSource:0}: Error finding container 3dfc8eb8dcff1b439f417494a8a06fcabe49e4174b1ba28d4b55d8834368e705: Status 404 returned error can't find the container with id 3dfc8eb8dcff1b439f417494a8a06fcabe49e4174b1ba28d4b55d8834368e705
Jul 08 16:44:03.757511 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:03.757462    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-hwd68" event={"ID":"0301a677-dd39-4272-ad0c-50dcbe108c13","Type":"ContainerStarted","Data":"a7a9c066a34e380c87e246587df8acd2b4f27cdef61e361044a526c9b6face0a"}
Jul 08 16:44:03.757971 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:03.757520    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-hwd68" event={"ID":"0301a677-dd39-4272-ad0c-50dcbe108c13","Type":"ContainerStarted","Data":"3dfc8eb8dcff1b439f417494a8a06fcabe49e4174b1ba28d4b55d8834368e705"}
Jul 08 16:44:03.775693 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:03.775639    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-hwd68" podStartSLOduration=1.491883789 podStartE2EDuration="1.77562105s" totalImagesPullingTime="283.737261ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:44:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:44:03.774363629 +0000 UTC m=+563.115328709" watchObservedRunningTime="2026-07-08 16:44:03.77562105 +0000 UTC m=+563.116586127"
Jul 08 16:44:04.853688 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:04.853647    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-hwd68"]
Jul 08 16:44:05.766063 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:05.766018    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-hwd68" podUID="0301a677-dd39-4272-ad0c-50dcbe108c13" containerName="seaweedfs-tls-custom" containerID="cri-o://a7a9c066a34e380c87e246587df8acd2b4f27cdef61e361044a526c9b6face0a" gracePeriod=30
Jul 08 16:44:07.009106 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:07.009077    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-hwd68"
Jul 08 16:44:07.120195 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:07.120095    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0301a677-dd39-4272-ad0c-50dcbe108c13-data\") pod \"0301a677-dd39-4272-ad0c-50dcbe108c13\" (UID: \"0301a677-dd39-4272-ad0c-50dcbe108c13\") "
Jul 08 16:44:07.120195 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:07.120141    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tnxp2\" (UniqueName: \"kubernetes.io/projected/0301a677-dd39-4272-ad0c-50dcbe108c13-kube-api-access-tnxp2\") pod \"0301a677-dd39-4272-ad0c-50dcbe108c13\" (UID: \"0301a677-dd39-4272-ad0c-50dcbe108c13\") "
Jul 08 16:44:07.121308 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:07.121284    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0301a677-dd39-4272-ad0c-50dcbe108c13-data" (OuterVolumeSpecName: "data") pod "0301a677-dd39-4272-ad0c-50dcbe108c13" (UID: "0301a677-dd39-4272-ad0c-50dcbe108c13"). InnerVolumeSpecName "data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 16:44:07.122276 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:07.122252    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0301a677-dd39-4272-ad0c-50dcbe108c13-kube-api-access-tnxp2" (OuterVolumeSpecName: "kube-api-access-tnxp2") pod "0301a677-dd39-4272-ad0c-50dcbe108c13" (UID: "0301a677-dd39-4272-ad0c-50dcbe108c13"). InnerVolumeSpecName "kube-api-access-tnxp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 16:44:07.220786 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:07.220744    2582 reconciler_common.go:299] "Volume detached for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0301a677-dd39-4272-ad0c-50dcbe108c13-data\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:44:07.220786 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:07.220769    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnxp2\" (UniqueName: \"kubernetes.io/projected/0301a677-dd39-4272-ad0c-50dcbe108c13-kube-api-access-tnxp2\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:44:07.775779 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:07.775741    2582 generic.go:356] "Generic (PLEG): container finished" podID="0301a677-dd39-4272-ad0c-50dcbe108c13" containerID="a7a9c066a34e380c87e246587df8acd2b4f27cdef61e361044a526c9b6face0a" exitCode=0
Jul 08 16:44:07.775964 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:07.775791    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-hwd68"
Jul 08 16:44:07.775964 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:07.775824    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-hwd68" event={"ID":"0301a677-dd39-4272-ad0c-50dcbe108c13","Type":"ContainerDied","Data":"a7a9c066a34e380c87e246587df8acd2b4f27cdef61e361044a526c9b6face0a"}
Jul 08 16:44:07.775964 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:07.775878    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-hwd68" event={"ID":"0301a677-dd39-4272-ad0c-50dcbe108c13","Type":"ContainerDied","Data":"3dfc8eb8dcff1b439f417494a8a06fcabe49e4174b1ba28d4b55d8834368e705"}
Jul 08 16:44:07.775964 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:07.775897    2582 scope.go:117] "RemoveContainer" containerID="a7a9c066a34e380c87e246587df8acd2b4f27cdef61e361044a526c9b6face0a"
Jul 08 16:44:07.789946 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:07.789926    2582 scope.go:117] "RemoveContainer" containerID="a7a9c066a34e380c87e246587df8acd2b4f27cdef61e361044a526c9b6face0a"
Jul 08 16:44:07.790290 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:44:07.790265    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7a9c066a34e380c87e246587df8acd2b4f27cdef61e361044a526c9b6face0a\": container with ID starting with a7a9c066a34e380c87e246587df8acd2b4f27cdef61e361044a526c9b6face0a not found: ID does not exist" containerID="a7a9c066a34e380c87e246587df8acd2b4f27cdef61e361044a526c9b6face0a"
Jul 08 16:44:07.790424 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:07.790296    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7a9c066a34e380c87e246587df8acd2b4f27cdef61e361044a526c9b6face0a"} err="failed to get container status \"a7a9c066a34e380c87e246587df8acd2b4f27cdef61e361044a526c9b6face0a\": rpc error: code = NotFound desc = could not find container \"a7a9c066a34e380c87e246587df8acd2b4f27cdef61e361044a526c9b6face0a\": container with ID starting with a7a9c066a34e380c87e246587df8acd2b4f27cdef61e361044a526c9b6face0a not found: ID does not exist"
Jul 08 16:44:07.799399 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:07.799374    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-hwd68"]
Jul 08 16:44:07.803852 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:07.803824    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-hwd68"]
Jul 08 16:44:09.216594 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:09.216561    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0301a677-dd39-4272-ad0c-50dcbe108c13" path="/var/lib/kubelet/pods/0301a677-dd39-4272-ad0c-50dcbe108c13/volumes"
Jul 08 16:44:12.837635 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:12.837602    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-tls-init-custom-srvgm"]
Jul 08 16:44:12.838041 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:12.837928    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0301a677-dd39-4272-ad0c-50dcbe108c13" containerName="seaweedfs-tls-custom"
Jul 08 16:44:12.838041 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:12.837939    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0301a677-dd39-4272-ad0c-50dcbe108c13" containerName="seaweedfs-tls-custom"
Jul 08 16:44:12.838041 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:12.837998    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0301a677-dd39-4272-ad0c-50dcbe108c13" containerName="seaweedfs-tls-custom"
Jul 08 16:44:12.842574 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:12.842557    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-custom-srvgm"
Jul 08 16:44:12.844854 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:12.844830    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom-artifact\""
Jul 08 16:44:12.849925 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:12.849556    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-tls-init-custom-srvgm"]
Jul 08 16:44:12.971700 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:12.971665    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d190cf7-84ec-4cad-8a3c-3e65a78b48d8-model-cache\") pod \"s3-tls-init-custom-srvgm\" (UID: \"0d190cf7-84ec-4cad-8a3c-3e65a78b48d8\") " pod="kserve/s3-tls-init-custom-srvgm"
Jul 08 16:44:12.971700 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:12.971702    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wk2nf\" (UniqueName: \"kubernetes.io/projected/0d190cf7-84ec-4cad-8a3c-3e65a78b48d8-kube-api-access-wk2nf\") pod \"s3-tls-init-custom-srvgm\" (UID: \"0d190cf7-84ec-4cad-8a3c-3e65a78b48d8\") " pod="kserve/s3-tls-init-custom-srvgm"
Jul 08 16:44:13.073149 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:13.073097    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d190cf7-84ec-4cad-8a3c-3e65a78b48d8-model-cache\") pod \"s3-tls-init-custom-srvgm\" (UID: \"0d190cf7-84ec-4cad-8a3c-3e65a78b48d8\") " pod="kserve/s3-tls-init-custom-srvgm"
Jul 08 16:44:13.073365 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:13.073158    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk2nf\" (UniqueName: \"kubernetes.io/projected/0d190cf7-84ec-4cad-8a3c-3e65a78b48d8-kube-api-access-wk2nf\") pod \"s3-tls-init-custom-srvgm\" (UID: \"0d190cf7-84ec-4cad-8a3c-3e65a78b48d8\") " pod="kserve/s3-tls-init-custom-srvgm"
Jul 08 16:44:13.073522 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:13.073501    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d190cf7-84ec-4cad-8a3c-3e65a78b48d8-model-cache\") pod \"s3-tls-init-custom-srvgm\" (UID: \"0d190cf7-84ec-4cad-8a3c-3e65a78b48d8\") " pod="kserve/s3-tls-init-custom-srvgm"
Jul 08 16:44:13.082230 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:13.082208    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wk2nf\" (UniqueName: \"kubernetes.io/projected/0d190cf7-84ec-4cad-8a3c-3e65a78b48d8-kube-api-access-wk2nf\") pod \"s3-tls-init-custom-srvgm\" (UID: \"0d190cf7-84ec-4cad-8a3c-3e65a78b48d8\") " pod="kserve/s3-tls-init-custom-srvgm"
Jul 08 16:44:13.164613 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:13.164589    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-custom-srvgm"
Jul 08 16:44:13.297861 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:13.297838    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-tls-init-custom-srvgm"]
Jul 08 16:44:13.300268 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:44:13.300239    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0d190cf7_84ec_4cad_8a3c_3e65a78b48d8.slice/crio-83b23cec6fb1963e85a880874888a56db20bf0b318147a213a38bebfde1398d1 WatchSource:0}: Error finding container 83b23cec6fb1963e85a880874888a56db20bf0b318147a213a38bebfde1398d1: Status 404 returned error can't find the container with id 83b23cec6fb1963e85a880874888a56db20bf0b318147a213a38bebfde1398d1
Jul 08 16:44:13.800511 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:13.800477    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-srvgm" event={"ID":"0d190cf7-84ec-4cad-8a3c-3e65a78b48d8","Type":"ContainerStarted","Data":"9f64fa6784729c53efe90a905589c216f444aebde40770c638fb51b11edf054f"}
Jul 08 16:44:13.800511 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:13.800517    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-srvgm" event={"ID":"0d190cf7-84ec-4cad-8a3c-3e65a78b48d8","Type":"ContainerStarted","Data":"83b23cec6fb1963e85a880874888a56db20bf0b318147a213a38bebfde1398d1"}
Jul 08 16:44:15.811472 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:15.811438    2582 generic.go:356] "Generic (PLEG): container finished" podID="0d190cf7-84ec-4cad-8a3c-3e65a78b48d8" containerID="9f64fa6784729c53efe90a905589c216f444aebde40770c638fb51b11edf054f" exitCode=0
Jul 08 16:44:15.811959 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:15.811489    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-srvgm" event={"ID":"0d190cf7-84ec-4cad-8a3c-3e65a78b48d8","Type":"ContainerDied","Data":"9f64fa6784729c53efe90a905589c216f444aebde40770c638fb51b11edf054f"}
Jul 08 16:44:16.816988 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:16.816955    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-srvgm" event={"ID":"0d190cf7-84ec-4cad-8a3c-3e65a78b48d8","Type":"ContainerStarted","Data":"dba984f00849713fceb30f1f2a07e39ee36096a954fc9d5a55aaf8b53bc96d08"}
Jul 08 16:44:16.837003 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:16.836942    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-tls-init-custom-srvgm" podStartSLOduration=3.103129759 podStartE2EDuration="4.836922208s" totalImagesPullingTime="0s" totalInitContainerRuntime="1.733792449s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:44:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:44:16.834441605 +0000 UTC m=+576.175406684" watchObservedRunningTime="2026-07-08 16:44:16.836922208 +0000 UTC m=+576.177887287"
Jul 08 16:44:22.845327 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:22.845277    2582 generic.go:356] "Generic (PLEG): container finished" podID="0d190cf7-84ec-4cad-8a3c-3e65a78b48d8" containerID="dba984f00849713fceb30f1f2a07e39ee36096a954fc9d5a55aaf8b53bc96d08" exitCode=0
Jul 08 16:44:22.845706 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:22.845356    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-srvgm" event={"ID":"0d190cf7-84ec-4cad-8a3c-3e65a78b48d8","Type":"ContainerDied","Data":"dba984f00849713fceb30f1f2a07e39ee36096a954fc9d5a55aaf8b53bc96d08"}
Jul 08 16:44:23.994850 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:23.994820    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-custom-srvgm"
Jul 08 16:44:24.176181 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:24.176149    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d190cf7-84ec-4cad-8a3c-3e65a78b48d8-model-cache\") pod \"0d190cf7-84ec-4cad-8a3c-3e65a78b48d8\" (UID: \"0d190cf7-84ec-4cad-8a3c-3e65a78b48d8\") "
Jul 08 16:44:24.176390 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:24.176211    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wk2nf\" (UniqueName: \"kubernetes.io/projected/0d190cf7-84ec-4cad-8a3c-3e65a78b48d8-kube-api-access-wk2nf\") pod \"0d190cf7-84ec-4cad-8a3c-3e65a78b48d8\" (UID: \"0d190cf7-84ec-4cad-8a3c-3e65a78b48d8\") "
Jul 08 16:44:24.178668 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:24.178643    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d190cf7-84ec-4cad-8a3c-3e65a78b48d8-kube-api-access-wk2nf" (OuterVolumeSpecName: "kube-api-access-wk2nf") pod "0d190cf7-84ec-4cad-8a3c-3e65a78b48d8" (UID: "0d190cf7-84ec-4cad-8a3c-3e65a78b48d8"). InnerVolumeSpecName "kube-api-access-wk2nf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 16:44:24.219356 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:24.219306    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d190cf7-84ec-4cad-8a3c-3e65a78b48d8-model-cache" (OuterVolumeSpecName: "model-cache") pod "0d190cf7-84ec-4cad-8a3c-3e65a78b48d8" (UID: "0d190cf7-84ec-4cad-8a3c-3e65a78b48d8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 16:44:24.276862 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:24.276826    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d190cf7-84ec-4cad-8a3c-3e65a78b48d8-model-cache\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:44:24.276862 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:24.276854    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wk2nf\" (UniqueName: \"kubernetes.io/projected/0d190cf7-84ec-4cad-8a3c-3e65a78b48d8-kube-api-access-wk2nf\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:44:24.854561 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:24.854535    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-custom-srvgm"
Jul 08 16:44:24.854561 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:24.854558    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-srvgm" event={"ID":"0d190cf7-84ec-4cad-8a3c-3e65a78b48d8","Type":"ContainerDied","Data":"83b23cec6fb1963e85a880874888a56db20bf0b318147a213a38bebfde1398d1"}
Jul 08 16:44:24.854850 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:24.854593    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="83b23cec6fb1963e85a880874888a56db20bf0b318147a213a38bebfde1398d1"
Jul 08 16:44:25.494382 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:25.494346    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-tls-serving-67b7477c6d-n6q6c"]
Jul 08 16:44:25.494760 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:25.494669    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d190cf7-84ec-4cad-8a3c-3e65a78b48d8" containerName="s3-tls-init-custom"
Jul 08 16:44:25.494760 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:25.494680    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d190cf7-84ec-4cad-8a3c-3e65a78b48d8" containerName="s3-tls-init-custom"
Jul 08 16:44:25.494760 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:25.494700    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d190cf7-84ec-4cad-8a3c-3e65a78b48d8" containerName="download-hf-model"
Jul 08 16:44:25.494760 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:25.494705    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d190cf7-84ec-4cad-8a3c-3e65a78b48d8" containerName="download-hf-model"
Jul 08 16:44:25.494908 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:25.494786    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d190cf7-84ec-4cad-8a3c-3e65a78b48d8" containerName="s3-tls-init-custom"
Jul 08 16:44:25.497997 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:25.497973    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-serving-67b7477c6d-n6q6c"
Jul 08 16:44:25.500806 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:25.500782    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-serving\""
Jul 08 16:44:25.500906 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:25.500841    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-serving-artifact\""
Jul 08 16:44:25.508621 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:25.508597    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-serving-67b7477c6d-n6q6c"]
Jul 08 16:44:25.587206 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:25.587170    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4c49f2e6-1f07-482a-a7c6-3124391420df-data\") pod \"seaweedfs-tls-serving-67b7477c6d-n6q6c\" (UID: \"4c49f2e6-1f07-482a-a7c6-3124391420df\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-n6q6c"
Jul 08 16:44:25.587410 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:25.587240    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54l8p\" (UniqueName: \"kubernetes.io/projected/4c49f2e6-1f07-482a-a7c6-3124391420df-kube-api-access-54l8p\") pod \"seaweedfs-tls-serving-67b7477c6d-n6q6c\" (UID: \"4c49f2e6-1f07-482a-a7c6-3124391420df\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-n6q6c"
Jul 08 16:44:25.587410 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:25.587276    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"seaweedfs-tls-serving\" (UniqueName: \"kubernetes.io/projected/4c49f2e6-1f07-482a-a7c6-3124391420df-seaweedfs-tls-serving\") pod \"seaweedfs-tls-serving-67b7477c6d-n6q6c\" (UID: \"4c49f2e6-1f07-482a-a7c6-3124391420df\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-n6q6c"
Jul 08 16:44:25.688621 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:25.688576    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4c49f2e6-1f07-482a-a7c6-3124391420df-data\") pod \"seaweedfs-tls-serving-67b7477c6d-n6q6c\" (UID: \"4c49f2e6-1f07-482a-a7c6-3124391420df\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-n6q6c"
Jul 08 16:44:25.688814 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:25.688653    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54l8p\" (UniqueName: \"kubernetes.io/projected/4c49f2e6-1f07-482a-a7c6-3124391420df-kube-api-access-54l8p\") pod \"seaweedfs-tls-serving-67b7477c6d-n6q6c\" (UID: \"4c49f2e6-1f07-482a-a7c6-3124391420df\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-n6q6c"
Jul 08 16:44:25.688814 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:25.688681    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"seaweedfs-tls-serving\" (UniqueName: \"kubernetes.io/projected/4c49f2e6-1f07-482a-a7c6-3124391420df-seaweedfs-tls-serving\") pod \"seaweedfs-tls-serving-67b7477c6d-n6q6c\" (UID: \"4c49f2e6-1f07-482a-a7c6-3124391420df\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-n6q6c"
Jul 08 16:44:25.688974 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:25.688954    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4c49f2e6-1f07-482a-a7c6-3124391420df-data\") pod \"seaweedfs-tls-serving-67b7477c6d-n6q6c\" (UID: \"4c49f2e6-1f07-482a-a7c6-3124391420df\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-n6q6c"
Jul 08 16:44:25.691146 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:25.691126    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"seaweedfs-tls-serving\" (UniqueName: \"kubernetes.io/projected/4c49f2e6-1f07-482a-a7c6-3124391420df-seaweedfs-tls-serving\") pod \"seaweedfs-tls-serving-67b7477c6d-n6q6c\" (UID: \"4c49f2e6-1f07-482a-a7c6-3124391420df\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-n6q6c"
Jul 08 16:44:25.699568 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:25.699542    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54l8p\" (UniqueName: \"kubernetes.io/projected/4c49f2e6-1f07-482a-a7c6-3124391420df-kube-api-access-54l8p\") pod \"seaweedfs-tls-serving-67b7477c6d-n6q6c\" (UID: \"4c49f2e6-1f07-482a-a7c6-3124391420df\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-n6q6c"
Jul 08 16:44:25.807992 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:25.807896    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-serving-67b7477c6d-n6q6c"
Jul 08 16:44:25.957303 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:25.957271    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-serving-67b7477c6d-n6q6c"]
Jul 08 16:44:25.960465 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:44:25.960432    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c49f2e6_1f07_482a_a7c6_3124391420df.slice/crio-0269346a761e25e398b1cfa4d7f51c5172502845d0f7c79bea8cbb6d74fb6499 WatchSource:0}: Error finding container 0269346a761e25e398b1cfa4d7f51c5172502845d0f7c79bea8cbb6d74fb6499: Status 404 returned error can't find the container with id 0269346a761e25e398b1cfa4d7f51c5172502845d0f7c79bea8cbb6d74fb6499
Jul 08 16:44:26.878775 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:26.878738    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-serving-67b7477c6d-n6q6c" event={"ID":"4c49f2e6-1f07-482a-a7c6-3124391420df","Type":"ContainerStarted","Data":"d1935247f1738088b8bf782754add164057e711b628352a3d776ca884d765151"}
Jul 08 16:44:26.879202 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:26.878792    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-serving-67b7477c6d-n6q6c" event={"ID":"4c49f2e6-1f07-482a-a7c6-3124391420df","Type":"ContainerStarted","Data":"0269346a761e25e398b1cfa4d7f51c5172502845d0f7c79bea8cbb6d74fb6499"}
Jul 08 16:44:26.897333 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:26.897255    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-tls-serving-67b7477c6d-n6q6c" podStartSLOduration=1.5409362519999998 podStartE2EDuration="1.897236281s" totalImagesPullingTime="356.300029ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:44:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:44:26.896747604 +0000 UTC m=+586.237712686" watchObservedRunningTime="2026-07-08 16:44:26.897236281 +0000 UTC m=+586.238201359"
Jul 08 16:44:27.483055 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:27.483018    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-tls-init-serving-dq8rv"]
Jul 08 16:44:27.487590 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:27.487573    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-serving-dq8rv"
Jul 08 16:44:27.495366 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:27.495339    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-tls-init-serving-dq8rv"]
Jul 08 16:44:27.504048 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:27.504015    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4aeec037-1338-42d6-8f4e-cb80fa564a3a-model-cache\") pod \"s3-tls-init-serving-dq8rv\" (UID: \"4aeec037-1338-42d6-8f4e-cb80fa564a3a\") " pod="kserve/s3-tls-init-serving-dq8rv"
Jul 08 16:44:27.504553 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:27.504513    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4549d\" (UniqueName: \"kubernetes.io/projected/4aeec037-1338-42d6-8f4e-cb80fa564a3a-kube-api-access-4549d\") pod \"s3-tls-init-serving-dq8rv\" (UID: \"4aeec037-1338-42d6-8f4e-cb80fa564a3a\") " pod="kserve/s3-tls-init-serving-dq8rv"
Jul 08 16:44:27.605635 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:27.605571    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4aeec037-1338-42d6-8f4e-cb80fa564a3a-model-cache\") pod \"s3-tls-init-serving-dq8rv\" (UID: \"4aeec037-1338-42d6-8f4e-cb80fa564a3a\") " pod="kserve/s3-tls-init-serving-dq8rv"
Jul 08 16:44:27.605837 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:27.605662    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4549d\" (UniqueName: \"kubernetes.io/projected/4aeec037-1338-42d6-8f4e-cb80fa564a3a-kube-api-access-4549d\") pod \"s3-tls-init-serving-dq8rv\" (UID: \"4aeec037-1338-42d6-8f4e-cb80fa564a3a\") " pod="kserve/s3-tls-init-serving-dq8rv"
Jul 08 16:44:27.605971 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:27.605947    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4aeec037-1338-42d6-8f4e-cb80fa564a3a-model-cache\") pod \"s3-tls-init-serving-dq8rv\" (UID: \"4aeec037-1338-42d6-8f4e-cb80fa564a3a\") " pod="kserve/s3-tls-init-serving-dq8rv"
Jul 08 16:44:27.614734 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:27.614701    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4549d\" (UniqueName: \"kubernetes.io/projected/4aeec037-1338-42d6-8f4e-cb80fa564a3a-kube-api-access-4549d\") pod \"s3-tls-init-serving-dq8rv\" (UID: \"4aeec037-1338-42d6-8f4e-cb80fa564a3a\") " pod="kserve/s3-tls-init-serving-dq8rv"
Jul 08 16:44:27.806287 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:27.806203    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-serving-dq8rv"
Jul 08 16:44:27.936751 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:27.936717    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-tls-init-serving-dq8rv"]
Jul 08 16:44:27.940459 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:44:27.940433    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4aeec037_1338_42d6_8f4e_cb80fa564a3a.slice/crio-f0fecddb74c6ac880e84702f57ced91cbb8e37465132aa67746e5d9217bc4767 WatchSource:0}: Error finding container f0fecddb74c6ac880e84702f57ced91cbb8e37465132aa67746e5d9217bc4767: Status 404 returned error can't find the container with id f0fecddb74c6ac880e84702f57ced91cbb8e37465132aa67746e5d9217bc4767
Jul 08 16:44:28.890373 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:28.890302    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-dq8rv" event={"ID":"4aeec037-1338-42d6-8f4e-cb80fa564a3a","Type":"ContainerStarted","Data":"854c5c11a7b85c8d523604ab882f9518c76add2411f67145f23c7b7cd893d5e8"}
Jul 08 16:44:28.890590 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:28.890389    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-dq8rv" event={"ID":"4aeec037-1338-42d6-8f4e-cb80fa564a3a","Type":"ContainerStarted","Data":"f0fecddb74c6ac880e84702f57ced91cbb8e37465132aa67746e5d9217bc4767"}
Jul 08 16:44:29.896173 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:29.896087    2582 generic.go:356] "Generic (PLEG): container finished" podID="4aeec037-1338-42d6-8f4e-cb80fa564a3a" containerID="854c5c11a7b85c8d523604ab882f9518c76add2411f67145f23c7b7cd893d5e8" exitCode=0
Jul 08 16:44:29.896173 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:29.896137    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-dq8rv" event={"ID":"4aeec037-1338-42d6-8f4e-cb80fa564a3a","Type":"ContainerDied","Data":"854c5c11a7b85c8d523604ab882f9518c76add2411f67145f23c7b7cd893d5e8"}
Jul 08 16:44:30.901860 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:30.901822    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-dq8rv" event={"ID":"4aeec037-1338-42d6-8f4e-cb80fa564a3a","Type":"ContainerStarted","Data":"1aa51769627d8696cb5d8b9966c455168bdad88a223546cc00b78513a77559c2"}
Jul 08 16:44:30.919296 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:30.919245    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-tls-init-serving-dq8rv" podStartSLOduration=2.287810451 podStartE2EDuration="3.919228127s" totalImagesPullingTime="0s" totalInitContainerRuntime="1.631417676s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:44:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:44:30.917203689 +0000 UTC m=+590.258168767" watchObservedRunningTime="2026-07-08 16:44:30.919228127 +0000 UTC m=+590.260193204"
Jul 08 16:44:37.935682 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:37.935652    2582 generic.go:356] "Generic (PLEG): container finished" podID="4aeec037-1338-42d6-8f4e-cb80fa564a3a" containerID="1aa51769627d8696cb5d8b9966c455168bdad88a223546cc00b78513a77559c2" exitCode=0
Jul 08 16:44:37.936083 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:37.935718    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-dq8rv" event={"ID":"4aeec037-1338-42d6-8f4e-cb80fa564a3a","Type":"ContainerDied","Data":"1aa51769627d8696cb5d8b9966c455168bdad88a223546cc00b78513a77559c2"}
Jul 08 16:44:39.081239 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:39.081214    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-serving-dq8rv"
Jul 08 16:44:39.213430 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:39.213350    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4549d\" (UniqueName: \"kubernetes.io/projected/4aeec037-1338-42d6-8f4e-cb80fa564a3a-kube-api-access-4549d\") pod \"4aeec037-1338-42d6-8f4e-cb80fa564a3a\" (UID: \"4aeec037-1338-42d6-8f4e-cb80fa564a3a\") "
Jul 08 16:44:39.213430 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:39.213392    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4aeec037-1338-42d6-8f4e-cb80fa564a3a-model-cache\") pod \"4aeec037-1338-42d6-8f4e-cb80fa564a3a\" (UID: \"4aeec037-1338-42d6-8f4e-cb80fa564a3a\") "
Jul 08 16:44:39.215689 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:39.215659    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4aeec037-1338-42d6-8f4e-cb80fa564a3a-kube-api-access-4549d" (OuterVolumeSpecName: "kube-api-access-4549d") pod "4aeec037-1338-42d6-8f4e-cb80fa564a3a" (UID: "4aeec037-1338-42d6-8f4e-cb80fa564a3a"). InnerVolumeSpecName "kube-api-access-4549d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 16:44:39.314112 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:39.314081    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4549d\" (UniqueName: \"kubernetes.io/projected/4aeec037-1338-42d6-8f4e-cb80fa564a3a-kube-api-access-4549d\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:44:39.410228 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:39.410186    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4aeec037-1338-42d6-8f4e-cb80fa564a3a-model-cache" (OuterVolumeSpecName: "model-cache") pod "4aeec037-1338-42d6-8f4e-cb80fa564a3a" (UID: "4aeec037-1338-42d6-8f4e-cb80fa564a3a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 16:44:39.414894 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:39.414870    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4aeec037-1338-42d6-8f4e-cb80fa564a3a-model-cache\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:44:39.948994 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:39.948963    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-serving-dq8rv"
Jul 08 16:44:39.948994 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:39.948983    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-dq8rv" event={"ID":"4aeec037-1338-42d6-8f4e-cb80fa564a3a","Type":"ContainerDied","Data":"f0fecddb74c6ac880e84702f57ced91cbb8e37465132aa67746e5d9217bc4767"}
Jul 08 16:44:39.949201 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:39.949014    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f0fecddb74c6ac880e84702f57ced91cbb8e37465132aa67746e5d9217bc4767"
Jul 08 16:44:41.090172 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:41.090139    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kvhpr_19df7751-4b62-4ae5-92a6-0bbda11fec85/ovn-acl-logging/0.log"
Jul 08 16:44:41.092774 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:41.092753    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kvhpr_19df7751-4b62-4ae5-92a6-0bbda11fec85/ovn-acl-logging/0.log"
Jul 08 16:44:52.387941 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.387900    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"]
Jul 08 16:44:52.388545 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.388420    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4aeec037-1338-42d6-8f4e-cb80fa564a3a" containerName="s3-tls-init-serving"
Jul 08 16:44:52.388545 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.388439    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4aeec037-1338-42d6-8f4e-cb80fa564a3a" containerName="s3-tls-init-serving"
Jul 08 16:44:52.388545 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.388471    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4aeec037-1338-42d6-8f4e-cb80fa564a3a" containerName="download-hf-model"
Jul 08 16:44:52.388545 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.388480    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4aeec037-1338-42d6-8f4e-cb80fa564a3a" containerName="download-hf-model"
Jul 08 16:44:52.388769 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.388570    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4aeec037-1338-42d6-8f4e-cb80fa564a3a" containerName="s3-tls-init-serving"
Jul 08 16:44:52.391565 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.391545    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"
Jul 08 16:44:52.394168 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.394148    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 08 16:44:52.394389 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.394147    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-kube-rbac-proxy-sar-config\""
Jul 08 16:44:52.394389 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.394283    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 08 16:44:52.394544 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.394423    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 08 16:44:52.394544 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.394427    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-predictor-serving-cert\""
Jul 08 16:44:52.394544 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.394447    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 08 16:44:52.394544 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.394504    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-jq2mg\""
Jul 08 16:44:52.408691 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.408667    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"]
Jul 08 16:44:52.538069 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.538031    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d\" (UID: \"01515c8e-ff97-4e25-87ad-05dbd49cb8d2\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"
Jul 08 16:44:52.538069 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.538068    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d\" (UID: \"01515c8e-ff97-4e25-87ad-05dbd49cb8d2\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"
Jul 08 16:44:52.538280 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.538114    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8h7z\" (UniqueName: \"kubernetes.io/projected/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-kube-api-access-d8h7z\") pod \"isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d\" (UID: \"01515c8e-ff97-4e25-87ad-05dbd49cb8d2\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"
Jul 08 16:44:52.538280 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.538165    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d\" (UID: \"01515c8e-ff97-4e25-87ad-05dbd49cb8d2\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"
Jul 08 16:44:52.538280 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.538207    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d\" (UID: \"01515c8e-ff97-4e25-87ad-05dbd49cb8d2\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"
Jul 08 16:44:52.639422 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.639335    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d\" (UID: \"01515c8e-ff97-4e25-87ad-05dbd49cb8d2\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"
Jul 08 16:44:52.639422 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.639378    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d\" (UID: \"01515c8e-ff97-4e25-87ad-05dbd49cb8d2\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"
Jul 08 16:44:52.639652 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.639432    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8h7z\" (UniqueName: \"kubernetes.io/projected/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-kube-api-access-d8h7z\") pod \"isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d\" (UID: \"01515c8e-ff97-4e25-87ad-05dbd49cb8d2\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"
Jul 08 16:44:52.639652 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.639465    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d\" (UID: \"01515c8e-ff97-4e25-87ad-05dbd49cb8d2\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"
Jul 08 16:44:52.639652 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.639501    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d\" (UID: \"01515c8e-ff97-4e25-87ad-05dbd49cb8d2\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"
Jul 08 16:44:52.639942 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.639919    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d\" (UID: \"01515c8e-ff97-4e25-87ad-05dbd49cb8d2\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"
Jul 08 16:44:52.640111 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.640091    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d\" (UID: \"01515c8e-ff97-4e25-87ad-05dbd49cb8d2\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"
Jul 08 16:44:52.640185 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.640155    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d\" (UID: \"01515c8e-ff97-4e25-87ad-05dbd49cb8d2\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"
Jul 08 16:44:52.641929 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.641910    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d\" (UID: \"01515c8e-ff97-4e25-87ad-05dbd49cb8d2\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"
Jul 08 16:44:52.648146 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.648122    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8h7z\" (UniqueName: \"kubernetes.io/projected/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-kube-api-access-d8h7z\") pod \"isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d\" (UID: \"01515c8e-ff97-4e25-87ad-05dbd49cb8d2\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"
Jul 08 16:44:52.702709 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.702663    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"
Jul 08 16:44:52.841477 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:52.841341    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"]
Jul 08 16:44:52.846279 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:44:52.846248    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01515c8e_ff97_4e25_87ad_05dbd49cb8d2.slice/crio-440d577c8237074b69ab533209b26220d23c0bc8d0cf19ee4716bf75040f6caf WatchSource:0}: Error finding container 440d577c8237074b69ab533209b26220d23c0bc8d0cf19ee4716bf75040f6caf: Status 404 returned error can't find the container with id 440d577c8237074b69ab533209b26220d23c0bc8d0cf19ee4716bf75040f6caf
Jul 08 16:44:53.005357 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:53.005296    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" event={"ID":"01515c8e-ff97-4e25-87ad-05dbd49cb8d2","Type":"ContainerStarted","Data":"0dd907a2d891cf079b52a8f6ed699c2c456b0b44066d337a89b95f78738d0960"}
Jul 08 16:44:53.005545 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:53.005396    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" event={"ID":"01515c8e-ff97-4e25-87ad-05dbd49cb8d2","Type":"ContainerStarted","Data":"440d577c8237074b69ab533209b26220d23c0bc8d0cf19ee4716bf75040f6caf"}
Jul 08 16:44:57.024500 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:57.024467    2582 generic.go:356] "Generic (PLEG): container finished" podID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerID="0dd907a2d891cf079b52a8f6ed699c2c456b0b44066d337a89b95f78738d0960" exitCode=0
Jul 08 16:44:57.024885 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:44:57.024539    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" event={"ID":"01515c8e-ff97-4e25-87ad-05dbd49cb8d2","Type":"ContainerDied","Data":"0dd907a2d891cf079b52a8f6ed699c2c456b0b44066d337a89b95f78738d0960"}
Jul 08 16:45:12.106270 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:45:12.106235    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" event={"ID":"01515c8e-ff97-4e25-87ad-05dbd49cb8d2","Type":"ContainerStarted","Data":"85e867747dfc49620a5eaa913e28a0460a4c524ed6a7d84e73946e83a5467114"}
Jul 08 16:45:15.121581 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:45:15.121549    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" event={"ID":"01515c8e-ff97-4e25-87ad-05dbd49cb8d2","Type":"ContainerStarted","Data":"b88bea8a8697ae70653881704c8a3d795be760303b40ea3125225b79100556f2"}
Jul 08 16:45:18.140261 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:45:18.140222    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" event={"ID":"01515c8e-ff97-4e25-87ad-05dbd49cb8d2","Type":"ContainerStarted","Data":"2a9aab3ee358c2ec0baa37da8e60fee12dfb839937b8af8be0d4998681f06da7"}
Jul 08 16:45:18.140668 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:45:18.140393    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"
Jul 08 16:45:18.165463 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:45:18.165415    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podStartSLOduration=2.161191516 podStartE2EDuration="26.165397002s" totalImagesPullingTime="20.345033247s" totalInitContainerRuntime="3.659172239s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:44:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:45:18.163821093 +0000 UTC m=+637.504786196" watchObservedRunningTime="2026-07-08 16:45:18.165397002 +0000 UTC m=+637.506362079"
Jul 08 16:45:19.145924 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:45:19.145899    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"
Jul 08 16:45:19.146302 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:45:19.145960    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"
Jul 08 16:45:19.147292 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:45:19.147246    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 08 16:45:19.148102 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:45:19.148073    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:45:19.151064 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:45:19.151047    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"
Jul 08 16:45:20.150528 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:45:20.150491    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 08 16:45:20.150922 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:45:20.150796    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:45:21.155307 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:45:21.155212    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 08 16:45:21.156553 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:45:21.155472    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:45:31.155946 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:45:31.155894    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 08 16:45:31.156467 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:45:31.156308    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:45:41.155879 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:45:41.155831    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 08 16:45:41.156362 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:45:41.156340    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:45:51.155899 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:45:51.155847    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 08 16:45:51.156358 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:45:51.156285    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:46:01.155665 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:01.155619    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 08 16:46:01.156152 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:01.156123    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:46:11.155179 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:11.155134    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 08 16:46:11.155729 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:11.155614    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:46:21.155639 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:21.155581    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 08 16:46:21.156068 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:21.156037    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:46:31.156129 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:31.156049    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"
Jul 08 16:46:31.156559 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:31.156330    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"
Jul 08 16:46:37.422137 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:37.422102    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"]
Jul 08 16:46:37.422613 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:37.422453    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="kserve-container" containerID="cri-o://85e867747dfc49620a5eaa913e28a0460a4c524ed6a7d84e73946e83a5467114" gracePeriod=30
Jul 08 16:46:37.422613 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:37.422509    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="kube-rbac-proxy" containerID="cri-o://b88bea8a8697ae70653881704c8a3d795be760303b40ea3125225b79100556f2" gracePeriod=30
Jul 08 16:46:37.422760 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:37.422502    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="agent" containerID="cri-o://2a9aab3ee358c2ec0baa37da8e60fee12dfb839937b8af8be0d4998681f06da7" gracePeriod=30
Jul 08 16:46:37.551950 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:37.551909    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"]
Jul 08 16:46:37.554551 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:37.554536    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"
Jul 08 16:46:37.556865 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:37.556847    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-custom-kube-rbac-proxy-sar-config\""
Jul 08 16:46:37.556982 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:37.556846    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-custom-predictor-serving-cert\""
Jul 08 16:46:37.567830 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:37.567804    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"]
Jul 08 16:46:37.680679 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:37.680595    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6\" (UID: \"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"
Jul 08 16:46:37.680679 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:37.680646    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsfjz\" (UniqueName: \"kubernetes.io/projected/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-kube-api-access-nsfjz\") pod \"isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6\" (UID: \"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"
Jul 08 16:46:37.680679 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:37.680666    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6\" (UID: \"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"
Jul 08 16:46:37.680942 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:37.680757    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6\" (UID: \"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"
Jul 08 16:46:37.680942 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:37.680829    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6\" (UID: \"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"
Jul 08 16:46:37.781898 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:37.781858    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6\" (UID: \"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"
Jul 08 16:46:37.781898 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:37.781906    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6\" (UID: \"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"
Jul 08 16:46:37.782162 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:37.781939    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsfjz\" (UniqueName: \"kubernetes.io/projected/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-kube-api-access-nsfjz\") pod \"isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6\" (UID: \"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"
Jul 08 16:46:37.782162 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:37.781958    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6\" (UID: \"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"
Jul 08 16:46:37.782162 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:37.782024    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6\" (UID: \"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"
Jul 08 16:46:37.782601 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:37.782574    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6\" (UID: \"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"
Jul 08 16:46:37.782724 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:37.782690    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6\" (UID: \"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"
Jul 08 16:46:37.782775 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:37.782762    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6\" (UID: \"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"
Jul 08 16:46:37.784766 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:37.784742    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6\" (UID: \"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"
Jul 08 16:46:37.790814 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:37.790781    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsfjz\" (UniqueName: \"kubernetes.io/projected/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-kube-api-access-nsfjz\") pod \"isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6\" (UID: \"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"
Jul 08 16:46:37.864725 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:37.864670    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"
Jul 08 16:46:38.007558 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:38.007531    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"]
Jul 08 16:46:38.009940 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:46:38.009914    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cf5ee2c_2516_4231_af20_dd1c8b77aaa6.slice/crio-16447d54a9bbec7d528844b0be9abc6c0273b19134f016ee50628161123dd69f WatchSource:0}: Error finding container 16447d54a9bbec7d528844b0be9abc6c0273b19134f016ee50628161123dd69f: Status 404 returned error can't find the container with id 16447d54a9bbec7d528844b0be9abc6c0273b19134f016ee50628161123dd69f
Jul 08 16:46:38.500201 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:38.500167    2582 generic.go:356] "Generic (PLEG): container finished" podID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerID="b88bea8a8697ae70653881704c8a3d795be760303b40ea3125225b79100556f2" exitCode=2
Jul 08 16:46:38.500596 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:38.500229    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" event={"ID":"01515c8e-ff97-4e25-87ad-05dbd49cb8d2","Type":"ContainerDied","Data":"b88bea8a8697ae70653881704c8a3d795be760303b40ea3125225b79100556f2"}
Jul 08 16:46:38.501984 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:38.501955    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" event={"ID":"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6","Type":"ContainerStarted","Data":"1607cc1fa805399f78791d739e6c49db866ec33733b8430ce389d4cfca91d31c"}
Jul 08 16:46:38.502114 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:38.502029    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" event={"ID":"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6","Type":"ContainerStarted","Data":"16447d54a9bbec7d528844b0be9abc6c0273b19134f016ee50628161123dd69f"}
Jul 08 16:46:39.146450 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:39.146409    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Jul 08 16:46:41.155202 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:41.155158    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 08 16:46:41.155609 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:41.155551    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:46:42.522577 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:42.522533    2582 generic.go:356] "Generic (PLEG): container finished" podID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerID="85e867747dfc49620a5eaa913e28a0460a4c524ed6a7d84e73946e83a5467114" exitCode=0
Jul 08 16:46:42.523088 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:42.522609    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" event={"ID":"01515c8e-ff97-4e25-87ad-05dbd49cb8d2","Type":"ContainerDied","Data":"85e867747dfc49620a5eaa913e28a0460a4c524ed6a7d84e73946e83a5467114"}
Jul 08 16:46:42.524198 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:42.524175    2582 generic.go:356] "Generic (PLEG): container finished" podID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerID="1607cc1fa805399f78791d739e6c49db866ec33733b8430ce389d4cfca91d31c" exitCode=0
Jul 08 16:46:42.524351 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:42.524206    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" event={"ID":"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6","Type":"ContainerDied","Data":"1607cc1fa805399f78791d739e6c49db866ec33733b8430ce389d4cfca91d31c"}
Jul 08 16:46:43.531677 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:43.531643    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" event={"ID":"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6","Type":"ContainerStarted","Data":"730317f1db1f9a87d3201b26d234c3e7c27b056c888091d73fa8451195269ee9"}
Jul 08 16:46:43.532062 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:43.531711    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" event={"ID":"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6","Type":"ContainerStarted","Data":"8048146f3aa0bd0bcc5fc8fc7cea822a2713265713becca7f4e17df7cef59779"}
Jul 08 16:46:43.532062 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:43.531735    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" event={"ID":"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6","Type":"ContainerStarted","Data":"8dca7e818f52a4f2809a5f36d660bf58cd8fccbc51ccd96a7eca7c6746fb17f6"}
Jul 08 16:46:43.532062 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:43.532015    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"
Jul 08 16:46:43.532212 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:43.532066    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"
Jul 08 16:46:43.533208 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:43.533185    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:5000: connect: connection refused"
Jul 08 16:46:43.553560 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:43.553516    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podStartSLOduration=2.780912412 podStartE2EDuration="6.553502704s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.772590292s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:46:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:46:43.551468796 +0000 UTC m=+722.892433884" watchObservedRunningTime="2026-07-08 16:46:43.553502704 +0000 UTC m=+722.894467783"
Jul 08 16:46:44.147128 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:44.147089    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Jul 08 16:46:44.536769 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:44.536737    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"
Jul 08 16:46:44.537238 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:44.536773    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:5000: connect: connection refused"
Jul 08 16:46:44.537945 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:44.537916    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:46:45.541274 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:45.541228    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:5000: connect: connection refused"
Jul 08 16:46:45.541716 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:45.541668    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:46:49.146219 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:49.146181    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Jul 08 16:46:49.146655 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:49.146304    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"
Jul 08 16:46:50.546026 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:50.546000    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"
Jul 08 16:46:50.546516 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:50.546465    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:5000: connect: connection refused"
Jul 08 16:46:50.546888 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:50.546866    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:46:51.155462 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:51.155417    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 08 16:46:51.155786 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:51.155759    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:46:54.146476 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:54.146430    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Jul 08 16:46:59.146744 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:46:59.146694    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Jul 08 16:47:00.547242 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:00.547196    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:5000: connect: connection refused"
Jul 08 16:47:00.547674 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:00.547634    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:47:01.155811 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:01.155765    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Jul 08 16:47:01.155992 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:01.155936    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"
Jul 08 16:47:01.156174 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:01.156149    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:47:01.156258 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:01.156247    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"
Jul 08 16:47:04.147069 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:04.147029    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Jul 08 16:47:07.577054 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.577024    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"
Jul 08 16:47:07.642774 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.642742    2582 generic.go:356] "Generic (PLEG): container finished" podID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerID="2a9aab3ee358c2ec0baa37da8e60fee12dfb839937b8af8be0d4998681f06da7" exitCode=0
Jul 08 16:47:07.642947 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.642807    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"
Jul 08 16:47:07.642947 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.642824    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" event={"ID":"01515c8e-ff97-4e25-87ad-05dbd49cb8d2","Type":"ContainerDied","Data":"2a9aab3ee358c2ec0baa37da8e60fee12dfb839937b8af8be0d4998681f06da7"}
Jul 08 16:47:07.642947 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.642893    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d" event={"ID":"01515c8e-ff97-4e25-87ad-05dbd49cb8d2","Type":"ContainerDied","Data":"440d577c8237074b69ab533209b26220d23c0bc8d0cf19ee4716bf75040f6caf"}
Jul 08 16:47:07.642947 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.642938    2582 scope.go:117] "RemoveContainer" containerID="2a9aab3ee358c2ec0baa37da8e60fee12dfb839937b8af8be0d4998681f06da7"
Jul 08 16:47:07.648590 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.648563    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-kserve-provision-location\") pod \"01515c8e-ff97-4e25-87ad-05dbd49cb8d2\" (UID: \"01515c8e-ff97-4e25-87ad-05dbd49cb8d2\") "
Jul 08 16:47:07.648702 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.648607    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8h7z\" (UniqueName: \"kubernetes.io/projected/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-kube-api-access-d8h7z\") pod \"01515c8e-ff97-4e25-87ad-05dbd49cb8d2\" (UID: \"01515c8e-ff97-4e25-87ad-05dbd49cb8d2\") "
Jul 08 16:47:07.648702 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.648629    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-proxy-tls\") pod \"01515c8e-ff97-4e25-87ad-05dbd49cb8d2\" (UID: \"01515c8e-ff97-4e25-87ad-05dbd49cb8d2\") "
Jul 08 16:47:07.648702 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.648682    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-kube-rbac-proxy-sar-config\") pod \"01515c8e-ff97-4e25-87ad-05dbd49cb8d2\" (UID: \"01515c8e-ff97-4e25-87ad-05dbd49cb8d2\") "
Jul 08 16:47:07.648857 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.648777    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-cabundle-cert\") pod \"01515c8e-ff97-4e25-87ad-05dbd49cb8d2\" (UID: \"01515c8e-ff97-4e25-87ad-05dbd49cb8d2\") "
Jul 08 16:47:07.648922 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.648882    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "01515c8e-ff97-4e25-87ad-05dbd49cb8d2" (UID: "01515c8e-ff97-4e25-87ad-05dbd49cb8d2"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 16:47:07.649059 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.649036    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "01515c8e-ff97-4e25-87ad-05dbd49cb8d2" (UID: "01515c8e-ff97-4e25-87ad-05dbd49cb8d2"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:47:07.649378 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.649259    2582 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:47:07.649378 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.649264    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "01515c8e-ff97-4e25-87ad-05dbd49cb8d2" (UID: "01515c8e-ff97-4e25-87ad-05dbd49cb8d2"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:47:07.649378 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.649281    2582 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-kserve-provision-location\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:47:07.650961 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.650936    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-kube-api-access-d8h7z" (OuterVolumeSpecName: "kube-api-access-d8h7z") pod "01515c8e-ff97-4e25-87ad-05dbd49cb8d2" (UID: "01515c8e-ff97-4e25-87ad-05dbd49cb8d2"). InnerVolumeSpecName "kube-api-access-d8h7z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 16:47:07.651191 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.651172    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "01515c8e-ff97-4e25-87ad-05dbd49cb8d2" (UID: "01515c8e-ff97-4e25-87ad-05dbd49cb8d2"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 16:47:07.652294 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.652275    2582 scope.go:117] "RemoveContainer" containerID="b88bea8a8697ae70653881704c8a3d795be760303b40ea3125225b79100556f2"
Jul 08 16:47:07.666499 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.666470    2582 scope.go:117] "RemoveContainer" containerID="85e867747dfc49620a5eaa913e28a0460a4c524ed6a7d84e73946e83a5467114"
Jul 08 16:47:07.674155 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.674136    2582 scope.go:117] "RemoveContainer" containerID="0dd907a2d891cf079b52a8f6ed699c2c456b0b44066d337a89b95f78738d0960"
Jul 08 16:47:07.682409 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.682389    2582 scope.go:117] "RemoveContainer" containerID="2a9aab3ee358c2ec0baa37da8e60fee12dfb839937b8af8be0d4998681f06da7"
Jul 08 16:47:07.682673 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:47:07.682655    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a9aab3ee358c2ec0baa37da8e60fee12dfb839937b8af8be0d4998681f06da7\": container with ID starting with 2a9aab3ee358c2ec0baa37da8e60fee12dfb839937b8af8be0d4998681f06da7 not found: ID does not exist" containerID="2a9aab3ee358c2ec0baa37da8e60fee12dfb839937b8af8be0d4998681f06da7"
Jul 08 16:47:07.682737 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.682687    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a9aab3ee358c2ec0baa37da8e60fee12dfb839937b8af8be0d4998681f06da7"} err="failed to get container status \"2a9aab3ee358c2ec0baa37da8e60fee12dfb839937b8af8be0d4998681f06da7\": rpc error: code = NotFound desc = could not find container \"2a9aab3ee358c2ec0baa37da8e60fee12dfb839937b8af8be0d4998681f06da7\": container with ID starting with 2a9aab3ee358c2ec0baa37da8e60fee12dfb839937b8af8be0d4998681f06da7 not found: ID does not exist"
Jul 08 16:47:07.682737 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.682713    2582 scope.go:117] "RemoveContainer" containerID="b88bea8a8697ae70653881704c8a3d795be760303b40ea3125225b79100556f2"
Jul 08 16:47:07.682944 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:47:07.682929    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b88bea8a8697ae70653881704c8a3d795be760303b40ea3125225b79100556f2\": container with ID starting with b88bea8a8697ae70653881704c8a3d795be760303b40ea3125225b79100556f2 not found: ID does not exist" containerID="b88bea8a8697ae70653881704c8a3d795be760303b40ea3125225b79100556f2"
Jul 08 16:47:07.682986 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.682950    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b88bea8a8697ae70653881704c8a3d795be760303b40ea3125225b79100556f2"} err="failed to get container status \"b88bea8a8697ae70653881704c8a3d795be760303b40ea3125225b79100556f2\": rpc error: code = NotFound desc = could not find container \"b88bea8a8697ae70653881704c8a3d795be760303b40ea3125225b79100556f2\": container with ID starting with b88bea8a8697ae70653881704c8a3d795be760303b40ea3125225b79100556f2 not found: ID does not exist"
Jul 08 16:47:07.682986 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.682971    2582 scope.go:117] "RemoveContainer" containerID="85e867747dfc49620a5eaa913e28a0460a4c524ed6a7d84e73946e83a5467114"
Jul 08 16:47:07.683214 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:47:07.683198    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85e867747dfc49620a5eaa913e28a0460a4c524ed6a7d84e73946e83a5467114\": container with ID starting with 85e867747dfc49620a5eaa913e28a0460a4c524ed6a7d84e73946e83a5467114 not found: ID does not exist" containerID="85e867747dfc49620a5eaa913e28a0460a4c524ed6a7d84e73946e83a5467114"
Jul 08 16:47:07.683278 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.683225    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85e867747dfc49620a5eaa913e28a0460a4c524ed6a7d84e73946e83a5467114"} err="failed to get container status \"85e867747dfc49620a5eaa913e28a0460a4c524ed6a7d84e73946e83a5467114\": rpc error: code = NotFound desc = could not find container \"85e867747dfc49620a5eaa913e28a0460a4c524ed6a7d84e73946e83a5467114\": container with ID starting with 85e867747dfc49620a5eaa913e28a0460a4c524ed6a7d84e73946e83a5467114 not found: ID does not exist"
Jul 08 16:47:07.683278 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.683246    2582 scope.go:117] "RemoveContainer" containerID="0dd907a2d891cf079b52a8f6ed699c2c456b0b44066d337a89b95f78738d0960"
Jul 08 16:47:07.683520 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:47:07.683504    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0dd907a2d891cf079b52a8f6ed699c2c456b0b44066d337a89b95f78738d0960\": container with ID starting with 0dd907a2d891cf079b52a8f6ed699c2c456b0b44066d337a89b95f78738d0960 not found: ID does not exist" containerID="0dd907a2d891cf079b52a8f6ed699c2c456b0b44066d337a89b95f78738d0960"
Jul 08 16:47:07.683562 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.683523    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dd907a2d891cf079b52a8f6ed699c2c456b0b44066d337a89b95f78738d0960"} err="failed to get container status \"0dd907a2d891cf079b52a8f6ed699c2c456b0b44066d337a89b95f78738d0960\": rpc error: code = NotFound desc = could not find container \"0dd907a2d891cf079b52a8f6ed699c2c456b0b44066d337a89b95f78738d0960\": container with ID starting with 0dd907a2d891cf079b52a8f6ed699c2c456b0b44066d337a89b95f78738d0960 not found: ID does not exist"
Jul 08 16:47:07.750306 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.750274    2582 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-cabundle-cert\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:47:07.750306 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.750306    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8h7z\" (UniqueName: \"kubernetes.io/projected/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-kube-api-access-d8h7z\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:47:07.750534 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.750346    2582 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/01515c8e-ff97-4e25-87ad-05dbd49cb8d2-proxy-tls\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:47:07.970348 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.970287    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"]
Jul 08 16:47:07.974969 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:07.974946    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7bcdbf6fbf-llv6d"]
Jul 08 16:47:09.217064 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:09.217030    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" path="/var/lib/kubelet/pods/01515c8e-ff97-4e25-87ad-05dbd49cb8d2/volumes"
Jul 08 16:47:10.546772 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:10.546713    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:5000: connect: connection refused"
Jul 08 16:47:10.547177 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:10.547155    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:47:20.547355 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:20.547291    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:5000: connect: connection refused"
Jul 08 16:47:20.547911 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:20.547749    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:47:30.546621 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:30.546567    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:5000: connect: connection refused"
Jul 08 16:47:30.547045 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:30.547022    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:47:40.546489 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:40.546443    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:5000: connect: connection refused"
Jul 08 16:47:40.546979 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:40.546817    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:47:50.547125 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:50.547086    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:5000: connect: connection refused"
Jul 08 16:47:50.547626 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:47:50.547601    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:48:00.547542 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:00.547501    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"
Jul 08 16:48:00.548008 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:00.547667    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"
Jul 08 16:48:12.662473 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.662436    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"]
Jul 08 16:48:12.662877 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.662785    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="kserve-container" containerID="cri-o://8dca7e818f52a4f2809a5f36d660bf58cd8fccbc51ccd96a7eca7c6746fb17f6" gracePeriod=30
Jul 08 16:48:12.662940 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.662859    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="agent" containerID="cri-o://730317f1db1f9a87d3201b26d234c3e7c27b056c888091d73fa8451195269ee9" gracePeriod=30
Jul 08 16:48:12.662940 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.662881    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="kube-rbac-proxy" containerID="cri-o://8048146f3aa0bd0bcc5fc8fc7cea822a2713265713becca7f4e17df7cef59779" gracePeriod=30
Jul 08 16:48:12.756288 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.756251    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2"]
Jul 08 16:48:12.756655 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.756640    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="kube-rbac-proxy"
Jul 08 16:48:12.756700 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.756657    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="kube-rbac-proxy"
Jul 08 16:48:12.756700 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.756671    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="kserve-container"
Jul 08 16:48:12.756700 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.756677    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="kserve-container"
Jul 08 16:48:12.756700 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.756693    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="storage-initializer"
Jul 08 16:48:12.756700 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.756699    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="storage-initializer"
Jul 08 16:48:12.756842 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.756706    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="agent"
Jul 08 16:48:12.756842 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.756711    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="agent"
Jul 08 16:48:12.756842 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.756766    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="kube-rbac-proxy"
Jul 08 16:48:12.756842 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.756773    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="agent"
Jul 08 16:48:12.756842 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.756782    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="01515c8e-ff97-4e25-87ad-05dbd49cb8d2" containerName="kserve-container"
Jul 08 16:48:12.760107 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.760087    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2"
Jul 08 16:48:12.762451 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.762429    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-predictor-serving-cert\""
Jul 08 16:48:12.762714 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.762693    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-kube-rbac-proxy-sar-config\""
Jul 08 16:48:12.771137 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.771116    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2"]
Jul 08 16:48:12.834002 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.833965    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/873bbe5a-e8a1-420f-aa8f-84bcaf656116-proxy-tls\") pod \"message-dumper-predictor-6575bcfdfd-mqzx2\" (UID: \"873bbe5a-e8a1-420f-aa8f-84bcaf656116\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2"
Jul 08 16:48:12.834153 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.834010    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trf6g\" (UniqueName: \"kubernetes.io/projected/873bbe5a-e8a1-420f-aa8f-84bcaf656116-kube-api-access-trf6g\") pod \"message-dumper-predictor-6575bcfdfd-mqzx2\" (UID: \"873bbe5a-e8a1-420f-aa8f-84bcaf656116\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2"
Jul 08 16:48:12.834153 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.834043    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/873bbe5a-e8a1-420f-aa8f-84bcaf656116-kube-rbac-proxy-sar-config\") pod \"message-dumper-predictor-6575bcfdfd-mqzx2\" (UID: \"873bbe5a-e8a1-420f-aa8f-84bcaf656116\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2"
Jul 08 16:48:12.929599 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.929569    2582 generic.go:356] "Generic (PLEG): container finished" podID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerID="8048146f3aa0bd0bcc5fc8fc7cea822a2713265713becca7f4e17df7cef59779" exitCode=2
Jul 08 16:48:12.929753 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.929623    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" event={"ID":"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6","Type":"ContainerDied","Data":"8048146f3aa0bd0bcc5fc8fc7cea822a2713265713becca7f4e17df7cef59779"}
Jul 08 16:48:12.935078 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.935052    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/873bbe5a-e8a1-420f-aa8f-84bcaf656116-proxy-tls\") pod \"message-dumper-predictor-6575bcfdfd-mqzx2\" (UID: \"873bbe5a-e8a1-420f-aa8f-84bcaf656116\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2"
Jul 08 16:48:12.935180 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.935089    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trf6g\" (UniqueName: \"kubernetes.io/projected/873bbe5a-e8a1-420f-aa8f-84bcaf656116-kube-api-access-trf6g\") pod \"message-dumper-predictor-6575bcfdfd-mqzx2\" (UID: \"873bbe5a-e8a1-420f-aa8f-84bcaf656116\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2"
Jul 08 16:48:12.935180 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.935116    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/873bbe5a-e8a1-420f-aa8f-84bcaf656116-kube-rbac-proxy-sar-config\") pod \"message-dumper-predictor-6575bcfdfd-mqzx2\" (UID: \"873bbe5a-e8a1-420f-aa8f-84bcaf656116\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2"
Jul 08 16:48:12.935811 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.935756    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/873bbe5a-e8a1-420f-aa8f-84bcaf656116-kube-rbac-proxy-sar-config\") pod \"message-dumper-predictor-6575bcfdfd-mqzx2\" (UID: \"873bbe5a-e8a1-420f-aa8f-84bcaf656116\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2"
Jul 08 16:48:12.937607 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.937587    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/873bbe5a-e8a1-420f-aa8f-84bcaf656116-proxy-tls\") pod \"message-dumper-predictor-6575bcfdfd-mqzx2\" (UID: \"873bbe5a-e8a1-420f-aa8f-84bcaf656116\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2"
Jul 08 16:48:12.943841 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:12.943818    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trf6g\" (UniqueName: \"kubernetes.io/projected/873bbe5a-e8a1-420f-aa8f-84bcaf656116-kube-api-access-trf6g\") pod \"message-dumper-predictor-6575bcfdfd-mqzx2\" (UID: \"873bbe5a-e8a1-420f-aa8f-84bcaf656116\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2"
Jul 08 16:48:13.070359 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:13.070300    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2"
Jul 08 16:48:13.213812 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:13.211713    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2"]
Jul 08 16:48:13.216943 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:48:13.216913    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod873bbe5a_e8a1_420f_aa8f_84bcaf656116.slice/crio-3ef5f09464cbc8f396646a6437f72c85aff97e4391bd3edf00556aaa0cdfa9c9 WatchSource:0}: Error finding container 3ef5f09464cbc8f396646a6437f72c85aff97e4391bd3edf00556aaa0cdfa9c9: Status 404 returned error can't find the container with id 3ef5f09464cbc8f396646a6437f72c85aff97e4391bd3edf00556aaa0cdfa9c9
Jul 08 16:48:13.218688 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:13.218673    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 16:48:13.935021 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:13.934980    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2" event={"ID":"873bbe5a-e8a1-420f-aa8f-84bcaf656116","Type":"ContainerStarted","Data":"3ef5f09464cbc8f396646a6437f72c85aff97e4391bd3edf00556aaa0cdfa9c9"}
Jul 08 16:48:14.941677 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:14.941638    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2" event={"ID":"873bbe5a-e8a1-420f-aa8f-84bcaf656116","Type":"ContainerStarted","Data":"4cfcbbc53116a8c32877e4ac83b8ca91e56acf12f8263d853f3de4f1864f8940"}
Jul 08 16:48:14.942177 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:14.942094    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2" event={"ID":"873bbe5a-e8a1-420f-aa8f-84bcaf656116","Type":"ContainerStarted","Data":"ee5440eda8814477d14de543be73baea85e212bc16841aa31ac701e46047f2a4"}
Jul 08 16:48:14.947531 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:14.942237    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2"
Jul 08 16:48:14.979572 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:14.979517    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2" podStartSLOduration=1.8109323339999999 podStartE2EDuration="2.979502192s" totalImagesPullingTime="1.168569858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:48:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:48:14.977547531 +0000 UTC m=+814.318512610" watchObservedRunningTime="2026-07-08 16:48:14.979502192 +0000 UTC m=+814.320467269"
Jul 08 16:48:15.541680 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:15.541638    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.28:8643/healthz\": dial tcp 10.134.0.28:8643: connect: connection refused"
Jul 08 16:48:15.945827 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:15.945803    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2"
Jul 08 16:48:15.947472 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:15.947451    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2"
Jul 08 16:48:17.957345 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:17.957294    2582 generic.go:356] "Generic (PLEG): container finished" podID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerID="8dca7e818f52a4f2809a5f36d660bf58cd8fccbc51ccd96a7eca7c6746fb17f6" exitCode=0
Jul 08 16:48:17.957710 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:17.957371    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" event={"ID":"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6","Type":"ContainerDied","Data":"8dca7e818f52a4f2809a5f36d660bf58cd8fccbc51ccd96a7eca7c6746fb17f6"}
Jul 08 16:48:20.542270 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:20.542228    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.28:8643/healthz\": dial tcp 10.134.0.28:8643: connect: connection refused"
Jul 08 16:48:20.546720 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:20.546692    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:5000: connect: connection refused"
Jul 08 16:48:20.547028 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:20.547010    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:48:22.962405 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:22.962379    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2"
Jul 08 16:48:25.542066 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:25.542022    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.28:8643/healthz\": dial tcp 10.134.0.28:8643: connect: connection refused"
Jul 08 16:48:25.542507 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:25.542162    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"
Jul 08 16:48:30.542517 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:30.542463    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.28:8643/healthz\": dial tcp 10.134.0.28:8643: connect: connection refused"
Jul 08 16:48:30.546840 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:30.546807    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:5000: connect: connection refused"
Jul 08 16:48:30.547121 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:30.547097    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:48:32.785732 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:32.785693    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"]
Jul 08 16:48:32.789722 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:32.789696    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"
Jul 08 16:48:32.792108 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:32.792084    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-logger-kube-rbac-proxy-sar-config\""
Jul 08 16:48:32.792217 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:32.792170    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-logger-predictor-serving-cert\""
Jul 08 16:48:32.800711 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:32.800689    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"]
Jul 08 16:48:32.923721 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:32.923684    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/06bfd315-7e36-4511-b9d7-c2f488b06b8a-proxy-tls\") pod \"isvc-logger-predictor-786d9c95cd-vznlq\" (UID: \"06bfd315-7e36-4511-b9d7-c2f488b06b8a\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"
Jul 08 16:48:32.923905 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:32.923728    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pklq5\" (UniqueName: \"kubernetes.io/projected/06bfd315-7e36-4511-b9d7-c2f488b06b8a-kube-api-access-pklq5\") pod \"isvc-logger-predictor-786d9c95cd-vznlq\" (UID: \"06bfd315-7e36-4511-b9d7-c2f488b06b8a\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"
Jul 08 16:48:32.923905 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:32.923776    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/06bfd315-7e36-4511-b9d7-c2f488b06b8a-cabundle-cert\") pod \"isvc-logger-predictor-786d9c95cd-vznlq\" (UID: \"06bfd315-7e36-4511-b9d7-c2f488b06b8a\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"
Jul 08 16:48:32.923905 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:32.923798    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/06bfd315-7e36-4511-b9d7-c2f488b06b8a-kserve-provision-location\") pod \"isvc-logger-predictor-786d9c95cd-vznlq\" (UID: \"06bfd315-7e36-4511-b9d7-c2f488b06b8a\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"
Jul 08 16:48:32.924022 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:32.923894    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/06bfd315-7e36-4511-b9d7-c2f488b06b8a-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-786d9c95cd-vznlq\" (UID: \"06bfd315-7e36-4511-b9d7-c2f488b06b8a\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"
Jul 08 16:48:33.024555 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:33.024525    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/06bfd315-7e36-4511-b9d7-c2f488b06b8a-cabundle-cert\") pod \"isvc-logger-predictor-786d9c95cd-vznlq\" (UID: \"06bfd315-7e36-4511-b9d7-c2f488b06b8a\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"
Jul 08 16:48:33.024737 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:33.024564    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/06bfd315-7e36-4511-b9d7-c2f488b06b8a-kserve-provision-location\") pod \"isvc-logger-predictor-786d9c95cd-vznlq\" (UID: \"06bfd315-7e36-4511-b9d7-c2f488b06b8a\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"
Jul 08 16:48:33.024737 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:33.024623    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/06bfd315-7e36-4511-b9d7-c2f488b06b8a-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-786d9c95cd-vznlq\" (UID: \"06bfd315-7e36-4511-b9d7-c2f488b06b8a\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"
Jul 08 16:48:33.024737 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:33.024649    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/06bfd315-7e36-4511-b9d7-c2f488b06b8a-proxy-tls\") pod \"isvc-logger-predictor-786d9c95cd-vznlq\" (UID: \"06bfd315-7e36-4511-b9d7-c2f488b06b8a\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"
Jul 08 16:48:33.024737 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:33.024681    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pklq5\" (UniqueName: \"kubernetes.io/projected/06bfd315-7e36-4511-b9d7-c2f488b06b8a-kube-api-access-pklq5\") pod \"isvc-logger-predictor-786d9c95cd-vznlq\" (UID: \"06bfd315-7e36-4511-b9d7-c2f488b06b8a\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"
Jul 08 16:48:33.025259 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:33.025235    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/06bfd315-7e36-4511-b9d7-c2f488b06b8a-kserve-provision-location\") pod \"isvc-logger-predictor-786d9c95cd-vznlq\" (UID: \"06bfd315-7e36-4511-b9d7-c2f488b06b8a\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"
Jul 08 16:48:33.025440 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:33.025416    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/06bfd315-7e36-4511-b9d7-c2f488b06b8a-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-786d9c95cd-vznlq\" (UID: \"06bfd315-7e36-4511-b9d7-c2f488b06b8a\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"
Jul 08 16:48:33.025728 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:33.025414    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/06bfd315-7e36-4511-b9d7-c2f488b06b8a-cabundle-cert\") pod \"isvc-logger-predictor-786d9c95cd-vznlq\" (UID: \"06bfd315-7e36-4511-b9d7-c2f488b06b8a\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"
Jul 08 16:48:33.027418 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:33.027398    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/06bfd315-7e36-4511-b9d7-c2f488b06b8a-proxy-tls\") pod \"isvc-logger-predictor-786d9c95cd-vznlq\" (UID: \"06bfd315-7e36-4511-b9d7-c2f488b06b8a\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"
Jul 08 16:48:33.034868 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:33.034842    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pklq5\" (UniqueName: \"kubernetes.io/projected/06bfd315-7e36-4511-b9d7-c2f488b06b8a-kube-api-access-pklq5\") pod \"isvc-logger-predictor-786d9c95cd-vznlq\" (UID: \"06bfd315-7e36-4511-b9d7-c2f488b06b8a\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"
Jul 08 16:48:33.101583 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:33.101485    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"
Jul 08 16:48:33.245812 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:33.245783    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"]
Jul 08 16:48:33.247732 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:48:33.247705    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06bfd315_7e36_4511_b9d7_c2f488b06b8a.slice/crio-221f1e88e8f77ac617a24a830a9e154dce2b75b697118c306ed455162c8f7917 WatchSource:0}: Error finding container 221f1e88e8f77ac617a24a830a9e154dce2b75b697118c306ed455162c8f7917: Status 404 returned error can't find the container with id 221f1e88e8f77ac617a24a830a9e154dce2b75b697118c306ed455162c8f7917
Jul 08 16:48:34.036010 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:34.035978    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" event={"ID":"06bfd315-7e36-4511-b9d7-c2f488b06b8a","Type":"ContainerStarted","Data":"5babb2aac454e8cbdbf83632b1f9b665e9523601b482bdc1d04918e4384a0cde"}
Jul 08 16:48:34.036367 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:34.036070    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" event={"ID":"06bfd315-7e36-4511-b9d7-c2f488b06b8a","Type":"ContainerStarted","Data":"221f1e88e8f77ac617a24a830a9e154dce2b75b697118c306ed455162c8f7917"}
Jul 08 16:48:35.542180 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:35.542140    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.28:8643/healthz\": dial tcp 10.134.0.28:8643: connect: connection refused"
Jul 08 16:48:37.052456 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:37.052363    2582 generic.go:356] "Generic (PLEG): container finished" podID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerID="5babb2aac454e8cbdbf83632b1f9b665e9523601b482bdc1d04918e4384a0cde" exitCode=0
Jul 08 16:48:37.052456 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:37.052442    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" event={"ID":"06bfd315-7e36-4511-b9d7-c2f488b06b8a","Type":"ContainerDied","Data":"5babb2aac454e8cbdbf83632b1f9b665e9523601b482bdc1d04918e4384a0cde"}
Jul 08 16:48:38.059880 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:38.059851    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" event={"ID":"06bfd315-7e36-4511-b9d7-c2f488b06b8a","Type":"ContainerStarted","Data":"adb2032b34480a345de76f7b84549dc798944e4750bee5e5bae2d251988fb9eb"}
Jul 08 16:48:38.060295 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:38.059918    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" event={"ID":"06bfd315-7e36-4511-b9d7-c2f488b06b8a","Type":"ContainerStarted","Data":"5cf74a3b48f936a951baa9137636a992d48f6cbe6521182f0f3d08c3d2df0311"}
Jul 08 16:48:38.060295 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:38.059948    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" event={"ID":"06bfd315-7e36-4511-b9d7-c2f488b06b8a","Type":"ContainerStarted","Data":"104ec5836b89d95366de58050f565892eb97fded6d4168c90fbec4170db0f8fb"}
Jul 08 16:48:38.060295 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:38.060246    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"
Jul 08 16:48:38.060460 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:38.060344    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"
Jul 08 16:48:38.060460 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:38.060378    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"
Jul 08 16:48:38.061696 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:38.061670    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Jul 08 16:48:38.062298 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:38.062276    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:48:38.084999 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:38.084946    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podStartSLOduration=2.602017166 podStartE2EDuration="6.084931649s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.482914483s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:48:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:48:38.082614719 +0000 UTC m=+837.423579821" watchObservedRunningTime="2026-07-08 16:48:38.084931649 +0000 UTC m=+837.425896727"
Jul 08 16:48:39.065405 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:39.065361    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Jul 08 16:48:39.065781 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:39.065743    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:48:40.542414 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:40.542374    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.28:8643/healthz\": dial tcp 10.134.0.28:8643: connect: connection refused"
Jul 08 16:48:40.546743 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:40.546708    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:5000: connect: connection refused"
Jul 08 16:48:40.546884 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:40.546836    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"
Jul 08 16:48:40.547032 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:40.547008    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:48:40.547132 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:40.547119    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"
Jul 08 16:48:42.816726 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:42.816696    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"
Jul 08 16:48:42.918836 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:42.918743    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-proxy-tls\") pod \"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6\" (UID: \"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6\") "
Jul 08 16:48:42.918836 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:42.918812    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-kube-rbac-proxy-sar-config\") pod \"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6\" (UID: \"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6\") "
Jul 08 16:48:42.919065 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:42.918855    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nsfjz\" (UniqueName: \"kubernetes.io/projected/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-kube-api-access-nsfjz\") pod \"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6\" (UID: \"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6\") "
Jul 08 16:48:42.919065 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:42.918872    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-cabundle-cert\") pod \"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6\" (UID: \"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6\") "
Jul 08 16:48:42.919065 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:42.919033    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-kserve-provision-location\") pod \"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6\" (UID: \"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6\") "
Jul 08 16:48:42.919282 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:42.919252    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" (UID: "7cf5ee2c-2516-4231-af20-dd1c8b77aaa6"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:48:42.919428 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:42.919303    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" (UID: "7cf5ee2c-2516-4231-af20-dd1c8b77aaa6"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 16:48:42.919428 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:42.919351    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" (UID: "7cf5ee2c-2516-4231-af20-dd1c8b77aaa6"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:48:42.919511 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:42.919429    2582 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:48:42.919511 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:42.919444    2582 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-cabundle-cert\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:48:42.919511 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:42.919453    2582 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-kserve-provision-location\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:48:42.921077 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:42.921055    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" (UID: "7cf5ee2c-2516-4231-af20-dd1c8b77aaa6"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 16:48:42.921166 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:42.921095    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-kube-api-access-nsfjz" (OuterVolumeSpecName: "kube-api-access-nsfjz") pod "7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" (UID: "7cf5ee2c-2516-4231-af20-dd1c8b77aaa6"). InnerVolumeSpecName "kube-api-access-nsfjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 16:48:43.019910 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:43.019878    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsfjz\" (UniqueName: \"kubernetes.io/projected/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-kube-api-access-nsfjz\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:48:43.019910 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:43.019906    2582 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6-proxy-tls\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:48:43.095457 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:43.095424    2582 generic.go:356] "Generic (PLEG): container finished" podID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerID="730317f1db1f9a87d3201b26d234c3e7c27b056c888091d73fa8451195269ee9" exitCode=0
Jul 08 16:48:43.095616 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:43.095472    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" event={"ID":"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6","Type":"ContainerDied","Data":"730317f1db1f9a87d3201b26d234c3e7c27b056c888091d73fa8451195269ee9"}
Jul 08 16:48:43.095616 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:43.095496    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"
Jul 08 16:48:43.095616 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:43.095547    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6" event={"ID":"7cf5ee2c-2516-4231-af20-dd1c8b77aaa6","Type":"ContainerDied","Data":"16447d54a9bbec7d528844b0be9abc6c0273b19134f016ee50628161123dd69f"}
Jul 08 16:48:43.095616 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:43.095578    2582 scope.go:117] "RemoveContainer" containerID="730317f1db1f9a87d3201b26d234c3e7c27b056c888091d73fa8451195269ee9"
Jul 08 16:48:43.104983 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:43.104964    2582 scope.go:117] "RemoveContainer" containerID="8048146f3aa0bd0bcc5fc8fc7cea822a2713265713becca7f4e17df7cef59779"
Jul 08 16:48:43.115779 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:43.115751    2582 scope.go:117] "RemoveContainer" containerID="8dca7e818f52a4f2809a5f36d660bf58cd8fccbc51ccd96a7eca7c6746fb17f6"
Jul 08 16:48:43.123364 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:43.123340    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"]
Jul 08 16:48:43.124381 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:43.124364    2582 scope.go:117] "RemoveContainer" containerID="1607cc1fa805399f78791d739e6c49db866ec33733b8430ce389d4cfca91d31c"
Jul 08 16:48:43.127139 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:43.127121    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-86d75f96d4-zcrk6"]
Jul 08 16:48:43.132592 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:43.132569    2582 scope.go:117] "RemoveContainer" containerID="730317f1db1f9a87d3201b26d234c3e7c27b056c888091d73fa8451195269ee9"
Jul 08 16:48:43.132847 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:48:43.132829    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"730317f1db1f9a87d3201b26d234c3e7c27b056c888091d73fa8451195269ee9\": container with ID starting with 730317f1db1f9a87d3201b26d234c3e7c27b056c888091d73fa8451195269ee9 not found: ID does not exist" containerID="730317f1db1f9a87d3201b26d234c3e7c27b056c888091d73fa8451195269ee9"
Jul 08 16:48:43.132917 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:43.132858    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"730317f1db1f9a87d3201b26d234c3e7c27b056c888091d73fa8451195269ee9"} err="failed to get container status \"730317f1db1f9a87d3201b26d234c3e7c27b056c888091d73fa8451195269ee9\": rpc error: code = NotFound desc = could not find container \"730317f1db1f9a87d3201b26d234c3e7c27b056c888091d73fa8451195269ee9\": container with ID starting with 730317f1db1f9a87d3201b26d234c3e7c27b056c888091d73fa8451195269ee9 not found: ID does not exist"
Jul 08 16:48:43.132917 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:43.132876    2582 scope.go:117] "RemoveContainer" containerID="8048146f3aa0bd0bcc5fc8fc7cea822a2713265713becca7f4e17df7cef59779"
Jul 08 16:48:43.133106 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:48:43.133087    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8048146f3aa0bd0bcc5fc8fc7cea822a2713265713becca7f4e17df7cef59779\": container with ID starting with 8048146f3aa0bd0bcc5fc8fc7cea822a2713265713becca7f4e17df7cef59779 not found: ID does not exist" containerID="8048146f3aa0bd0bcc5fc8fc7cea822a2713265713becca7f4e17df7cef59779"
Jul 08 16:48:43.133155 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:43.133113    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8048146f3aa0bd0bcc5fc8fc7cea822a2713265713becca7f4e17df7cef59779"} err="failed to get container status \"8048146f3aa0bd0bcc5fc8fc7cea822a2713265713becca7f4e17df7cef59779\": rpc error: code = NotFound desc = could not find container \"8048146f3aa0bd0bcc5fc8fc7cea822a2713265713becca7f4e17df7cef59779\": container with ID starting with 8048146f3aa0bd0bcc5fc8fc7cea822a2713265713becca7f4e17df7cef59779 not found: ID does not exist"
Jul 08 16:48:43.133155 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:43.133131    2582 scope.go:117] "RemoveContainer" containerID="8dca7e818f52a4f2809a5f36d660bf58cd8fccbc51ccd96a7eca7c6746fb17f6"
Jul 08 16:48:43.133369 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:48:43.133351    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dca7e818f52a4f2809a5f36d660bf58cd8fccbc51ccd96a7eca7c6746fb17f6\": container with ID starting with 8dca7e818f52a4f2809a5f36d660bf58cd8fccbc51ccd96a7eca7c6746fb17f6 not found: ID does not exist" containerID="8dca7e818f52a4f2809a5f36d660bf58cd8fccbc51ccd96a7eca7c6746fb17f6"
Jul 08 16:48:43.133418 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:43.133378    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dca7e818f52a4f2809a5f36d660bf58cd8fccbc51ccd96a7eca7c6746fb17f6"} err="failed to get container status \"8dca7e818f52a4f2809a5f36d660bf58cd8fccbc51ccd96a7eca7c6746fb17f6\": rpc error: code = NotFound desc = could not find container \"8dca7e818f52a4f2809a5f36d660bf58cd8fccbc51ccd96a7eca7c6746fb17f6\": container with ID starting with 8dca7e818f52a4f2809a5f36d660bf58cd8fccbc51ccd96a7eca7c6746fb17f6 not found: ID does not exist"
Jul 08 16:48:43.133418 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:43.133394    2582 scope.go:117] "RemoveContainer" containerID="1607cc1fa805399f78791d739e6c49db866ec33733b8430ce389d4cfca91d31c"
Jul 08 16:48:43.133643 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:48:43.133627    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1607cc1fa805399f78791d739e6c49db866ec33733b8430ce389d4cfca91d31c\": container with ID starting with 1607cc1fa805399f78791d739e6c49db866ec33733b8430ce389d4cfca91d31c not found: ID does not exist" containerID="1607cc1fa805399f78791d739e6c49db866ec33733b8430ce389d4cfca91d31c"
Jul 08 16:48:43.133679 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:43.133647    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1607cc1fa805399f78791d739e6c49db866ec33733b8430ce389d4cfca91d31c"} err="failed to get container status \"1607cc1fa805399f78791d739e6c49db866ec33733b8430ce389d4cfca91d31c\": rpc error: code = NotFound desc = could not find container \"1607cc1fa805399f78791d739e6c49db866ec33733b8430ce389d4cfca91d31c\": container with ID starting with 1607cc1fa805399f78791d739e6c49db866ec33733b8430ce389d4cfca91d31c not found: ID does not exist"
Jul 08 16:48:43.216715 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:43.216642    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" path="/var/lib/kubelet/pods/7cf5ee2c-2516-4231-af20-dd1c8b77aaa6/volumes"
Jul 08 16:48:44.070654 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:44.070360    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"
Jul 08 16:48:44.071049 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:44.070928    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Jul 08 16:48:44.071580 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:44.071545    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:48:54.071550 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:54.071506    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Jul 08 16:48:54.072028 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:48:54.071939    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:49:04.071902 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:49:04.071854    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Jul 08 16:49:04.072512 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:49:04.072426    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:49:14.071306 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:49:14.071263    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Jul 08 16:49:14.071777 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:49:14.071754    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:49:24.071417 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:49:24.071362    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Jul 08 16:49:24.071926 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:49:24.071900    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:49:34.071273 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:49:34.071173    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Jul 08 16:49:34.071745 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:49:34.071627    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:49:41.122045 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:49:41.122004    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kvhpr_19df7751-4b62-4ae5-92a6-0bbda11fec85/ovn-acl-logging/0.log"
Jul 08 16:49:41.124200 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:49:41.124180    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kvhpr_19df7751-4b62-4ae5-92a6-0bbda11fec85/ovn-acl-logging/0.log"
Jul 08 16:49:44.071755 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:49:44.071711    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Jul 08 16:49:44.072224 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:49:44.072192    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:49:54.071937 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:49:54.071903    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"
Jul 08 16:49:54.072390 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:49:54.072175    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"
Jul 08 16:50:07.844192 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:07.844158    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_message-dumper-predictor-6575bcfdfd-mqzx2_873bbe5a-e8a1-420f-aa8f-84bcaf656116/kserve-container/0.log"
Jul 08 16:50:08.076922 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.076885    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"]
Jul 08 16:50:08.077185 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.077137    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="kserve-container" containerID="cri-o://104ec5836b89d95366de58050f565892eb97fded6d4168c90fbec4170db0f8fb" gracePeriod=30
Jul 08 16:50:08.077185 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.077155    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="agent" containerID="cri-o://adb2032b34480a345de76f7b84549dc798944e4750bee5e5bae2d251988fb9eb" gracePeriod=30
Jul 08 16:50:08.077428 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.077155    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="kube-rbac-proxy" containerID="cri-o://5cf74a3b48f936a951baa9137636a992d48f6cbe6521182f0f3d08c3d2df0311" gracePeriod=30
Jul 08 16:50:08.157850 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.157758    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q"]
Jul 08 16:50:08.158362 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.158337    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="kserve-container"
Jul 08 16:50:08.158362 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.158364    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="kserve-container"
Jul 08 16:50:08.158538 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.158395    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="agent"
Jul 08 16:50:08.158538 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.158401    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="agent"
Jul 08 16:50:08.158538 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.158422    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="storage-initializer"
Jul 08 16:50:08.158538 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.158432    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="storage-initializer"
Jul 08 16:50:08.158538 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.158446    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="kube-rbac-proxy"
Jul 08 16:50:08.158538 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.158451    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="kube-rbac-proxy"
Jul 08 16:50:08.158538 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.158524    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="kube-rbac-proxy"
Jul 08 16:50:08.158538 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.158531    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="agent"
Jul 08 16:50:08.158538 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.158538    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cf5ee2c-2516-4231-af20-dd1c8b77aaa6" containerName="kserve-container"
Jul 08 16:50:08.161778 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.161761    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q"
Jul 08 16:50:08.166077 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.165413    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v1-predictor-serving-cert\""
Jul 08 16:50:08.166077 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.165751    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v1-kube-rbac-proxy-sar-config\""
Jul 08 16:50:08.168556 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.168395    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2"]
Jul 08 16:50:08.168867 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.168782    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2" podUID="873bbe5a-e8a1-420f-aa8f-84bcaf656116" containerName="kserve-container" containerID="cri-o://ee5440eda8814477d14de543be73baea85e212bc16841aa31ac701e46047f2a4" gracePeriod=30
Jul 08 16:50:08.169097 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.169007    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2" podUID="873bbe5a-e8a1-420f-aa8f-84bcaf656116" containerName="kube-rbac-proxy" containerID="cri-o://4cfcbbc53116a8c32877e4ac83b8ca91e56acf12f8263d853f3de4f1864f8940" gracePeriod=30
Jul 08 16:50:08.173150 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.173122    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q"]
Jul 08 16:50:08.277494 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.277464    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5271cd43-db59-417f-88ec-336654783e7b-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q\" (UID: \"5271cd43-db59-417f-88ec-336654783e7b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q"
Jul 08 16:50:08.277808 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.277776    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5271cd43-db59-417f-88ec-336654783e7b-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q\" (UID: \"5271cd43-db59-417f-88ec-336654783e7b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q"
Jul 08 16:50:08.277935 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.277847    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5271cd43-db59-417f-88ec-336654783e7b-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q\" (UID: \"5271cd43-db59-417f-88ec-336654783e7b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q"
Jul 08 16:50:08.277935 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.277889    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5271cd43-db59-417f-88ec-336654783e7b-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q\" (UID: \"5271cd43-db59-417f-88ec-336654783e7b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q"
Jul 08 16:50:08.277935 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.277916    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dq98\" (UniqueName: \"kubernetes.io/projected/5271cd43-db59-417f-88ec-336654783e7b-kube-api-access-6dq98\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q\" (UID: \"5271cd43-db59-417f-88ec-336654783e7b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q"
Jul 08 16:50:08.379606 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.379555    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5271cd43-db59-417f-88ec-336654783e7b-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q\" (UID: \"5271cd43-db59-417f-88ec-336654783e7b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q"
Jul 08 16:50:08.379606 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.379605    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5271cd43-db59-417f-88ec-336654783e7b-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q\" (UID: \"5271cd43-db59-417f-88ec-336654783e7b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q"
Jul 08 16:50:08.379836 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.379634    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5271cd43-db59-417f-88ec-336654783e7b-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q\" (UID: \"5271cd43-db59-417f-88ec-336654783e7b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q"
Jul 08 16:50:08.379836 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.379655    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dq98\" (UniqueName: \"kubernetes.io/projected/5271cd43-db59-417f-88ec-336654783e7b-kube-api-access-6dq98\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q\" (UID: \"5271cd43-db59-417f-88ec-336654783e7b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q"
Jul 08 16:50:08.379836 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.379742    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5271cd43-db59-417f-88ec-336654783e7b-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q\" (UID: \"5271cd43-db59-417f-88ec-336654783e7b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q"
Jul 08 16:50:08.380339 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.380296    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5271cd43-db59-417f-88ec-336654783e7b-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q\" (UID: \"5271cd43-db59-417f-88ec-336654783e7b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q"
Jul 08 16:50:08.380459 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.380305    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5271cd43-db59-417f-88ec-336654783e7b-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q\" (UID: \"5271cd43-db59-417f-88ec-336654783e7b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q"
Jul 08 16:50:08.380526 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.380451    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5271cd43-db59-417f-88ec-336654783e7b-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q\" (UID: \"5271cd43-db59-417f-88ec-336654783e7b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q"
Jul 08 16:50:08.382489 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.382466    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5271cd43-db59-417f-88ec-336654783e7b-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q\" (UID: \"5271cd43-db59-417f-88ec-336654783e7b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q"
Jul 08 16:50:08.388111 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.388087    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dq98\" (UniqueName: \"kubernetes.io/projected/5271cd43-db59-417f-88ec-336654783e7b-kube-api-access-6dq98\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q\" (UID: \"5271cd43-db59-417f-88ec-336654783e7b\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q"
Jul 08 16:50:08.416668 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.416647    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2"
Jul 08 16:50:08.480384 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.480354    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trf6g\" (UniqueName: \"kubernetes.io/projected/873bbe5a-e8a1-420f-aa8f-84bcaf656116-kube-api-access-trf6g\") pod \"873bbe5a-e8a1-420f-aa8f-84bcaf656116\" (UID: \"873bbe5a-e8a1-420f-aa8f-84bcaf656116\") "
Jul 08 16:50:08.480570 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.480407    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/873bbe5a-e8a1-420f-aa8f-84bcaf656116-kube-rbac-proxy-sar-config\") pod \"873bbe5a-e8a1-420f-aa8f-84bcaf656116\" (UID: \"873bbe5a-e8a1-420f-aa8f-84bcaf656116\") "
Jul 08 16:50:08.480570 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.480465    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/873bbe5a-e8a1-420f-aa8f-84bcaf656116-proxy-tls\") pod \"873bbe5a-e8a1-420f-aa8f-84bcaf656116\" (UID: \"873bbe5a-e8a1-420f-aa8f-84bcaf656116\") "
Jul 08 16:50:08.480860 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.480833    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/873bbe5a-e8a1-420f-aa8f-84bcaf656116-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "873bbe5a-e8a1-420f-aa8f-84bcaf656116" (UID: "873bbe5a-e8a1-420f-aa8f-84bcaf656116"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:50:08.482693 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.482672    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/873bbe5a-e8a1-420f-aa8f-84bcaf656116-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "873bbe5a-e8a1-420f-aa8f-84bcaf656116" (UID: "873bbe5a-e8a1-420f-aa8f-84bcaf656116"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 16:50:08.482991 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.482967    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/873bbe5a-e8a1-420f-aa8f-84bcaf656116-kube-api-access-trf6g" (OuterVolumeSpecName: "kube-api-access-trf6g") pod "873bbe5a-e8a1-420f-aa8f-84bcaf656116" (UID: "873bbe5a-e8a1-420f-aa8f-84bcaf656116"). InnerVolumeSpecName "kube-api-access-trf6g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 16:50:08.487089 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.487063    2582 generic.go:356] "Generic (PLEG): container finished" podID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerID="5cf74a3b48f936a951baa9137636a992d48f6cbe6521182f0f3d08c3d2df0311" exitCode=2
Jul 08 16:50:08.487181 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.487138    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" event={"ID":"06bfd315-7e36-4511-b9d7-c2f488b06b8a","Type":"ContainerDied","Data":"5cf74a3b48f936a951baa9137636a992d48f6cbe6521182f0f3d08c3d2df0311"}
Jul 08 16:50:08.488630 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.488611    2582 generic.go:356] "Generic (PLEG): container finished" podID="873bbe5a-e8a1-420f-aa8f-84bcaf656116" containerID="4cfcbbc53116a8c32877e4ac83b8ca91e56acf12f8263d853f3de4f1864f8940" exitCode=2
Jul 08 16:50:08.488630 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.488629    2582 generic.go:356] "Generic (PLEG): container finished" podID="873bbe5a-e8a1-420f-aa8f-84bcaf656116" containerID="ee5440eda8814477d14de543be73baea85e212bc16841aa31ac701e46047f2a4" exitCode=2
Jul 08 16:50:08.488750 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.488647    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2" event={"ID":"873bbe5a-e8a1-420f-aa8f-84bcaf656116","Type":"ContainerDied","Data":"4cfcbbc53116a8c32877e4ac83b8ca91e56acf12f8263d853f3de4f1864f8940"}
Jul 08 16:50:08.488750 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.488659    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2"
Jul 08 16:50:08.488750 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.488689    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2" event={"ID":"873bbe5a-e8a1-420f-aa8f-84bcaf656116","Type":"ContainerDied","Data":"ee5440eda8814477d14de543be73baea85e212bc16841aa31ac701e46047f2a4"}
Jul 08 16:50:08.488750 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.488709    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2" event={"ID":"873bbe5a-e8a1-420f-aa8f-84bcaf656116","Type":"ContainerDied","Data":"3ef5f09464cbc8f396646a6437f72c85aff97e4391bd3edf00556aaa0cdfa9c9"}
Jul 08 16:50:08.488750 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.488731    2582 scope.go:117] "RemoveContainer" containerID="4cfcbbc53116a8c32877e4ac83b8ca91e56acf12f8263d853f3de4f1864f8940"
Jul 08 16:50:08.497562 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.497544    2582 scope.go:117] "RemoveContainer" containerID="ee5440eda8814477d14de543be73baea85e212bc16841aa31ac701e46047f2a4"
Jul 08 16:50:08.505633 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.505612    2582 scope.go:117] "RemoveContainer" containerID="4cfcbbc53116a8c32877e4ac83b8ca91e56acf12f8263d853f3de4f1864f8940"
Jul 08 16:50:08.505871 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:50:08.505853    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cfcbbc53116a8c32877e4ac83b8ca91e56acf12f8263d853f3de4f1864f8940\": container with ID starting with 4cfcbbc53116a8c32877e4ac83b8ca91e56acf12f8263d853f3de4f1864f8940 not found: ID does not exist" containerID="4cfcbbc53116a8c32877e4ac83b8ca91e56acf12f8263d853f3de4f1864f8940"
Jul 08 16:50:08.505920 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.505879    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cfcbbc53116a8c32877e4ac83b8ca91e56acf12f8263d853f3de4f1864f8940"} err="failed to get container status \"4cfcbbc53116a8c32877e4ac83b8ca91e56acf12f8263d853f3de4f1864f8940\": rpc error: code = NotFound desc = could not find container \"4cfcbbc53116a8c32877e4ac83b8ca91e56acf12f8263d853f3de4f1864f8940\": container with ID starting with 4cfcbbc53116a8c32877e4ac83b8ca91e56acf12f8263d853f3de4f1864f8940 not found: ID does not exist"
Jul 08 16:50:08.505920 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.505897    2582 scope.go:117] "RemoveContainer" containerID="ee5440eda8814477d14de543be73baea85e212bc16841aa31ac701e46047f2a4"
Jul 08 16:50:08.506155 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:50:08.506137    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee5440eda8814477d14de543be73baea85e212bc16841aa31ac701e46047f2a4\": container with ID starting with ee5440eda8814477d14de543be73baea85e212bc16841aa31ac701e46047f2a4 not found: ID does not exist" containerID="ee5440eda8814477d14de543be73baea85e212bc16841aa31ac701e46047f2a4"
Jul 08 16:50:08.506194 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.506161    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee5440eda8814477d14de543be73baea85e212bc16841aa31ac701e46047f2a4"} err="failed to get container status \"ee5440eda8814477d14de543be73baea85e212bc16841aa31ac701e46047f2a4\": rpc error: code = NotFound desc = could not find container \"ee5440eda8814477d14de543be73baea85e212bc16841aa31ac701e46047f2a4\": container with ID starting with ee5440eda8814477d14de543be73baea85e212bc16841aa31ac701e46047f2a4 not found: ID does not exist"
Jul 08 16:50:08.506194 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.506178    2582 scope.go:117] "RemoveContainer" containerID="4cfcbbc53116a8c32877e4ac83b8ca91e56acf12f8263d853f3de4f1864f8940"
Jul 08 16:50:08.506486 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.506431    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cfcbbc53116a8c32877e4ac83b8ca91e56acf12f8263d853f3de4f1864f8940"} err="failed to get container status \"4cfcbbc53116a8c32877e4ac83b8ca91e56acf12f8263d853f3de4f1864f8940\": rpc error: code = NotFound desc = could not find container \"4cfcbbc53116a8c32877e4ac83b8ca91e56acf12f8263d853f3de4f1864f8940\": container with ID starting with 4cfcbbc53116a8c32877e4ac83b8ca91e56acf12f8263d853f3de4f1864f8940 not found: ID does not exist"
Jul 08 16:50:08.506486 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.506452    2582 scope.go:117] "RemoveContainer" containerID="ee5440eda8814477d14de543be73baea85e212bc16841aa31ac701e46047f2a4"
Jul 08 16:50:08.506870 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.506796    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee5440eda8814477d14de543be73baea85e212bc16841aa31ac701e46047f2a4"} err="failed to get container status \"ee5440eda8814477d14de543be73baea85e212bc16841aa31ac701e46047f2a4\": rpc error: code = NotFound desc = could not find container \"ee5440eda8814477d14de543be73baea85e212bc16841aa31ac701e46047f2a4\": container with ID starting with ee5440eda8814477d14de543be73baea85e212bc16841aa31ac701e46047f2a4 not found: ID does not exist"
Jul 08 16:50:08.509112 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.509088    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2"]
Jul 08 16:50:08.512469 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.512449    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-mqzx2"]
Jul 08 16:50:08.554679 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.554645    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q"
Jul 08 16:50:08.581887 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.581858    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trf6g\" (UniqueName: \"kubernetes.io/projected/873bbe5a-e8a1-420f-aa8f-84bcaf656116-kube-api-access-trf6g\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:50:08.581887 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.581884    2582 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/873bbe5a-e8a1-420f-aa8f-84bcaf656116-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:50:08.582106 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.581895    2582 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/873bbe5a-e8a1-420f-aa8f-84bcaf656116-proxy-tls\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:50:08.689109 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:08.689034    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q"]
Jul 08 16:50:08.692600 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:50:08.692575    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5271cd43_db59_417f_88ec_336654783e7b.slice/crio-96d5bcfdf9e85bcc747827341c4dba5bee4a87b4a1338537bc88d8e56eb8e995 WatchSource:0}: Error finding container 96d5bcfdf9e85bcc747827341c4dba5bee4a87b4a1338537bc88d8e56eb8e995: Status 404 returned error can't find the container with id 96d5bcfdf9e85bcc747827341c4dba5bee4a87b4a1338537bc88d8e56eb8e995
Jul 08 16:50:09.065853 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:09.065757    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.30:8643/healthz\": dial tcp 10.134.0.30:8643: connect: connection refused"
Jul 08 16:50:09.218699 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:09.218664    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="873bbe5a-e8a1-420f-aa8f-84bcaf656116" path="/var/lib/kubelet/pods/873bbe5a-e8a1-420f-aa8f-84bcaf656116/volumes"
Jul 08 16:50:09.498659 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:09.498621    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q" event={"ID":"5271cd43-db59-417f-88ec-336654783e7b","Type":"ContainerStarted","Data":"d04f32cc44b7fb54d2c41be9d898358cc24220f6953a73517f90d5854e8f2322"}
Jul 08 16:50:09.498831 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:09.498689    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q" event={"ID":"5271cd43-db59-417f-88ec-336654783e7b","Type":"ContainerStarted","Data":"96d5bcfdf9e85bcc747827341c4dba5bee4a87b4a1338537bc88d8e56eb8e995"}
Jul 08 16:50:13.520153 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:13.520112    2582 generic.go:356] "Generic (PLEG): container finished" podID="5271cd43-db59-417f-88ec-336654783e7b" containerID="d04f32cc44b7fb54d2c41be9d898358cc24220f6953a73517f90d5854e8f2322" exitCode=0
Jul 08 16:50:13.520595 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:13.520182    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q" event={"ID":"5271cd43-db59-417f-88ec-336654783e7b","Type":"ContainerDied","Data":"d04f32cc44b7fb54d2c41be9d898358cc24220f6953a73517f90d5854e8f2322"}
Jul 08 16:50:13.522982 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:13.522954    2582 generic.go:356] "Generic (PLEG): container finished" podID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerID="104ec5836b89d95366de58050f565892eb97fded6d4168c90fbec4170db0f8fb" exitCode=0
Jul 08 16:50:13.523072 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:13.523002    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" event={"ID":"06bfd315-7e36-4511-b9d7-c2f488b06b8a","Type":"ContainerDied","Data":"104ec5836b89d95366de58050f565892eb97fded6d4168c90fbec4170db0f8fb"}
Jul 08 16:50:14.066494 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:14.066087    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.30:8643/healthz\": dial tcp 10.134.0.30:8643: connect: connection refused"
Jul 08 16:50:14.073549 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:14.072611    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Jul 08 16:50:14.073748 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:14.073608    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:50:19.065744 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:19.065696    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.30:8643/healthz\": dial tcp 10.134.0.30:8643: connect: connection refused"
Jul 08 16:50:19.066125 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:19.065875    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"
Jul 08 16:50:24.065495 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:24.065452    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.30:8643/healthz\": dial tcp 10.134.0.30:8643: connect: connection refused"
Jul 08 16:50:24.071959 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:24.071926    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Jul 08 16:50:24.072392 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:24.072358    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:50:29.066547 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:29.066506    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.30:8643/healthz\": dial tcp 10.134.0.30:8643: connect: connection refused"
Jul 08 16:50:34.065675 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:34.065631    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.30:8643/healthz\": dial tcp 10.134.0.30:8643: connect: connection refused"
Jul 08 16:50:34.070982 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:34.070958    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Jul 08 16:50:34.071101 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:34.071060    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"
Jul 08 16:50:34.071383 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:34.071357    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 16:50:34.071484 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:34.071473    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"
Jul 08 16:50:38.665090 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:38.665055    2582 generic.go:356] "Generic (PLEG): container finished" podID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerID="adb2032b34480a345de76f7b84549dc798944e4750bee5e5bae2d251988fb9eb" exitCode=0
Jul 08 16:50:38.665559 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:38.665130    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" event={"ID":"06bfd315-7e36-4511-b9d7-c2f488b06b8a","Type":"ContainerDied","Data":"adb2032b34480a345de76f7b84549dc798944e4750bee5e5bae2d251988fb9eb"}
Jul 08 16:50:38.777974 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:38.777946    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"
Jul 08 16:50:38.848084 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:38.848050    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/06bfd315-7e36-4511-b9d7-c2f488b06b8a-proxy-tls\") pod \"06bfd315-7e36-4511-b9d7-c2f488b06b8a\" (UID: \"06bfd315-7e36-4511-b9d7-c2f488b06b8a\") "
Jul 08 16:50:38.848084 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:38.848087    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pklq5\" (UniqueName: \"kubernetes.io/projected/06bfd315-7e36-4511-b9d7-c2f488b06b8a-kube-api-access-pklq5\") pod \"06bfd315-7e36-4511-b9d7-c2f488b06b8a\" (UID: \"06bfd315-7e36-4511-b9d7-c2f488b06b8a\") "
Jul 08 16:50:38.848298 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:38.848105    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/06bfd315-7e36-4511-b9d7-c2f488b06b8a-cabundle-cert\") pod \"06bfd315-7e36-4511-b9d7-c2f488b06b8a\" (UID: \"06bfd315-7e36-4511-b9d7-c2f488b06b8a\") "
Jul 08 16:50:38.848298 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:38.848126    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/06bfd315-7e36-4511-b9d7-c2f488b06b8a-kube-rbac-proxy-sar-config\") pod \"06bfd315-7e36-4511-b9d7-c2f488b06b8a\" (UID: \"06bfd315-7e36-4511-b9d7-c2f488b06b8a\") "
Jul 08 16:50:38.848298 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:38.848205    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/06bfd315-7e36-4511-b9d7-c2f488b06b8a-kserve-provision-location\") pod \"06bfd315-7e36-4511-b9d7-c2f488b06b8a\" (UID: \"06bfd315-7e36-4511-b9d7-c2f488b06b8a\") "
Jul 08 16:50:38.848563 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:38.848535    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06bfd315-7e36-4511-b9d7-c2f488b06b8a-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "06bfd315-7e36-4511-b9d7-c2f488b06b8a" (UID: "06bfd315-7e36-4511-b9d7-c2f488b06b8a"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:50:38.848632 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:38.848565    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06bfd315-7e36-4511-b9d7-c2f488b06b8a-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "06bfd315-7e36-4511-b9d7-c2f488b06b8a" (UID: "06bfd315-7e36-4511-b9d7-c2f488b06b8a"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 16:50:38.848632 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:38.848576    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06bfd315-7e36-4511-b9d7-c2f488b06b8a-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "06bfd315-7e36-4511-b9d7-c2f488b06b8a" (UID: "06bfd315-7e36-4511-b9d7-c2f488b06b8a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:50:38.850404 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:38.850377    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06bfd315-7e36-4511-b9d7-c2f488b06b8a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "06bfd315-7e36-4511-b9d7-c2f488b06b8a" (UID: "06bfd315-7e36-4511-b9d7-c2f488b06b8a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 16:50:38.850602 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:38.850582    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06bfd315-7e36-4511-b9d7-c2f488b06b8a-kube-api-access-pklq5" (OuterVolumeSpecName: "kube-api-access-pklq5") pod "06bfd315-7e36-4511-b9d7-c2f488b06b8a" (UID: "06bfd315-7e36-4511-b9d7-c2f488b06b8a"). InnerVolumeSpecName "kube-api-access-pklq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 16:50:38.949188 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:38.949151    2582 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/06bfd315-7e36-4511-b9d7-c2f488b06b8a-proxy-tls\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:50:38.949188 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:38.949188    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pklq5\" (UniqueName: \"kubernetes.io/projected/06bfd315-7e36-4511-b9d7-c2f488b06b8a-kube-api-access-pklq5\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:50:38.949424 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:38.949203    2582 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/06bfd315-7e36-4511-b9d7-c2f488b06b8a-cabundle-cert\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:50:38.949424 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:38.949215    2582 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/06bfd315-7e36-4511-b9d7-c2f488b06b8a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:50:38.949424 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:38.949226    2582 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/06bfd315-7e36-4511-b9d7-c2f488b06b8a-kserve-provision-location\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:50:39.673973 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:39.673934    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq" event={"ID":"06bfd315-7e36-4511-b9d7-c2f488b06b8a","Type":"ContainerDied","Data":"221f1e88e8f77ac617a24a830a9e154dce2b75b697118c306ed455162c8f7917"}
Jul 08 16:50:39.673973 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:39.673968    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"
Jul 08 16:50:39.674506 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:39.674036    2582 scope.go:117] "RemoveContainer" containerID="adb2032b34480a345de76f7b84549dc798944e4750bee5e5bae2d251988fb9eb"
Jul 08 16:50:39.684196 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:39.684085    2582 scope.go:117] "RemoveContainer" containerID="5cf74a3b48f936a951baa9137636a992d48f6cbe6521182f0f3d08c3d2df0311"
Jul 08 16:50:39.693552 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:39.693512    2582 scope.go:117] "RemoveContainer" containerID="104ec5836b89d95366de58050f565892eb97fded6d4168c90fbec4170db0f8fb"
Jul 08 16:50:39.695711 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:39.695640    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"]
Jul 08 16:50:39.700131 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:39.700109    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-786d9c95cd-vznlq"]
Jul 08 16:50:39.703427 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:39.703302    2582 scope.go:117] "RemoveContainer" containerID="5babb2aac454e8cbdbf83632b1f9b665e9523601b482bdc1d04918e4384a0cde"
Jul 08 16:50:41.218070 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:50:41.218039    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" path="/var/lib/kubelet/pods/06bfd315-7e36-4511-b9d7-c2f488b06b8a/volumes"
Jul 08 16:52:23.188024 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:23.187993    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q" event={"ID":"5271cd43-db59-417f-88ec-336654783e7b","Type":"ContainerStarted","Data":"0a8ade175cf6db4a5c03d0791a78f0c18dfabc2c59fc36faa489c3134c0cd4bb"}
Jul 08 16:52:23.188546 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:23.188055    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q" event={"ID":"5271cd43-db59-417f-88ec-336654783e7b","Type":"ContainerStarted","Data":"2d9412740a73fe082b1cc7d6d80e57c91d868fcf7a4f4fb727e7faddc3ce57bc"}
Jul 08 16:52:23.188546 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:23.188172    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q"
Jul 08 16:52:23.188546 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:23.188228    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q"
Jul 08 16:52:23.189531 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:23.189507    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q" podUID="5271cd43-db59-417f-88ec-336654783e7b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Jul 08 16:52:23.218963 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:23.218903    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q" podStartSLOduration=1.790359295 podStartE2EDuration="2m15.218883599s" totalImagesPullingTime="2m8.869615374s" totalInitContainerRuntime="4.55890893s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:50:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:52:23.216849481 +0000 UTC m=+1062.557814561" watchObservedRunningTime="2026-07-08 16:52:23.218883599 +0000 UTC m=+1062.559848680"
Jul 08 16:52:24.194002 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:24.193963    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q" podUID="5271cd43-db59-417f-88ec-336654783e7b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Jul 08 16:52:29.200361 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:29.200332    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q"
Jul 08 16:52:29.200977 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:29.200880    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q" podUID="5271cd43-db59-417f-88ec-336654783e7b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Jul 08 16:52:39.201552 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:39.201467    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q"
Jul 08 16:52:48.310197 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:48.310155    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q"]
Jul 08 16:52:48.310721 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:48.310482    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q" podUID="5271cd43-db59-417f-88ec-336654783e7b" containerName="kserve-container" containerID="cri-o://2d9412740a73fe082b1cc7d6d80e57c91d868fcf7a4f4fb727e7faddc3ce57bc" gracePeriod=30
Jul 08 16:52:48.310721 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:48.310497    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q" podUID="5271cd43-db59-417f-88ec-336654783e7b" containerName="kube-rbac-proxy" containerID="cri-o://0a8ade175cf6db4a5c03d0791a78f0c18dfabc2c59fc36faa489c3134c0cd4bb" gracePeriod=30
Jul 08 16:52:49.194897 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:49.194852    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q" podUID="5271cd43-db59-417f-88ec-336654783e7b" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.31:8643/healthz\": dial tcp 10.134.0.31:8643: connect: connection refused"
Jul 08 16:52:49.201171 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:49.201147    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q" podUID="5271cd43-db59-417f-88ec-336654783e7b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Jul 08 16:52:49.309173 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:49.309090    2582 generic.go:356] "Generic (PLEG): container finished" podID="5271cd43-db59-417f-88ec-336654783e7b" containerID="0a8ade175cf6db4a5c03d0791a78f0c18dfabc2c59fc36faa489c3134c0cd4bb" exitCode=2
Jul 08 16:52:49.309173 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:49.309116    2582 generic.go:356] "Generic (PLEG): container finished" podID="5271cd43-db59-417f-88ec-336654783e7b" containerID="2d9412740a73fe082b1cc7d6d80e57c91d868fcf7a4f4fb727e7faddc3ce57bc" exitCode=0
Jul 08 16:52:49.309395 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:49.309165    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q" event={"ID":"5271cd43-db59-417f-88ec-336654783e7b","Type":"ContainerDied","Data":"0a8ade175cf6db4a5c03d0791a78f0c18dfabc2c59fc36faa489c3134c0cd4bb"}
Jul 08 16:52:49.309395 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:49.309222    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q" event={"ID":"5271cd43-db59-417f-88ec-336654783e7b","Type":"ContainerDied","Data":"2d9412740a73fe082b1cc7d6d80e57c91d868fcf7a4f4fb727e7faddc3ce57bc"}
Jul 08 16:52:49.686869 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:49.686830    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q"
Jul 08 16:52:49.748108 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:49.748069    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5271cd43-db59-417f-88ec-336654783e7b-kube-rbac-proxy-sar-config\") pod \"5271cd43-db59-417f-88ec-336654783e7b\" (UID: \"5271cd43-db59-417f-88ec-336654783e7b\") "
Jul 08 16:52:49.748290 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:49.748124    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5271cd43-db59-417f-88ec-336654783e7b-cabundle-cert\") pod \"5271cd43-db59-417f-88ec-336654783e7b\" (UID: \"5271cd43-db59-417f-88ec-336654783e7b\") "
Jul 08 16:52:49.748290 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:49.748204    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5271cd43-db59-417f-88ec-336654783e7b-proxy-tls\") pod \"5271cd43-db59-417f-88ec-336654783e7b\" (UID: \"5271cd43-db59-417f-88ec-336654783e7b\") "
Jul 08 16:52:49.748290 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:49.748239    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dq98\" (UniqueName: \"kubernetes.io/projected/5271cd43-db59-417f-88ec-336654783e7b-kube-api-access-6dq98\") pod \"5271cd43-db59-417f-88ec-336654783e7b\" (UID: \"5271cd43-db59-417f-88ec-336654783e7b\") "
Jul 08 16:52:49.748290 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:49.748275    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5271cd43-db59-417f-88ec-336654783e7b-kserve-provision-location\") pod \"5271cd43-db59-417f-88ec-336654783e7b\" (UID: \"5271cd43-db59-417f-88ec-336654783e7b\") "
Jul 08 16:52:49.748516 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:49.748482    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5271cd43-db59-417f-88ec-336654783e7b-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "5271cd43-db59-417f-88ec-336654783e7b" (UID: "5271cd43-db59-417f-88ec-336654783e7b"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:52:49.748516 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:49.748489    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5271cd43-db59-417f-88ec-336654783e7b-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "5271cd43-db59-417f-88ec-336654783e7b" (UID: "5271cd43-db59-417f-88ec-336654783e7b"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:52:49.748627 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:49.748572    2582 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5271cd43-db59-417f-88ec-336654783e7b-cabundle-cert\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:52:49.748627 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:49.748593    2582 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5271cd43-db59-417f-88ec-336654783e7b-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:52:49.750402 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:49.750371    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5271cd43-db59-417f-88ec-336654783e7b-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "5271cd43-db59-417f-88ec-336654783e7b" (UID: "5271cd43-db59-417f-88ec-336654783e7b"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 16:52:49.750543 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:49.750525    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5271cd43-db59-417f-88ec-336654783e7b-kube-api-access-6dq98" (OuterVolumeSpecName: "kube-api-access-6dq98") pod "5271cd43-db59-417f-88ec-336654783e7b" (UID: "5271cd43-db59-417f-88ec-336654783e7b"). InnerVolumeSpecName "kube-api-access-6dq98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 16:52:49.753209 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:49.753189    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5271cd43-db59-417f-88ec-336654783e7b-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "5271cd43-db59-417f-88ec-336654783e7b" (UID: "5271cd43-db59-417f-88ec-336654783e7b"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 16:52:49.849169 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:49.849084    2582 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5271cd43-db59-417f-88ec-336654783e7b-proxy-tls\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:52:49.849169 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:49.849116    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dq98\" (UniqueName: \"kubernetes.io/projected/5271cd43-db59-417f-88ec-336654783e7b-kube-api-access-6dq98\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:52:49.849169 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:49.849127    2582 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5271cd43-db59-417f-88ec-336654783e7b-kserve-provision-location\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:52:50.314963 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:50.314934    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q"
Jul 08 16:52:50.315211 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:50.314938    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q" event={"ID":"5271cd43-db59-417f-88ec-336654783e7b","Type":"ContainerDied","Data":"96d5bcfdf9e85bcc747827341c4dba5bee4a87b4a1338537bc88d8e56eb8e995"}
Jul 08 16:52:50.315211 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:50.315068    2582 scope.go:117] "RemoveContainer" containerID="0a8ade175cf6db4a5c03d0791a78f0c18dfabc2c59fc36faa489c3134c0cd4bb"
Jul 08 16:52:50.324228 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:50.324211    2582 scope.go:117] "RemoveContainer" containerID="2d9412740a73fe082b1cc7d6d80e57c91d868fcf7a4f4fb727e7faddc3ce57bc"
Jul 08 16:52:50.331899 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:50.331882    2582 scope.go:117] "RemoveContainer" containerID="d04f32cc44b7fb54d2c41be9d898358cc24220f6953a73517f90d5854e8f2322"
Jul 08 16:52:50.337237 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:50.337212    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q"]
Jul 08 16:52:50.340995 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:50.340976    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-l6s6q"]
Jul 08 16:52:51.217731 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:51.217699    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5271cd43-db59-417f-88ec-336654783e7b" path="/var/lib/kubelet/pods/5271cd43-db59-417f-88ec-336654783e7b/volumes"
Jul 08 16:52:53.716408 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.716373    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl"]
Jul 08 16:52:53.716815 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.716740    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="agent"
Jul 08 16:52:53.716815 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.716755    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="agent"
Jul 08 16:52:53.716815 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.716768    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5271cd43-db59-417f-88ec-336654783e7b" containerName="storage-initializer"
Jul 08 16:52:53.716815 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.716774    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5271cd43-db59-417f-88ec-336654783e7b" containerName="storage-initializer"
Jul 08 16:52:53.716815 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.716784    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="kube-rbac-proxy"
Jul 08 16:52:53.716815 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.716790    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="kube-rbac-proxy"
Jul 08 16:52:53.716815 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.716797    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="873bbe5a-e8a1-420f-aa8f-84bcaf656116" containerName="kube-rbac-proxy"
Jul 08 16:52:53.716815 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.716805    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="873bbe5a-e8a1-420f-aa8f-84bcaf656116" containerName="kube-rbac-proxy"
Jul 08 16:52:53.716815 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.716812    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5271cd43-db59-417f-88ec-336654783e7b" containerName="kube-rbac-proxy"
Jul 08 16:52:53.717099 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.716834    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5271cd43-db59-417f-88ec-336654783e7b" containerName="kube-rbac-proxy"
Jul 08 16:52:53.717099 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.716847    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5271cd43-db59-417f-88ec-336654783e7b" containerName="kserve-container"
Jul 08 16:52:53.717099 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.716853    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5271cd43-db59-417f-88ec-336654783e7b" containerName="kserve-container"
Jul 08 16:52:53.717099 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.716862    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="kserve-container"
Jul 08 16:52:53.717099 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.716866    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="kserve-container"
Jul 08 16:52:53.717099 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.716876    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="storage-initializer"
Jul 08 16:52:53.717099 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.716881    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="storage-initializer"
Jul 08 16:52:53.717099 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.716891    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="873bbe5a-e8a1-420f-aa8f-84bcaf656116" containerName="kserve-container"
Jul 08 16:52:53.717099 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.716899    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="873bbe5a-e8a1-420f-aa8f-84bcaf656116" containerName="kserve-container"
Jul 08 16:52:53.717099 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.716964    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="kserve-container"
Jul 08 16:52:53.717099 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.716972    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="873bbe5a-e8a1-420f-aa8f-84bcaf656116" containerName="kserve-container"
Jul 08 16:52:53.717099 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.716981    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="873bbe5a-e8a1-420f-aa8f-84bcaf656116" containerName="kube-rbac-proxy"
Jul 08 16:52:53.717099 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.716990    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="kube-rbac-proxy"
Jul 08 16:52:53.717099 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.716998    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5271cd43-db59-417f-88ec-336654783e7b" containerName="kserve-container"
Jul 08 16:52:53.717099 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.717004    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="06bfd315-7e36-4511-b9d7-c2f488b06b8a" containerName="agent"
Jul 08 16:52:53.717099 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.717011    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5271cd43-db59-417f-88ec-336654783e7b" containerName="kube-rbac-proxy"
Jul 08 16:52:53.747837 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.747812    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl"]
Jul 08 16:52:53.747988 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.747936    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl"
Jul 08 16:52:53.751148 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.751116    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-predictor-serving-cert\""
Jul 08 16:52:53.751148 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.751117    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 08 16:52:53.751405 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.751207    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 08 16:52:53.751405 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.751211    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-jq2mg\""
Jul 08 16:52:53.751405 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.751120    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-kube-rbac-proxy-sar-config\""
Jul 08 16:52:53.751405 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.751289    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 08 16:52:53.751405 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.751225    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 08 16:52:53.786242 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.786216    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a343106f-89e5-4fe2-965c-b1f5055699c5-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-c4hzl\" (UID: \"a343106f-89e5-4fe2-965c-b1f5055699c5\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl"
Jul 08 16:52:53.786411 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.786254    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a343106f-89e5-4fe2-965c-b1f5055699c5-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-c4hzl\" (UID: \"a343106f-89e5-4fe2-965c-b1f5055699c5\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl"
Jul 08 16:52:53.786411 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.786333    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a343106f-89e5-4fe2-965c-b1f5055699c5-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-c4hzl\" (UID: \"a343106f-89e5-4fe2-965c-b1f5055699c5\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl"
Jul 08 16:52:53.786411 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.786387    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq6t8\" (UniqueName: \"kubernetes.io/projected/a343106f-89e5-4fe2-965c-b1f5055699c5-kube-api-access-mq6t8\") pod \"isvc-autogluon-v2-predictor-67b849d456-c4hzl\" (UID: \"a343106f-89e5-4fe2-965c-b1f5055699c5\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl"
Jul 08 16:52:53.786517 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.786432    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a343106f-89e5-4fe2-965c-b1f5055699c5-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-c4hzl\" (UID: \"a343106f-89e5-4fe2-965c-b1f5055699c5\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl"
Jul 08 16:52:53.887632 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.887595    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a343106f-89e5-4fe2-965c-b1f5055699c5-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-c4hzl\" (UID: \"a343106f-89e5-4fe2-965c-b1f5055699c5\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl"
Jul 08 16:52:53.887797 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.887655    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a343106f-89e5-4fe2-965c-b1f5055699c5-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-c4hzl\" (UID: \"a343106f-89e5-4fe2-965c-b1f5055699c5\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl"
Jul 08 16:52:53.887797 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.887691    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a343106f-89e5-4fe2-965c-b1f5055699c5-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-c4hzl\" (UID: \"a343106f-89e5-4fe2-965c-b1f5055699c5\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl"
Jul 08 16:52:53.887797 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.887740    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a343106f-89e5-4fe2-965c-b1f5055699c5-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-c4hzl\" (UID: \"a343106f-89e5-4fe2-965c-b1f5055699c5\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl"
Jul 08 16:52:53.887901 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.887792    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq6t8\" (UniqueName: \"kubernetes.io/projected/a343106f-89e5-4fe2-965c-b1f5055699c5-kube-api-access-mq6t8\") pod \"isvc-autogluon-v2-predictor-67b849d456-c4hzl\" (UID: \"a343106f-89e5-4fe2-965c-b1f5055699c5\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl"
Jul 08 16:52:53.888130 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.888111    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a343106f-89e5-4fe2-965c-b1f5055699c5-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-c4hzl\" (UID: \"a343106f-89e5-4fe2-965c-b1f5055699c5\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl"
Jul 08 16:52:53.888378 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.888359    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a343106f-89e5-4fe2-965c-b1f5055699c5-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-c4hzl\" (UID: \"a343106f-89e5-4fe2-965c-b1f5055699c5\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl"
Jul 08 16:52:53.888437 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.888421    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a343106f-89e5-4fe2-965c-b1f5055699c5-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-c4hzl\" (UID: \"a343106f-89e5-4fe2-965c-b1f5055699c5\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl"
Jul 08 16:52:53.890253 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.890237    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a343106f-89e5-4fe2-965c-b1f5055699c5-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-c4hzl\" (UID: \"a343106f-89e5-4fe2-965c-b1f5055699c5\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl"
Jul 08 16:52:53.896446 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:53.896419    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq6t8\" (UniqueName: \"kubernetes.io/projected/a343106f-89e5-4fe2-965c-b1f5055699c5-kube-api-access-mq6t8\") pod \"isvc-autogluon-v2-predictor-67b849d456-c4hzl\" (UID: \"a343106f-89e5-4fe2-965c-b1f5055699c5\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl"
Jul 08 16:52:54.058465 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:54.058374    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl"
Jul 08 16:52:54.213130 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:54.208226    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl"]
Jul 08 16:52:54.216105 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:52:54.216071    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda343106f_89e5_4fe2_965c_b1f5055699c5.slice/crio-5c8c4e5a101ed861ec9b2992ae4b2f202ff670b4cca0b92a452d1822a6ec82b7 WatchSource:0}: Error finding container 5c8c4e5a101ed861ec9b2992ae4b2f202ff670b4cca0b92a452d1822a6ec82b7: Status 404 returned error can't find the container with id 5c8c4e5a101ed861ec9b2992ae4b2f202ff670b4cca0b92a452d1822a6ec82b7
Jul 08 16:52:54.339140 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:54.339109    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl" event={"ID":"a343106f-89e5-4fe2-965c-b1f5055699c5","Type":"ContainerStarted","Data":"ffef2a2d994e9d1ec459a22485b241d8f0bbf8f243cf19e55a40ce6f6cf3861b"}
Jul 08 16:52:54.339276 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:52:54.339170    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl" event={"ID":"a343106f-89e5-4fe2-965c-b1f5055699c5","Type":"ContainerStarted","Data":"5c8c4e5a101ed861ec9b2992ae4b2f202ff670b4cca0b92a452d1822a6ec82b7"}
Jul 08 16:53:23.463199 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:53:23.463163    2582 generic.go:356] "Generic (PLEG): container finished" podID="a343106f-89e5-4fe2-965c-b1f5055699c5" containerID="ffef2a2d994e9d1ec459a22485b241d8f0bbf8f243cf19e55a40ce6f6cf3861b" exitCode=0
Jul 08 16:53:23.463632 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:53:23.463236    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl" event={"ID":"a343106f-89e5-4fe2-965c-b1f5055699c5","Type":"ContainerDied","Data":"ffef2a2d994e9d1ec459a22485b241d8f0bbf8f243cf19e55a40ce6f6cf3861b"}
Jul 08 16:53:24.471283 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:53:24.471243    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl" event={"ID":"a343106f-89e5-4fe2-965c-b1f5055699c5","Type":"ContainerStarted","Data":"171cc28b0f903c9200cb4da9cd1ddc9768aad9014b1a0193c2c2930a213f9891"}
Jul 08 16:53:24.471723 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:53:24.471342    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl" event={"ID":"a343106f-89e5-4fe2-965c-b1f5055699c5","Type":"ContainerStarted","Data":"196a9e62c57997b4aa77798da6d5f9c5bf45861b69da4d60feb34daaa5a786c8"}
Jul 08 16:53:24.471723 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:53:24.471580    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl"
Jul 08 16:53:24.490885 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:53:24.490822    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl" podStartSLOduration=2.464496709 podStartE2EDuration="31.490801101s" totalImagesPullingTime="0s" totalInitContainerRuntime="29.026304392s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:52:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:53:24.489813129 +0000 UTC m=+1123.830778206" watchObservedRunningTime="2026-07-08 16:53:24.490801101 +0000 UTC m=+1123.831766181"
Jul 08 16:53:25.476786 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:53:25.476756    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl"
Jul 08 16:53:31.487520 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:53:31.487489    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl"
Jul 08 16:54:41.155279 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:41.155247    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kvhpr_19df7751-4b62-4ae5-92a6-0bbda11fec85/ovn-acl-logging/0.log"
Jul 08 16:54:41.158739 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:41.158716    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kvhpr_19df7751-4b62-4ae5-92a6-0bbda11fec85/ovn-acl-logging/0.log"
Jul 08 16:54:53.217456 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:53.217426    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl"
Jul 08 16:54:53.880456 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:53.880426    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl"]
Jul 08 16:54:53.880691 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:53.880652    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl" podUID="a343106f-89e5-4fe2-965c-b1f5055699c5" containerName="kserve-container" containerID="cri-o://196a9e62c57997b4aa77798da6d5f9c5bf45861b69da4d60feb34daaa5a786c8" gracePeriod=30
Jul 08 16:54:53.880818 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:53.880699    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl" podUID="a343106f-89e5-4fe2-965c-b1f5055699c5" containerName="kube-rbac-proxy" containerID="cri-o://171cc28b0f903c9200cb4da9cd1ddc9768aad9014b1a0193c2c2930a213f9891" gracePeriod=30
Jul 08 16:54:54.832854 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.832830    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl"
Jul 08 16:54:54.875593 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.875512    2582 generic.go:356] "Generic (PLEG): container finished" podID="a343106f-89e5-4fe2-965c-b1f5055699c5" containerID="171cc28b0f903c9200cb4da9cd1ddc9768aad9014b1a0193c2c2930a213f9891" exitCode=2
Jul 08 16:54:54.875593 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.875534    2582 generic.go:356] "Generic (PLEG): container finished" podID="a343106f-89e5-4fe2-965c-b1f5055699c5" containerID="196a9e62c57997b4aa77798da6d5f9c5bf45861b69da4d60feb34daaa5a786c8" exitCode=0
Jul 08 16:54:54.875593 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.875568    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl"
Jul 08 16:54:54.875788 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.875571    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl" event={"ID":"a343106f-89e5-4fe2-965c-b1f5055699c5","Type":"ContainerDied","Data":"171cc28b0f903c9200cb4da9cd1ddc9768aad9014b1a0193c2c2930a213f9891"}
Jul 08 16:54:54.875788 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.875721    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl" event={"ID":"a343106f-89e5-4fe2-965c-b1f5055699c5","Type":"ContainerDied","Data":"196a9e62c57997b4aa77798da6d5f9c5bf45861b69da4d60feb34daaa5a786c8"}
Jul 08 16:54:54.875788 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.875747    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl" event={"ID":"a343106f-89e5-4fe2-965c-b1f5055699c5","Type":"ContainerDied","Data":"5c8c4e5a101ed861ec9b2992ae4b2f202ff670b4cca0b92a452d1822a6ec82b7"}
Jul 08 16:54:54.875788 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.875770    2582 scope.go:117] "RemoveContainer" containerID="171cc28b0f903c9200cb4da9cd1ddc9768aad9014b1a0193c2c2930a213f9891"
Jul 08 16:54:54.884209 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.884006    2582 scope.go:117] "RemoveContainer" containerID="196a9e62c57997b4aa77798da6d5f9c5bf45861b69da4d60feb34daaa5a786c8"
Jul 08 16:54:54.891530 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.891512    2582 scope.go:117] "RemoveContainer" containerID="ffef2a2d994e9d1ec459a22485b241d8f0bbf8f243cf19e55a40ce6f6cf3861b"
Jul 08 16:54:54.898778 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.898761    2582 scope.go:117] "RemoveContainer" containerID="171cc28b0f903c9200cb4da9cd1ddc9768aad9014b1a0193c2c2930a213f9891"
Jul 08 16:54:54.899032 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:54:54.899014    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"171cc28b0f903c9200cb4da9cd1ddc9768aad9014b1a0193c2c2930a213f9891\": container with ID starting with 171cc28b0f903c9200cb4da9cd1ddc9768aad9014b1a0193c2c2930a213f9891 not found: ID does not exist" containerID="171cc28b0f903c9200cb4da9cd1ddc9768aad9014b1a0193c2c2930a213f9891"
Jul 08 16:54:54.899082 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.899043    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"171cc28b0f903c9200cb4da9cd1ddc9768aad9014b1a0193c2c2930a213f9891"} err="failed to get container status \"171cc28b0f903c9200cb4da9cd1ddc9768aad9014b1a0193c2c2930a213f9891\": rpc error: code = NotFound desc = could not find container \"171cc28b0f903c9200cb4da9cd1ddc9768aad9014b1a0193c2c2930a213f9891\": container with ID starting with 171cc28b0f903c9200cb4da9cd1ddc9768aad9014b1a0193c2c2930a213f9891 not found: ID does not exist"
Jul 08 16:54:54.899082 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.899061    2582 scope.go:117] "RemoveContainer" containerID="196a9e62c57997b4aa77798da6d5f9c5bf45861b69da4d60feb34daaa5a786c8"
Jul 08 16:54:54.899301 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:54:54.899285    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"196a9e62c57997b4aa77798da6d5f9c5bf45861b69da4d60feb34daaa5a786c8\": container with ID starting with 196a9e62c57997b4aa77798da6d5f9c5bf45861b69da4d60feb34daaa5a786c8 not found: ID does not exist" containerID="196a9e62c57997b4aa77798da6d5f9c5bf45861b69da4d60feb34daaa5a786c8"
Jul 08 16:54:54.899378 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.899306    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"196a9e62c57997b4aa77798da6d5f9c5bf45861b69da4d60feb34daaa5a786c8"} err="failed to get container status \"196a9e62c57997b4aa77798da6d5f9c5bf45861b69da4d60feb34daaa5a786c8\": rpc error: code = NotFound desc = could not find container \"196a9e62c57997b4aa77798da6d5f9c5bf45861b69da4d60feb34daaa5a786c8\": container with ID starting with 196a9e62c57997b4aa77798da6d5f9c5bf45861b69da4d60feb34daaa5a786c8 not found: ID does not exist"
Jul 08 16:54:54.899378 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.899343    2582 scope.go:117] "RemoveContainer" containerID="ffef2a2d994e9d1ec459a22485b241d8f0bbf8f243cf19e55a40ce6f6cf3861b"
Jul 08 16:54:54.899575 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:54:54.899561    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffef2a2d994e9d1ec459a22485b241d8f0bbf8f243cf19e55a40ce6f6cf3861b\": container with ID starting with ffef2a2d994e9d1ec459a22485b241d8f0bbf8f243cf19e55a40ce6f6cf3861b not found: ID does not exist" containerID="ffef2a2d994e9d1ec459a22485b241d8f0bbf8f243cf19e55a40ce6f6cf3861b"
Jul 08 16:54:54.899618 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.899577    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffef2a2d994e9d1ec459a22485b241d8f0bbf8f243cf19e55a40ce6f6cf3861b"} err="failed to get container status \"ffef2a2d994e9d1ec459a22485b241d8f0bbf8f243cf19e55a40ce6f6cf3861b\": rpc error: code = NotFound desc = could not find container \"ffef2a2d994e9d1ec459a22485b241d8f0bbf8f243cf19e55a40ce6f6cf3861b\": container with ID starting with ffef2a2d994e9d1ec459a22485b241d8f0bbf8f243cf19e55a40ce6f6cf3861b not found: ID does not exist"
Jul 08 16:54:54.899618 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.899588    2582 scope.go:117] "RemoveContainer" containerID="171cc28b0f903c9200cb4da9cd1ddc9768aad9014b1a0193c2c2930a213f9891"
Jul 08 16:54:54.899776 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.899761    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"171cc28b0f903c9200cb4da9cd1ddc9768aad9014b1a0193c2c2930a213f9891"} err="failed to get container status \"171cc28b0f903c9200cb4da9cd1ddc9768aad9014b1a0193c2c2930a213f9891\": rpc error: code = NotFound desc = could not find container \"171cc28b0f903c9200cb4da9cd1ddc9768aad9014b1a0193c2c2930a213f9891\": container with ID starting with 171cc28b0f903c9200cb4da9cd1ddc9768aad9014b1a0193c2c2930a213f9891 not found: ID does not exist"
Jul 08 16:54:54.899824 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.899776    2582 scope.go:117] "RemoveContainer" containerID="196a9e62c57997b4aa77798da6d5f9c5bf45861b69da4d60feb34daaa5a786c8"
Jul 08 16:54:54.899939 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.899924    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"196a9e62c57997b4aa77798da6d5f9c5bf45861b69da4d60feb34daaa5a786c8"} err="failed to get container status \"196a9e62c57997b4aa77798da6d5f9c5bf45861b69da4d60feb34daaa5a786c8\": rpc error: code = NotFound desc = could not find container \"196a9e62c57997b4aa77798da6d5f9c5bf45861b69da4d60feb34daaa5a786c8\": container with ID starting with 196a9e62c57997b4aa77798da6d5f9c5bf45861b69da4d60feb34daaa5a786c8 not found: ID does not exist"
Jul 08 16:54:54.899982 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.899938    2582 scope.go:117] "RemoveContainer" containerID="ffef2a2d994e9d1ec459a22485b241d8f0bbf8f243cf19e55a40ce6f6cf3861b"
Jul 08 16:54:54.900139 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.900120    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffef2a2d994e9d1ec459a22485b241d8f0bbf8f243cf19e55a40ce6f6cf3861b"} err="failed to get container status \"ffef2a2d994e9d1ec459a22485b241d8f0bbf8f243cf19e55a40ce6f6cf3861b\": rpc error: code = NotFound desc = could not find container \"ffef2a2d994e9d1ec459a22485b241d8f0bbf8f243cf19e55a40ce6f6cf3861b\": container with ID starting with ffef2a2d994e9d1ec459a22485b241d8f0bbf8f243cf19e55a40ce6f6cf3861b not found: ID does not exist"
Jul 08 16:54:54.911564 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.911543    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mq6t8\" (UniqueName: \"kubernetes.io/projected/a343106f-89e5-4fe2-965c-b1f5055699c5-kube-api-access-mq6t8\") pod \"a343106f-89e5-4fe2-965c-b1f5055699c5\" (UID: \"a343106f-89e5-4fe2-965c-b1f5055699c5\") "
Jul 08 16:54:54.911671 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.911586    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a343106f-89e5-4fe2-965c-b1f5055699c5-proxy-tls\") pod \"a343106f-89e5-4fe2-965c-b1f5055699c5\" (UID: \"a343106f-89e5-4fe2-965c-b1f5055699c5\") "
Jul 08 16:54:54.911671 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.911634    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a343106f-89e5-4fe2-965c-b1f5055699c5-kserve-provision-location\") pod \"a343106f-89e5-4fe2-965c-b1f5055699c5\" (UID: \"a343106f-89e5-4fe2-965c-b1f5055699c5\") "
Jul 08 16:54:54.911774 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.911693    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a343106f-89e5-4fe2-965c-b1f5055699c5-cabundle-cert\") pod \"a343106f-89e5-4fe2-965c-b1f5055699c5\" (UID: \"a343106f-89e5-4fe2-965c-b1f5055699c5\") "
Jul 08 16:54:54.911774 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.911727    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a343106f-89e5-4fe2-965c-b1f5055699c5-kube-rbac-proxy-sar-config\") pod \"a343106f-89e5-4fe2-965c-b1f5055699c5\" (UID: \"a343106f-89e5-4fe2-965c-b1f5055699c5\") "
Jul 08 16:54:54.912229 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.912104    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a343106f-89e5-4fe2-965c-b1f5055699c5-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "a343106f-89e5-4fe2-965c-b1f5055699c5" (UID: "a343106f-89e5-4fe2-965c-b1f5055699c5"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:54:54.912229 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.912142    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a343106f-89e5-4fe2-965c-b1f5055699c5-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "a343106f-89e5-4fe2-965c-b1f5055699c5" (UID: "a343106f-89e5-4fe2-965c-b1f5055699c5"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:54:54.913834 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.913812    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a343106f-89e5-4fe2-965c-b1f5055699c5-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "a343106f-89e5-4fe2-965c-b1f5055699c5" (UID: "a343106f-89e5-4fe2-965c-b1f5055699c5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 16:54:54.913908 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.913853    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a343106f-89e5-4fe2-965c-b1f5055699c5-kube-api-access-mq6t8" (OuterVolumeSpecName: "kube-api-access-mq6t8") pod "a343106f-89e5-4fe2-965c-b1f5055699c5" (UID: "a343106f-89e5-4fe2-965c-b1f5055699c5"). InnerVolumeSpecName "kube-api-access-mq6t8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 16:54:54.916692 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:54.916669    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a343106f-89e5-4fe2-965c-b1f5055699c5-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "a343106f-89e5-4fe2-965c-b1f5055699c5" (UID: "a343106f-89e5-4fe2-965c-b1f5055699c5"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 16:54:55.012629 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:55.012593    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mq6t8\" (UniqueName: \"kubernetes.io/projected/a343106f-89e5-4fe2-965c-b1f5055699c5-kube-api-access-mq6t8\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:54:55.012796 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:55.012645    2582 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a343106f-89e5-4fe2-965c-b1f5055699c5-proxy-tls\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:54:55.012796 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:55.012655    2582 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a343106f-89e5-4fe2-965c-b1f5055699c5-kserve-provision-location\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:54:55.012796 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:55.012664    2582 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a343106f-89e5-4fe2-965c-b1f5055699c5-cabundle-cert\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:54:55.012796 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:55.012672    2582 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a343106f-89e5-4fe2-965c-b1f5055699c5-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:54:55.199609 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:55.199572    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl"]
Jul 08 16:54:55.204665 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:55.204640    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-c4hzl"]
Jul 08 16:54:55.216516 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:55.216491    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a343106f-89e5-4fe2-965c-b1f5055699c5" path="/var/lib/kubelet/pods/a343106f-89e5-4fe2-965c-b1f5055699c5/volumes"
Jul 08 16:54:59.064136 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.064100    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5"]
Jul 08 16:54:59.064733 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.064632    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a343106f-89e5-4fe2-965c-b1f5055699c5" containerName="storage-initializer"
Jul 08 16:54:59.064733 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.064653    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a343106f-89e5-4fe2-965c-b1f5055699c5" containerName="storage-initializer"
Jul 08 16:54:59.064733 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.064672    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a343106f-89e5-4fe2-965c-b1f5055699c5" containerName="kube-rbac-proxy"
Jul 08 16:54:59.064733 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.064681    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a343106f-89e5-4fe2-965c-b1f5055699c5" containerName="kube-rbac-proxy"
Jul 08 16:54:59.064733 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.064706    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a343106f-89e5-4fe2-965c-b1f5055699c5" containerName="kserve-container"
Jul 08 16:54:59.064733 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.064714    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a343106f-89e5-4fe2-965c-b1f5055699c5" containerName="kserve-container"
Jul 08 16:54:59.065055 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.064790    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a343106f-89e5-4fe2-965c-b1f5055699c5" containerName="kserve-container"
Jul 08 16:54:59.065055 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.064805    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a343106f-89e5-4fe2-965c-b1f5055699c5" containerName="kube-rbac-proxy"
Jul 08 16:54:59.068650 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.068627    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5"
Jul 08 16:54:59.073130 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.073108    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 08 16:54:59.073230 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.073104    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-variants-kube-rbac-proxy-sar-config\""
Jul 08 16:54:59.073230 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.073138    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 08 16:54:59.073453 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.073440    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 08 16:54:59.074005 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.073987    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 08 16:54:59.074005 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.073997    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-variants-predictor-serving-cert\""
Jul 08 16:54:59.074144 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.074054    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-jq2mg\""
Jul 08 16:54:59.084224 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.084202    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5"]
Jul 08 16:54:59.151142 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.151100    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5\" (UID: \"6bf36aa4-8d38-4847-ae5c-37a22b024fc7\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5"
Jul 08 16:54:59.151352 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.151227    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5\" (UID: \"6bf36aa4-8d38-4847-ae5c-37a22b024fc7\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5"
Jul 08 16:54:59.151352 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.151261    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5\" (UID: \"6bf36aa4-8d38-4847-ae5c-37a22b024fc7\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5"
Jul 08 16:54:59.151352 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.151283    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5\" (UID: \"6bf36aa4-8d38-4847-ae5c-37a22b024fc7\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5"
Jul 08 16:54:59.151352 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.151337    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwrzj\" (UniqueName: \"kubernetes.io/projected/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-kube-api-access-vwrzj\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5\" (UID: \"6bf36aa4-8d38-4847-ae5c-37a22b024fc7\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5"
Jul 08 16:54:59.252261 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.252222    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5\" (UID: \"6bf36aa4-8d38-4847-ae5c-37a22b024fc7\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5"
Jul 08 16:54:59.252484 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.252270    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5\" (UID: \"6bf36aa4-8d38-4847-ae5c-37a22b024fc7\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5"
Jul 08 16:54:59.252484 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.252309    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5\" (UID: \"6bf36aa4-8d38-4847-ae5c-37a22b024fc7\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5"
Jul 08 16:54:59.252484 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.252376    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwrzj\" (UniqueName: \"kubernetes.io/projected/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-kube-api-access-vwrzj\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5\" (UID: \"6bf36aa4-8d38-4847-ae5c-37a22b024fc7\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5"
Jul 08 16:54:59.252484 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.252427    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5\" (UID: \"6bf36aa4-8d38-4847-ae5c-37a22b024fc7\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5"
Jul 08 16:54:59.252807 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.252781    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5\" (UID: \"6bf36aa4-8d38-4847-ae5c-37a22b024fc7\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5"
Jul 08 16:54:59.253005 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.252987    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5\" (UID: \"6bf36aa4-8d38-4847-ae5c-37a22b024fc7\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5"
Jul 08 16:54:59.253064 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.253017    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5\" (UID: \"6bf36aa4-8d38-4847-ae5c-37a22b024fc7\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5"
Jul 08 16:54:59.254943 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.254921    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5\" (UID: \"6bf36aa4-8d38-4847-ae5c-37a22b024fc7\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5"
Jul 08 16:54:59.262260 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.262239    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwrzj\" (UniqueName: \"kubernetes.io/projected/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-kube-api-access-vwrzj\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5\" (UID: \"6bf36aa4-8d38-4847-ae5c-37a22b024fc7\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5"
Jul 08 16:54:59.378958 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.378872    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5"
Jul 08 16:54:59.515147 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.515123    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5"]
Jul 08 16:54:59.517861 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:54:59.517833    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bf36aa4_8d38_4847_ae5c_37a22b024fc7.slice/crio-abb493f12d16e32f7fff62f242ac35f05b80975264b53d3ba73e5b4ccb47a88a WatchSource:0}: Error finding container abb493f12d16e32f7fff62f242ac35f05b80975264b53d3ba73e5b4ccb47a88a: Status 404 returned error can't find the container with id abb493f12d16e32f7fff62f242ac35f05b80975264b53d3ba73e5b4ccb47a88a
Jul 08 16:54:59.900373 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.900308    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5" event={"ID":"6bf36aa4-8d38-4847-ae5c-37a22b024fc7","Type":"ContainerStarted","Data":"e51bc6fcc4dd38834b8e8d4f856d326cc105878bf2b4a98988198020604e9ff8"}
Jul 08 16:54:59.900570 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:54:59.900445    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5" event={"ID":"6bf36aa4-8d38-4847-ae5c-37a22b024fc7","Type":"ContainerStarted","Data":"abb493f12d16e32f7fff62f242ac35f05b80975264b53d3ba73e5b4ccb47a88a"}
Jul 08 16:55:04.923625 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:55:04.923592    2582 generic.go:356] "Generic (PLEG): container finished" podID="6bf36aa4-8d38-4847-ae5c-37a22b024fc7" containerID="e51bc6fcc4dd38834b8e8d4f856d326cc105878bf2b4a98988198020604e9ff8" exitCode=0
Jul 08 16:55:04.924062 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:55:04.923639    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5" event={"ID":"6bf36aa4-8d38-4847-ae5c-37a22b024fc7","Type":"ContainerDied","Data":"e51bc6fcc4dd38834b8e8d4f856d326cc105878bf2b4a98988198020604e9ff8"}
Jul 08 16:55:05.930097 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:55:05.930050    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5" event={"ID":"6bf36aa4-8d38-4847-ae5c-37a22b024fc7","Type":"ContainerStarted","Data":"82d933aa8d910f2312f5abd30ac213c1029a35b8bf58e11a0c77616bf4bcc283"}
Jul 08 16:55:05.930479 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:55:05.930171    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5" event={"ID":"6bf36aa4-8d38-4847-ae5c-37a22b024fc7","Type":"ContainerStarted","Data":"d0fe0f1e4db33e5b1853fb1e5387473752a1a7b326a3e821552cfe6cc34c5cba"}
Jul 08 16:55:05.951561 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:55:05.951508    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5" podStartSLOduration=2.295743368 podStartE2EDuration="6.951490783s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.655747415s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:54:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:55:05.949485036 +0000 UTC m=+1225.290450115" watchObservedRunningTime="2026-07-08 16:55:05.951490783 +0000 UTC m=+1225.292455908"
Jul 08 16:55:10.931099 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:55:10.931057    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5"
Jul 08 16:55:10.931566 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:55:10.931495    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5"
Jul 08 16:55:10.936053 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:55:10.936025    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5"
Jul 08 16:56:37.217207 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:37.217181    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5"
Jul 08 16:56:39.186060 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:39.186023    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5"]
Jul 08 16:56:39.186548 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:39.186232    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5" podUID="6bf36aa4-8d38-4847-ae5c-37a22b024fc7" containerName="kserve-container" containerID="cri-o://d0fe0f1e4db33e5b1853fb1e5387473752a1a7b326a3e821552cfe6cc34c5cba" gracePeriod=30
Jul 08 16:56:39.186548 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:39.186304    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5" podUID="6bf36aa4-8d38-4847-ae5c-37a22b024fc7" containerName="kube-rbac-proxy" containerID="cri-o://82d933aa8d910f2312f5abd30ac213c1029a35b8bf58e11a0c77616bf4bcc283" gracePeriod=30
Jul 08 16:56:39.338306 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:39.338274    2582 generic.go:356] "Generic (PLEG): container finished" podID="6bf36aa4-8d38-4847-ae5c-37a22b024fc7" containerID="82d933aa8d910f2312f5abd30ac213c1029a35b8bf58e11a0c77616bf4bcc283" exitCode=2
Jul 08 16:56:39.338476 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:39.338359    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5" event={"ID":"6bf36aa4-8d38-4847-ae5c-37a22b024fc7","Type":"ContainerDied","Data":"82d933aa8d910f2312f5abd30ac213c1029a35b8bf58e11a0c77616bf4bcc283"}
Jul 08 16:56:40.030578 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.030552    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5"
Jul 08 16:56:40.071791 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.071765    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwrzj\" (UniqueName: \"kubernetes.io/projected/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-kube-api-access-vwrzj\") pod \"6bf36aa4-8d38-4847-ae5c-37a22b024fc7\" (UID: \"6bf36aa4-8d38-4847-ae5c-37a22b024fc7\") "
Jul 08 16:56:40.071952 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.071842    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-kube-rbac-proxy-sar-config\") pod \"6bf36aa4-8d38-4847-ae5c-37a22b024fc7\" (UID: \"6bf36aa4-8d38-4847-ae5c-37a22b024fc7\") "
Jul 08 16:56:40.071952 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.071883    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-proxy-tls\") pod \"6bf36aa4-8d38-4847-ae5c-37a22b024fc7\" (UID: \"6bf36aa4-8d38-4847-ae5c-37a22b024fc7\") "
Jul 08 16:56:40.072034 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.071951    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-kserve-provision-location\") pod \"6bf36aa4-8d38-4847-ae5c-37a22b024fc7\" (UID: \"6bf36aa4-8d38-4847-ae5c-37a22b024fc7\") "
Jul 08 16:56:40.072034 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.072016    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-cabundle-cert\") pod \"6bf36aa4-8d38-4847-ae5c-37a22b024fc7\" (UID: \"6bf36aa4-8d38-4847-ae5c-37a22b024fc7\") "
Jul 08 16:56:40.072282 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.072249    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "6bf36aa4-8d38-4847-ae5c-37a22b024fc7" (UID: "6bf36aa4-8d38-4847-ae5c-37a22b024fc7"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:56:40.072444 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.072388    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "6bf36aa4-8d38-4847-ae5c-37a22b024fc7" (UID: "6bf36aa4-8d38-4847-ae5c-37a22b024fc7"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:56:40.074114 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.074094    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "6bf36aa4-8d38-4847-ae5c-37a22b024fc7" (UID: "6bf36aa4-8d38-4847-ae5c-37a22b024fc7"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 16:56:40.074408 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.074391    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-kube-api-access-vwrzj" (OuterVolumeSpecName: "kube-api-access-vwrzj") pod "6bf36aa4-8d38-4847-ae5c-37a22b024fc7" (UID: "6bf36aa4-8d38-4847-ae5c-37a22b024fc7"). InnerVolumeSpecName "kube-api-access-vwrzj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 16:56:40.077335 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.077298    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "6bf36aa4-8d38-4847-ae5c-37a22b024fc7" (UID: "6bf36aa4-8d38-4847-ae5c-37a22b024fc7"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 16:56:40.172920 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.172887    2582 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:56:40.172920 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.172919    2582 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-proxy-tls\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:56:40.172920 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.172928    2582 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-kserve-provision-location\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:56:40.173158 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.172937    2582 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-cabundle-cert\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:56:40.173158 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.172948    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwrzj\" (UniqueName: \"kubernetes.io/projected/6bf36aa4-8d38-4847-ae5c-37a22b024fc7-kube-api-access-vwrzj\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:56:40.345053 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.345015    2582 generic.go:356] "Generic (PLEG): container finished" podID="6bf36aa4-8d38-4847-ae5c-37a22b024fc7" containerID="d0fe0f1e4db33e5b1853fb1e5387473752a1a7b326a3e821552cfe6cc34c5cba" exitCode=0
Jul 08 16:56:40.345053 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.345058    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5" event={"ID":"6bf36aa4-8d38-4847-ae5c-37a22b024fc7","Type":"ContainerDied","Data":"d0fe0f1e4db33e5b1853fb1e5387473752a1a7b326a3e821552cfe6cc34c5cba"}
Jul 08 16:56:40.345539 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.345073    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5"
Jul 08 16:56:40.345539 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.345107    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5" event={"ID":"6bf36aa4-8d38-4847-ae5c-37a22b024fc7","Type":"ContainerDied","Data":"abb493f12d16e32f7fff62f242ac35f05b80975264b53d3ba73e5b4ccb47a88a"}
Jul 08 16:56:40.345539 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.345139    2582 scope.go:117] "RemoveContainer" containerID="82d933aa8d910f2312f5abd30ac213c1029a35b8bf58e11a0c77616bf4bcc283"
Jul 08 16:56:40.354536 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.354519    2582 scope.go:117] "RemoveContainer" containerID="d0fe0f1e4db33e5b1853fb1e5387473752a1a7b326a3e821552cfe6cc34c5cba"
Jul 08 16:56:40.362387 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.362367    2582 scope.go:117] "RemoveContainer" containerID="e51bc6fcc4dd38834b8e8d4f856d326cc105878bf2b4a98988198020604e9ff8"
Jul 08 16:56:40.367009 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.366984    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5"]
Jul 08 16:56:40.370929 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.370901    2582 scope.go:117] "RemoveContainer" containerID="82d933aa8d910f2312f5abd30ac213c1029a35b8bf58e11a0c77616bf4bcc283"
Jul 08 16:56:40.371283 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:56:40.371260    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82d933aa8d910f2312f5abd30ac213c1029a35b8bf58e11a0c77616bf4bcc283\": container with ID starting with 82d933aa8d910f2312f5abd30ac213c1029a35b8bf58e11a0c77616bf4bcc283 not found: ID does not exist" containerID="82d933aa8d910f2312f5abd30ac213c1029a35b8bf58e11a0c77616bf4bcc283"
Jul 08 16:56:40.371399 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.371296    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82d933aa8d910f2312f5abd30ac213c1029a35b8bf58e11a0c77616bf4bcc283"} err="failed to get container status \"82d933aa8d910f2312f5abd30ac213c1029a35b8bf58e11a0c77616bf4bcc283\": rpc error: code = NotFound desc = could not find container \"82d933aa8d910f2312f5abd30ac213c1029a35b8bf58e11a0c77616bf4bcc283\": container with ID starting with 82d933aa8d910f2312f5abd30ac213c1029a35b8bf58e11a0c77616bf4bcc283 not found: ID does not exist"
Jul 08 16:56:40.371399 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.371345    2582 scope.go:117] "RemoveContainer" containerID="d0fe0f1e4db33e5b1853fb1e5387473752a1a7b326a3e821552cfe6cc34c5cba"
Jul 08 16:56:40.371617 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:56:40.371599    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0fe0f1e4db33e5b1853fb1e5387473752a1a7b326a3e821552cfe6cc34c5cba\": container with ID starting with d0fe0f1e4db33e5b1853fb1e5387473752a1a7b326a3e821552cfe6cc34c5cba not found: ID does not exist" containerID="d0fe0f1e4db33e5b1853fb1e5387473752a1a7b326a3e821552cfe6cc34c5cba"
Jul 08 16:56:40.371662 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.371623    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0fe0f1e4db33e5b1853fb1e5387473752a1a7b326a3e821552cfe6cc34c5cba"} err="failed to get container status \"d0fe0f1e4db33e5b1853fb1e5387473752a1a7b326a3e821552cfe6cc34c5cba\": rpc error: code = NotFound desc = could not find container \"d0fe0f1e4db33e5b1853fb1e5387473752a1a7b326a3e821552cfe6cc34c5cba\": container with ID starting with d0fe0f1e4db33e5b1853fb1e5387473752a1a7b326a3e821552cfe6cc34c5cba not found: ID does not exist"
Jul 08 16:56:40.371662 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.371638    2582 scope.go:117] "RemoveContainer" containerID="e51bc6fcc4dd38834b8e8d4f856d326cc105878bf2b4a98988198020604e9ff8"
Jul 08 16:56:40.371905 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:56:40.371887    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e51bc6fcc4dd38834b8e8d4f856d326cc105878bf2b4a98988198020604e9ff8\": container with ID starting with e51bc6fcc4dd38834b8e8d4f856d326cc105878bf2b4a98988198020604e9ff8 not found: ID does not exist" containerID="e51bc6fcc4dd38834b8e8d4f856d326cc105878bf2b4a98988198020604e9ff8"
Jul 08 16:56:40.371945 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.371910    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e51bc6fcc4dd38834b8e8d4f856d326cc105878bf2b4a98988198020604e9ff8"} err="failed to get container status \"e51bc6fcc4dd38834b8e8d4f856d326cc105878bf2b4a98988198020604e9ff8\": rpc error: code = NotFound desc = could not find container \"e51bc6fcc4dd38834b8e8d4f856d326cc105878bf2b4a98988198020604e9ff8\": container with ID starting with e51bc6fcc4dd38834b8e8d4f856d326cc105878bf2b4a98988198020604e9ff8 not found: ID does not exist"
Jul 08 16:56:40.371983 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:40.371962    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-wz2v5"]
Jul 08 16:56:41.217878 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:41.217849    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bf36aa4-8d38-4847-ae5c-37a22b024fc7" path="/var/lib/kubelet/pods/6bf36aa4-8d38-4847-ae5c-37a22b024fc7/volumes"
Jul 08 16:56:44.393426 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.393389    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"]
Jul 08 16:56:44.393823 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.393718    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bf36aa4-8d38-4847-ae5c-37a22b024fc7" containerName="kserve-container"
Jul 08 16:56:44.393823 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.393728    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bf36aa4-8d38-4847-ae5c-37a22b024fc7" containerName="kserve-container"
Jul 08 16:56:44.393823 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.393748    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bf36aa4-8d38-4847-ae5c-37a22b024fc7" containerName="storage-initializer"
Jul 08 16:56:44.393823 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.393754    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bf36aa4-8d38-4847-ae5c-37a22b024fc7" containerName="storage-initializer"
Jul 08 16:56:44.393823 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.393761    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bf36aa4-8d38-4847-ae5c-37a22b024fc7" containerName="kube-rbac-proxy"
Jul 08 16:56:44.393823 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.393767    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bf36aa4-8d38-4847-ae5c-37a22b024fc7" containerName="kube-rbac-proxy"
Jul 08 16:56:44.394033 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.393833    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bf36aa4-8d38-4847-ae5c-37a22b024fc7" containerName="kserve-container"
Jul 08 16:56:44.394033 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.393843    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bf36aa4-8d38-4847-ae5c-37a22b024fc7" containerName="kube-rbac-proxy"
Jul 08 16:56:44.398839 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.398820    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"
Jul 08 16:56:44.401411 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.401385    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 08 16:56:44.401567 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.401541    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-noslash-predictor-serving-cert\""
Jul 08 16:56:44.401567 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.401558    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-noslash-kube-rbac-proxy-sar-config\""
Jul 08 16:56:44.401709 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.401558    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 08 16:56:44.401709 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.401558    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 08 16:56:44.401850 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.401830    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-jq2mg\""
Jul 08 16:56:44.402262 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.402247    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 08 16:56:44.407304 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.407282    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"]
Jul 08 16:56:44.510145 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.510107    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgmcs\" (UniqueName: \"kubernetes.io/projected/733ffc72-f62e-4330-afdc-bc7b2464dd39-kube-api-access-tgmcs\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp\" (UID: \"733ffc72-f62e-4330-afdc-bc7b2464dd39\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"
Jul 08 16:56:44.510145 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.510153    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/733ffc72-f62e-4330-afdc-bc7b2464dd39-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp\" (UID: \"733ffc72-f62e-4330-afdc-bc7b2464dd39\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"
Jul 08 16:56:44.510398 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.510236    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/733ffc72-f62e-4330-afdc-bc7b2464dd39-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp\" (UID: \"733ffc72-f62e-4330-afdc-bc7b2464dd39\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"
Jul 08 16:56:44.510398 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.510271    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/733ffc72-f62e-4330-afdc-bc7b2464dd39-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp\" (UID: \"733ffc72-f62e-4330-afdc-bc7b2464dd39\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"
Jul 08 16:56:44.510398 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.510302    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/733ffc72-f62e-4330-afdc-bc7b2464dd39-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp\" (UID: \"733ffc72-f62e-4330-afdc-bc7b2464dd39\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"
Jul 08 16:56:44.611156 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.611120    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/733ffc72-f62e-4330-afdc-bc7b2464dd39-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp\" (UID: \"733ffc72-f62e-4330-afdc-bc7b2464dd39\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"
Jul 08 16:56:44.611156 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.611164    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/733ffc72-f62e-4330-afdc-bc7b2464dd39-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp\" (UID: \"733ffc72-f62e-4330-afdc-bc7b2464dd39\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"
Jul 08 16:56:44.611566 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.611196    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/733ffc72-f62e-4330-afdc-bc7b2464dd39-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp\" (UID: \"733ffc72-f62e-4330-afdc-bc7b2464dd39\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"
Jul 08 16:56:44.611566 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.611263    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgmcs\" (UniqueName: \"kubernetes.io/projected/733ffc72-f62e-4330-afdc-bc7b2464dd39-kube-api-access-tgmcs\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp\" (UID: \"733ffc72-f62e-4330-afdc-bc7b2464dd39\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"
Jul 08 16:56:44.611566 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:56:44.611289    2582 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-serving-cert: secret "isvc-autogluon-v2-noslash-predictor-serving-cert" not found
Jul 08 16:56:44.611566 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.611300    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/733ffc72-f62e-4330-afdc-bc7b2464dd39-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp\" (UID: \"733ffc72-f62e-4330-afdc-bc7b2464dd39\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"
Jul 08 16:56:44.611566 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:56:44.611396    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/733ffc72-f62e-4330-afdc-bc7b2464dd39-proxy-tls podName:733ffc72-f62e-4330-afdc-bc7b2464dd39 nodeName:}" failed. No retries permitted until 2026-07-08 16:56:45.111374141 +0000 UTC m=+1324.452339205 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/733ffc72-f62e-4330-afdc-bc7b2464dd39-proxy-tls") pod "isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp" (UID: "733ffc72-f62e-4330-afdc-bc7b2464dd39") : secret "isvc-autogluon-v2-noslash-predictor-serving-cert" not found
Jul 08 16:56:44.611807 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.611671    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/733ffc72-f62e-4330-afdc-bc7b2464dd39-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp\" (UID: \"733ffc72-f62e-4330-afdc-bc7b2464dd39\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"
Jul 08 16:56:44.611967 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.611944    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/733ffc72-f62e-4330-afdc-bc7b2464dd39-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp\" (UID: \"733ffc72-f62e-4330-afdc-bc7b2464dd39\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"
Jul 08 16:56:44.612004 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.611944    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/733ffc72-f62e-4330-afdc-bc7b2464dd39-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp\" (UID: \"733ffc72-f62e-4330-afdc-bc7b2464dd39\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"
Jul 08 16:56:44.620605 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:44.620582    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgmcs\" (UniqueName: \"kubernetes.io/projected/733ffc72-f62e-4330-afdc-bc7b2464dd39-kube-api-access-tgmcs\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp\" (UID: \"733ffc72-f62e-4330-afdc-bc7b2464dd39\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"
Jul 08 16:56:45.116456 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:45.116412    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/733ffc72-f62e-4330-afdc-bc7b2464dd39-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp\" (UID: \"733ffc72-f62e-4330-afdc-bc7b2464dd39\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"
Jul 08 16:56:45.119076 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:45.119040    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/733ffc72-f62e-4330-afdc-bc7b2464dd39-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp\" (UID: \"733ffc72-f62e-4330-afdc-bc7b2464dd39\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"
Jul 08 16:56:45.309886 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:45.309851    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"
Jul 08 16:56:45.469050 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:45.469024    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"]
Jul 08 16:56:45.471538 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:56:45.471510    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod733ffc72_f62e_4330_afdc_bc7b2464dd39.slice/crio-e645ba9be758540d34cb1abbdc26ad48aa285953e312558c089f210b2bbf8d48 WatchSource:0}: Error finding container e645ba9be758540d34cb1abbdc26ad48aa285953e312558c089f210b2bbf8d48: Status 404 returned error can't find the container with id e645ba9be758540d34cb1abbdc26ad48aa285953e312558c089f210b2bbf8d48
Jul 08 16:56:46.375732 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:46.375691    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp" event={"ID":"733ffc72-f62e-4330-afdc-bc7b2464dd39","Type":"ContainerStarted","Data":"f250f01e44eaa8a1fb9ccbabe343e7a701528b0f06e2f15dfda2d18be990ccc6"}
Jul 08 16:56:46.375939 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:56:46.375754    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp" event={"ID":"733ffc72-f62e-4330-afdc-bc7b2464dd39","Type":"ContainerStarted","Data":"e645ba9be758540d34cb1abbdc26ad48aa285953e312558c089f210b2bbf8d48"}
Jul 08 16:57:07.466257 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:57:07.466226    2582 generic.go:356] "Generic (PLEG): container finished" podID="733ffc72-f62e-4330-afdc-bc7b2464dd39" containerID="f250f01e44eaa8a1fb9ccbabe343e7a701528b0f06e2f15dfda2d18be990ccc6" exitCode=0
Jul 08 16:57:07.466791 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:57:07.466344    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp" event={"ID":"733ffc72-f62e-4330-afdc-bc7b2464dd39","Type":"ContainerDied","Data":"f250f01e44eaa8a1fb9ccbabe343e7a701528b0f06e2f15dfda2d18be990ccc6"}
Jul 08 16:57:08.473706 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:57:08.473675    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp" event={"ID":"733ffc72-f62e-4330-afdc-bc7b2464dd39","Type":"ContainerStarted","Data":"5d8fe0897afb85abdfa8171d1b9eba01b850995be8a9f9a7100f0d8fd2f4982b"}
Jul 08 16:57:08.474199 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:57:08.473738    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp" event={"ID":"733ffc72-f62e-4330-afdc-bc7b2464dd39","Type":"ContainerStarted","Data":"6aa0c4f8b9b6e743ca16cd302dc51db711bc5326853fe38281ece0a2bc410510"}
Jul 08 16:57:08.474199 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:57:08.474045    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"
Jul 08 16:57:08.474199 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:57:08.474131    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"
Jul 08 16:57:08.493895 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:57:08.493844    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp" podStartSLOduration=3.219222737 podStartE2EDuration="24.493829147s" totalImagesPullingTime="0s" totalInitContainerRuntime="21.27460641s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:56:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:57:08.492657121 +0000 UTC m=+1347.833622200" watchObservedRunningTime="2026-07-08 16:57:08.493829147 +0000 UTC m=+1347.834794225"
Jul 08 16:57:14.485066 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:57:14.485035    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"
Jul 08 16:58:44.216033 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:44.215946    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"
Jul 08 16:58:54.584234 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:54.584145    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"]
Jul 08 16:58:54.584672 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:54.584396    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp" podUID="733ffc72-f62e-4330-afdc-bc7b2464dd39" containerName="kserve-container" containerID="cri-o://6aa0c4f8b9b6e743ca16cd302dc51db711bc5326853fe38281ece0a2bc410510" gracePeriod=30
Jul 08 16:58:54.584672 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:54.584428    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp" podUID="733ffc72-f62e-4330-afdc-bc7b2464dd39" containerName="kube-rbac-proxy" containerID="cri-o://5d8fe0897afb85abdfa8171d1b9eba01b850995be8a9f9a7100f0d8fd2f4982b" gracePeriod=30
Jul 08 16:58:54.942157 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:54.942113    2582 generic.go:356] "Generic (PLEG): container finished" podID="733ffc72-f62e-4330-afdc-bc7b2464dd39" containerID="5d8fe0897afb85abdfa8171d1b9eba01b850995be8a9f9a7100f0d8fd2f4982b" exitCode=2
Jul 08 16:58:54.942360 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:54.942185    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp" event={"ID":"733ffc72-f62e-4330-afdc-bc7b2464dd39","Type":"ContainerDied","Data":"5d8fe0897afb85abdfa8171d1b9eba01b850995be8a9f9a7100f0d8fd2f4982b"}
Jul 08 16:58:55.433626 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.433602    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"
Jul 08 16:58:55.458045 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.458016    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/733ffc72-f62e-4330-afdc-bc7b2464dd39-kserve-provision-location\") pod \"733ffc72-f62e-4330-afdc-bc7b2464dd39\" (UID: \"733ffc72-f62e-4330-afdc-bc7b2464dd39\") "
Jul 08 16:58:55.458045 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.458049    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/733ffc72-f62e-4330-afdc-bc7b2464dd39-cabundle-cert\") pod \"733ffc72-f62e-4330-afdc-bc7b2464dd39\" (UID: \"733ffc72-f62e-4330-afdc-bc7b2464dd39\") "
Jul 08 16:58:55.458264 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.458078    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/733ffc72-f62e-4330-afdc-bc7b2464dd39-proxy-tls\") pod \"733ffc72-f62e-4330-afdc-bc7b2464dd39\" (UID: \"733ffc72-f62e-4330-afdc-bc7b2464dd39\") "
Jul 08 16:58:55.458264 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.458096    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/733ffc72-f62e-4330-afdc-bc7b2464dd39-kube-rbac-proxy-sar-config\") pod \"733ffc72-f62e-4330-afdc-bc7b2464dd39\" (UID: \"733ffc72-f62e-4330-afdc-bc7b2464dd39\") "
Jul 08 16:58:55.458264 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.458156    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgmcs\" (UniqueName: \"kubernetes.io/projected/733ffc72-f62e-4330-afdc-bc7b2464dd39-kube-api-access-tgmcs\") pod \"733ffc72-f62e-4330-afdc-bc7b2464dd39\" (UID: \"733ffc72-f62e-4330-afdc-bc7b2464dd39\") "
Jul 08 16:58:55.458454 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.458408    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/733ffc72-f62e-4330-afdc-bc7b2464dd39-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "733ffc72-f62e-4330-afdc-bc7b2464dd39" (UID: "733ffc72-f62e-4330-afdc-bc7b2464dd39"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:58:55.458564 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.458541    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/733ffc72-f62e-4330-afdc-bc7b2464dd39-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "733ffc72-f62e-4330-afdc-bc7b2464dd39" (UID: "733ffc72-f62e-4330-afdc-bc7b2464dd39"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:58:55.461343 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.461267    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/733ffc72-f62e-4330-afdc-bc7b2464dd39-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "733ffc72-f62e-4330-afdc-bc7b2464dd39" (UID: "733ffc72-f62e-4330-afdc-bc7b2464dd39"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 16:58:55.461487 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.461438    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/733ffc72-f62e-4330-afdc-bc7b2464dd39-kube-api-access-tgmcs" (OuterVolumeSpecName: "kube-api-access-tgmcs") pod "733ffc72-f62e-4330-afdc-bc7b2464dd39" (UID: "733ffc72-f62e-4330-afdc-bc7b2464dd39"). InnerVolumeSpecName "kube-api-access-tgmcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 16:58:55.464817 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.464786    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/733ffc72-f62e-4330-afdc-bc7b2464dd39-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "733ffc72-f62e-4330-afdc-bc7b2464dd39" (UID: "733ffc72-f62e-4330-afdc-bc7b2464dd39"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 16:58:55.559648 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.559564    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgmcs\" (UniqueName: \"kubernetes.io/projected/733ffc72-f62e-4330-afdc-bc7b2464dd39-kube-api-access-tgmcs\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:58:55.559648 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.559592    2582 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/733ffc72-f62e-4330-afdc-bc7b2464dd39-kserve-provision-location\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:58:55.559648 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.559602    2582 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/733ffc72-f62e-4330-afdc-bc7b2464dd39-cabundle-cert\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:58:55.559648 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.559611    2582 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/733ffc72-f62e-4330-afdc-bc7b2464dd39-proxy-tls\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:58:55.559648 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.559620    2582 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/733ffc72-f62e-4330-afdc-bc7b2464dd39-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:58:55.948779 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.948747    2582 generic.go:356] "Generic (PLEG): container finished" podID="733ffc72-f62e-4330-afdc-bc7b2464dd39" containerID="6aa0c4f8b9b6e743ca16cd302dc51db711bc5326853fe38281ece0a2bc410510" exitCode=0
Jul 08 16:58:55.949248 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.948803    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"
Jul 08 16:58:55.949248 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.948829    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp" event={"ID":"733ffc72-f62e-4330-afdc-bc7b2464dd39","Type":"ContainerDied","Data":"6aa0c4f8b9b6e743ca16cd302dc51db711bc5326853fe38281ece0a2bc410510"}
Jul 08 16:58:55.949248 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.948913    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp" event={"ID":"733ffc72-f62e-4330-afdc-bc7b2464dd39","Type":"ContainerDied","Data":"e645ba9be758540d34cb1abbdc26ad48aa285953e312558c089f210b2bbf8d48"}
Jul 08 16:58:55.949248 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.948959    2582 scope.go:117] "RemoveContainer" containerID="5d8fe0897afb85abdfa8171d1b9eba01b850995be8a9f9a7100f0d8fd2f4982b"
Jul 08 16:58:55.958721 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.958703    2582 scope.go:117] "RemoveContainer" containerID="6aa0c4f8b9b6e743ca16cd302dc51db711bc5326853fe38281ece0a2bc410510"
Jul 08 16:58:55.966770 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.966754    2582 scope.go:117] "RemoveContainer" containerID="f250f01e44eaa8a1fb9ccbabe343e7a701528b0f06e2f15dfda2d18be990ccc6"
Jul 08 16:58:55.976453 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.976434    2582 scope.go:117] "RemoveContainer" containerID="5d8fe0897afb85abdfa8171d1b9eba01b850995be8a9f9a7100f0d8fd2f4982b"
Jul 08 16:58:55.976534 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.976500    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"]
Jul 08 16:58:55.978445 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:58:55.978231    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d8fe0897afb85abdfa8171d1b9eba01b850995be8a9f9a7100f0d8fd2f4982b\": container with ID starting with 5d8fe0897afb85abdfa8171d1b9eba01b850995be8a9f9a7100f0d8fd2f4982b not found: ID does not exist" containerID="5d8fe0897afb85abdfa8171d1b9eba01b850995be8a9f9a7100f0d8fd2f4982b"
Jul 08 16:58:55.978445 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.978286    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d8fe0897afb85abdfa8171d1b9eba01b850995be8a9f9a7100f0d8fd2f4982b"} err="failed to get container status \"5d8fe0897afb85abdfa8171d1b9eba01b850995be8a9f9a7100f0d8fd2f4982b\": rpc error: code = NotFound desc = could not find container \"5d8fe0897afb85abdfa8171d1b9eba01b850995be8a9f9a7100f0d8fd2f4982b\": container with ID starting with 5d8fe0897afb85abdfa8171d1b9eba01b850995be8a9f9a7100f0d8fd2f4982b not found: ID does not exist"
Jul 08 16:58:55.978445 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.978341    2582 scope.go:117] "RemoveContainer" containerID="6aa0c4f8b9b6e743ca16cd302dc51db711bc5326853fe38281ece0a2bc410510"
Jul 08 16:58:55.979293 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:58:55.979143    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6aa0c4f8b9b6e743ca16cd302dc51db711bc5326853fe38281ece0a2bc410510\": container with ID starting with 6aa0c4f8b9b6e743ca16cd302dc51db711bc5326853fe38281ece0a2bc410510 not found: ID does not exist" containerID="6aa0c4f8b9b6e743ca16cd302dc51db711bc5326853fe38281ece0a2bc410510"
Jul 08 16:58:55.979293 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.979181    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6aa0c4f8b9b6e743ca16cd302dc51db711bc5326853fe38281ece0a2bc410510"} err="failed to get container status \"6aa0c4f8b9b6e743ca16cd302dc51db711bc5326853fe38281ece0a2bc410510\": rpc error: code = NotFound desc = could not find container \"6aa0c4f8b9b6e743ca16cd302dc51db711bc5326853fe38281ece0a2bc410510\": container with ID starting with 6aa0c4f8b9b6e743ca16cd302dc51db711bc5326853fe38281ece0a2bc410510 not found: ID does not exist"
Jul 08 16:58:55.979293 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.979203    2582 scope.go:117] "RemoveContainer" containerID="f250f01e44eaa8a1fb9ccbabe343e7a701528b0f06e2f15dfda2d18be990ccc6"
Jul 08 16:58:55.982704 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:58:55.982674    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f250f01e44eaa8a1fb9ccbabe343e7a701528b0f06e2f15dfda2d18be990ccc6\": container with ID starting with f250f01e44eaa8a1fb9ccbabe343e7a701528b0f06e2f15dfda2d18be990ccc6 not found: ID does not exist" containerID="f250f01e44eaa8a1fb9ccbabe343e7a701528b0f06e2f15dfda2d18be990ccc6"
Jul 08 16:58:55.982811 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.982707    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f250f01e44eaa8a1fb9ccbabe343e7a701528b0f06e2f15dfda2d18be990ccc6"} err="failed to get container status \"f250f01e44eaa8a1fb9ccbabe343e7a701528b0f06e2f15dfda2d18be990ccc6\": rpc error: code = NotFound desc = could not find container \"f250f01e44eaa8a1fb9ccbabe343e7a701528b0f06e2f15dfda2d18be990ccc6\": container with ID starting with f250f01e44eaa8a1fb9ccbabe343e7a701528b0f06e2f15dfda2d18be990ccc6 not found: ID does not exist"
Jul 08 16:58:55.983065 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:55.983042    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-677kp"]
Jul 08 16:58:57.216895 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:57.216865    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="733ffc72-f62e-4330-afdc-bc7b2464dd39" path="/var/lib/kubelet/pods/733ffc72-f62e-4330-afdc-bc7b2464dd39/volumes"
Jul 08 16:58:59.760725 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.760697    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm"]
Jul 08 16:58:59.761122 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.761053    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="733ffc72-f62e-4330-afdc-bc7b2464dd39" containerName="kserve-container"
Jul 08 16:58:59.761122 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.761064    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="733ffc72-f62e-4330-afdc-bc7b2464dd39" containerName="kserve-container"
Jul 08 16:58:59.761122 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.761081    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="733ffc72-f62e-4330-afdc-bc7b2464dd39" containerName="storage-initializer"
Jul 08 16:58:59.761122 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.761087    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="733ffc72-f62e-4330-afdc-bc7b2464dd39" containerName="storage-initializer"
Jul 08 16:58:59.761122 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.761097    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="733ffc72-f62e-4330-afdc-bc7b2464dd39" containerName="kube-rbac-proxy"
Jul 08 16:58:59.761122 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.761103    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="733ffc72-f62e-4330-afdc-bc7b2464dd39" containerName="kube-rbac-proxy"
Jul 08 16:58:59.761343 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.761159    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="733ffc72-f62e-4330-afdc-bc7b2464dd39" containerName="kube-rbac-proxy"
Jul 08 16:58:59.761343 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.761169    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="733ffc72-f62e-4330-afdc-bc7b2464dd39" containerName="kserve-container"
Jul 08 16:58:59.766092 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.766075    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm"
Jul 08 16:58:59.769223 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.769197    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 08 16:58:59.769405 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.769247    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 08 16:58:59.769405 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.769197    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 08 16:58:59.769542 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.769411    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-jq2mg\""
Jul 08 16:58:59.769599 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.769549    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-ts-v1-kube-rbac-proxy-sar-config\""
Jul 08 16:58:59.769599 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.769572    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-ts-v1-predictor-serving-cert\""
Jul 08 16:58:59.770026 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.770010    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 08 16:58:59.775152 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.775130    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm"]
Jul 08 16:58:59.791784 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.791755    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-kserve-provision-location\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm\" (UID: \"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm"
Jul 08 16:58:59.791907 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.791801    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-cabundle-cert\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm\" (UID: \"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm"
Jul 08 16:58:59.791907 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.791823    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b894k\" (UniqueName: \"kubernetes.io/projected/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-kube-api-access-b894k\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm\" (UID: \"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm"
Jul 08 16:58:59.791979 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.791913    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm\" (UID: \"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm"
Jul 08 16:58:59.791979 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.791943    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm\" (UID: \"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm"
Jul 08 16:58:59.892727 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.892693    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-kserve-provision-location\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm\" (UID: \"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm"
Jul 08 16:58:59.892932 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.892741    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-cabundle-cert\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm\" (UID: \"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm"
Jul 08 16:58:59.892932 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.892766    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b894k\" (UniqueName: \"kubernetes.io/projected/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-kube-api-access-b894k\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm\" (UID: \"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm"
Jul 08 16:58:59.892932 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.892808    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm\" (UID: \"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm"
Jul 08 16:58:59.892932 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.892828    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm\" (UID: \"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm"
Jul 08 16:58:59.893188 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.893165    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-kserve-provision-location\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm\" (UID: \"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm"
Jul 08 16:58:59.893615 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.893594    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm\" (UID: \"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm"
Jul 08 16:58:59.893678 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.893594    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-cabundle-cert\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm\" (UID: \"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm"
Jul 08 16:58:59.895386 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.895367    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm\" (UID: \"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm"
Jul 08 16:58:59.904959 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:58:59.904934    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b894k\" (UniqueName: \"kubernetes.io/projected/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-kube-api-access-b894k\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm\" (UID: \"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm"
Jul 08 16:59:00.078217 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:00.078129    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm"
Jul 08 16:59:00.211232 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:00.211206    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm"]
Jul 08 16:59:00.213387 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:59:00.213359    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ea1c4d5_4681_4fc6_9311_97ef52cd8f64.slice/crio-7bb3c385d230cf340c92cc12978703ef38c6d6b956f0c954c1d3ff3d4ea7cdee WatchSource:0}: Error finding container 7bb3c385d230cf340c92cc12978703ef38c6d6b956f0c954c1d3ff3d4ea7cdee: Status 404 returned error can't find the container with id 7bb3c385d230cf340c92cc12978703ef38c6d6b956f0c954c1d3ff3d4ea7cdee
Jul 08 16:59:00.972474 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:00.972435    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm" event={"ID":"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64","Type":"ContainerStarted","Data":"ad6d3c1641697c3ffd1f92d27246838b34c0b017a0817cc4276c65da7fd194fa"}
Jul 08 16:59:00.972911 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:00.972490    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm" event={"ID":"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64","Type":"ContainerStarted","Data":"7bb3c385d230cf340c92cc12978703ef38c6d6b956f0c954c1d3ff3d4ea7cdee"}
Jul 08 16:59:09.019090 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:09.019059    2582 generic.go:356] "Generic (PLEG): container finished" podID="9ea1c4d5-4681-4fc6-9311-97ef52cd8f64" containerID="ad6d3c1641697c3ffd1f92d27246838b34c0b017a0817cc4276c65da7fd194fa" exitCode=0
Jul 08 16:59:09.019499 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:09.019129    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm" event={"ID":"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64","Type":"ContainerDied","Data":"ad6d3c1641697c3ffd1f92d27246838b34c0b017a0817cc4276c65da7fd194fa"}
Jul 08 16:59:10.025768 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:10.025731    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm" event={"ID":"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64","Type":"ContainerStarted","Data":"53d7c98bce09a3fa8b64b7a2b211569d6c98fabef08f996bd95d6f13ca89707f"}
Jul 08 16:59:10.026172 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:10.025794    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm" event={"ID":"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64","Type":"ContainerStarted","Data":"c617cf2e56ebe22c81895a5c94e1e2c45445f2464969b4d8b029cb3e0465c652"}
Jul 08 16:59:10.026172 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:10.026074    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm"
Jul 08 16:59:10.026263 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:10.026227    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm"
Jul 08 16:59:10.027425 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:10.027401    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm" podUID="9ea1c4d5-4681-4fc6-9311-97ef52cd8f64" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Jul 08 16:59:10.046438 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:10.046386    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm" podStartSLOduration=2.748523393 podStartE2EDuration="11.046369214s" totalImagesPullingTime="0s" totalInitContainerRuntime="8.297845821s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:58:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:59:10.044794162 +0000 UTC m=+1469.385759242" watchObservedRunningTime="2026-07-08 16:59:10.046369214 +0000 UTC m=+1469.387334292"
Jul 08 16:59:11.030640 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:11.030599    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm" podUID="9ea1c4d5-4681-4fc6-9311-97ef52cd8f64" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Jul 08 16:59:16.035202 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:16.035172    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm"
Jul 08 16:59:16.035872 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:16.035664    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm" podUID="9ea1c4d5-4681-4fc6-9311-97ef52cd8f64" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Jul 08 16:59:26.036523 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:26.036489    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm"
Jul 08 16:59:29.792673 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:29.792637    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm"]
Jul 08 16:59:29.793075 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:29.792878    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm" podUID="9ea1c4d5-4681-4fc6-9311-97ef52cd8f64" containerName="kserve-container" containerID="cri-o://c617cf2e56ebe22c81895a5c94e1e2c45445f2464969b4d8b029cb3e0465c652" gracePeriod=30
Jul 08 16:59:29.793075 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:29.792925    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm" podUID="9ea1c4d5-4681-4fc6-9311-97ef52cd8f64" containerName="kube-rbac-proxy" containerID="cri-o://53d7c98bce09a3fa8b64b7a2b211569d6c98fabef08f996bd95d6f13ca89707f" gracePeriod=30
Jul 08 16:59:30.116014 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:30.115923    2582 generic.go:356] "Generic (PLEG): container finished" podID="9ea1c4d5-4681-4fc6-9311-97ef52cd8f64" containerID="53d7c98bce09a3fa8b64b7a2b211569d6c98fabef08f996bd95d6f13ca89707f" exitCode=2
Jul 08 16:59:30.116014 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:30.116003    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm" event={"ID":"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64","Type":"ContainerDied","Data":"53d7c98bce09a3fa8b64b7a2b211569d6c98fabef08f996bd95d6f13ca89707f"}
Jul 08 16:59:30.639364 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:30.639334    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm"
Jul 08 16:59:30.778484 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:30.778394    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-proxy-tls\") pod \"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64\" (UID: \"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64\") "
Jul 08 16:59:30.778484 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:30.778442    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-kserve-provision-location\") pod \"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64\" (UID: \"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64\") "
Jul 08 16:59:30.778484 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:30.778462    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b894k\" (UniqueName: \"kubernetes.io/projected/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-kube-api-access-b894k\") pod \"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64\" (UID: \"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64\") "
Jul 08 16:59:30.778776 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:30.778510    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-cabundle-cert\") pod \"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64\" (UID: \"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64\") "
Jul 08 16:59:30.778776 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:30.778529    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-kube-rbac-proxy-sar-config\") pod \"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64\" (UID: \"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64\") "
Jul 08 16:59:30.778883 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:30.778864    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "9ea1c4d5-4681-4fc6-9311-97ef52cd8f64" (UID: "9ea1c4d5-4681-4fc6-9311-97ef52cd8f64"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:59:30.778963 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:30.778936    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "9ea1c4d5-4681-4fc6-9311-97ef52cd8f64" (UID: "9ea1c4d5-4681-4fc6-9311-97ef52cd8f64"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 16:59:30.779626 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:30.779601    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "9ea1c4d5-4681-4fc6-9311-97ef52cd8f64" (UID: "9ea1c4d5-4681-4fc6-9311-97ef52cd8f64"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 16:59:30.780786 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:30.780766    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-kube-api-access-b894k" (OuterVolumeSpecName: "kube-api-access-b894k") pod "9ea1c4d5-4681-4fc6-9311-97ef52cd8f64" (UID: "9ea1c4d5-4681-4fc6-9311-97ef52cd8f64"). InnerVolumeSpecName "kube-api-access-b894k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 16:59:30.780786 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:30.780772    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "9ea1c4d5-4681-4fc6-9311-97ef52cd8f64" (UID: "9ea1c4d5-4681-4fc6-9311-97ef52cd8f64"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 16:59:30.879395 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:30.879358    2582 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-proxy-tls\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:59:30.879395 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:30.879389    2582 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-kserve-provision-location\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:59:30.879395 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:30.879399    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b894k\" (UniqueName: \"kubernetes.io/projected/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-kube-api-access-b894k\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:59:30.879838 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:30.879410    2582 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-cabundle-cert\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:59:30.879838 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:30.879418    2582 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 16:59:31.122509 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:31.122418    2582 generic.go:356] "Generic (PLEG): container finished" podID="9ea1c4d5-4681-4fc6-9311-97ef52cd8f64" containerID="c617cf2e56ebe22c81895a5c94e1e2c45445f2464969b4d8b029cb3e0465c652" exitCode=0
Jul 08 16:59:31.122509 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:31.122461    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm" event={"ID":"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64","Type":"ContainerDied","Data":"c617cf2e56ebe22c81895a5c94e1e2c45445f2464969b4d8b029cb3e0465c652"}
Jul 08 16:59:31.122509 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:31.122484    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm"
Jul 08 16:59:31.122509 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:31.122508    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm" event={"ID":"9ea1c4d5-4681-4fc6-9311-97ef52cd8f64","Type":"ContainerDied","Data":"7bb3c385d230cf340c92cc12978703ef38c6d6b956f0c954c1d3ff3d4ea7cdee"}
Jul 08 16:59:31.122822 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:31.122534    2582 scope.go:117] "RemoveContainer" containerID="53d7c98bce09a3fa8b64b7a2b211569d6c98fabef08f996bd95d6f13ca89707f"
Jul 08 16:59:31.131864 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:31.131847    2582 scope.go:117] "RemoveContainer" containerID="c617cf2e56ebe22c81895a5c94e1e2c45445f2464969b4d8b029cb3e0465c652"
Jul 08 16:59:31.139947 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:31.139929    2582 scope.go:117] "RemoveContainer" containerID="ad6d3c1641697c3ffd1f92d27246838b34c0b017a0817cc4276c65da7fd194fa"
Jul 08 16:59:31.144413 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:31.144391    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm"]
Jul 08 16:59:31.147519 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:31.147499    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-txkbm"]
Jul 08 16:59:31.147886 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:31.147871    2582 scope.go:117] "RemoveContainer" containerID="53d7c98bce09a3fa8b64b7a2b211569d6c98fabef08f996bd95d6f13ca89707f"
Jul 08 16:59:31.148141 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:59:31.148122    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53d7c98bce09a3fa8b64b7a2b211569d6c98fabef08f996bd95d6f13ca89707f\": container with ID starting with 53d7c98bce09a3fa8b64b7a2b211569d6c98fabef08f996bd95d6f13ca89707f not found: ID does not exist" containerID="53d7c98bce09a3fa8b64b7a2b211569d6c98fabef08f996bd95d6f13ca89707f"
Jul 08 16:59:31.148188 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:31.148148    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53d7c98bce09a3fa8b64b7a2b211569d6c98fabef08f996bd95d6f13ca89707f"} err="failed to get container status \"53d7c98bce09a3fa8b64b7a2b211569d6c98fabef08f996bd95d6f13ca89707f\": rpc error: code = NotFound desc = could not find container \"53d7c98bce09a3fa8b64b7a2b211569d6c98fabef08f996bd95d6f13ca89707f\": container with ID starting with 53d7c98bce09a3fa8b64b7a2b211569d6c98fabef08f996bd95d6f13ca89707f not found: ID does not exist"
Jul 08 16:59:31.148188 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:31.148163    2582 scope.go:117] "RemoveContainer" containerID="c617cf2e56ebe22c81895a5c94e1e2c45445f2464969b4d8b029cb3e0465c652"
Jul 08 16:59:31.148439 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:59:31.148422    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c617cf2e56ebe22c81895a5c94e1e2c45445f2464969b4d8b029cb3e0465c652\": container with ID starting with c617cf2e56ebe22c81895a5c94e1e2c45445f2464969b4d8b029cb3e0465c652 not found: ID does not exist" containerID="c617cf2e56ebe22c81895a5c94e1e2c45445f2464969b4d8b029cb3e0465c652"
Jul 08 16:59:31.148487 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:31.148445    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c617cf2e56ebe22c81895a5c94e1e2c45445f2464969b4d8b029cb3e0465c652"} err="failed to get container status \"c617cf2e56ebe22c81895a5c94e1e2c45445f2464969b4d8b029cb3e0465c652\": rpc error: code = NotFound desc = could not find container \"c617cf2e56ebe22c81895a5c94e1e2c45445f2464969b4d8b029cb3e0465c652\": container with ID starting with c617cf2e56ebe22c81895a5c94e1e2c45445f2464969b4d8b029cb3e0465c652 not found: ID does not exist"
Jul 08 16:59:31.148487 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:31.148461    2582 scope.go:117] "RemoveContainer" containerID="ad6d3c1641697c3ffd1f92d27246838b34c0b017a0817cc4276c65da7fd194fa"
Jul 08 16:59:31.148711 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:59:31.148692    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad6d3c1641697c3ffd1f92d27246838b34c0b017a0817cc4276c65da7fd194fa\": container with ID starting with ad6d3c1641697c3ffd1f92d27246838b34c0b017a0817cc4276c65da7fd194fa not found: ID does not exist" containerID="ad6d3c1641697c3ffd1f92d27246838b34c0b017a0817cc4276c65da7fd194fa"
Jul 08 16:59:31.148750 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:31.148718    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad6d3c1641697c3ffd1f92d27246838b34c0b017a0817cc4276c65da7fd194fa"} err="failed to get container status \"ad6d3c1641697c3ffd1f92d27246838b34c0b017a0817cc4276c65da7fd194fa\": rpc error: code = NotFound desc = could not find container \"ad6d3c1641697c3ffd1f92d27246838b34c0b017a0817cc4276c65da7fd194fa\": container with ID starting with ad6d3c1641697c3ffd1f92d27246838b34c0b017a0817cc4276c65da7fd194fa not found: ID does not exist"
Jul 08 16:59:31.217010 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:31.216975    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ea1c4d5-4681-4fc6-9311-97ef52cd8f64" path="/var/lib/kubelet/pods/9ea1c4d5-4681-4fc6-9311-97ef52cd8f64/volumes"
Jul 08 16:59:36.113328 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:36.113287    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-k7rxk/must-gather-xgtf4"]
Jul 08 16:59:36.113721 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:36.113637    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ea1c4d5-4681-4fc6-9311-97ef52cd8f64" containerName="kserve-container"
Jul 08 16:59:36.113721 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:36.113649    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ea1c4d5-4681-4fc6-9311-97ef52cd8f64" containerName="kserve-container"
Jul 08 16:59:36.113721 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:36.113657    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ea1c4d5-4681-4fc6-9311-97ef52cd8f64" containerName="kube-rbac-proxy"
Jul 08 16:59:36.113721 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:36.113662    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ea1c4d5-4681-4fc6-9311-97ef52cd8f64" containerName="kube-rbac-proxy"
Jul 08 16:59:36.113721 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:36.113682    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ea1c4d5-4681-4fc6-9311-97ef52cd8f64" containerName="storage-initializer"
Jul 08 16:59:36.113721 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:36.113688    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ea1c4d5-4681-4fc6-9311-97ef52cd8f64" containerName="storage-initializer"
Jul 08 16:59:36.113911 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:36.113746    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ea1c4d5-4681-4fc6-9311-97ef52cd8f64" containerName="kserve-container"
Jul 08 16:59:36.113911 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:36.113755    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ea1c4d5-4681-4fc6-9311-97ef52cd8f64" containerName="kube-rbac-proxy"
Jul 08 16:59:36.118388 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:36.118367    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k7rxk/must-gather-xgtf4"
Jul 08 16:59:36.120724 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:36.120705    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-k7rxk\"/\"kube-root-ca.crt\""
Jul 08 16:59:36.120823 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:36.120707    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-k7rxk\"/\"openshift-service-ca.crt\""
Jul 08 16:59:36.121569 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:36.121552    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-k7rxk\"/\"default-dockercfg-dgtk5\""
Jul 08 16:59:36.124544 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:36.124523    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k7rxk/must-gather-xgtf4"]
Jul 08 16:59:36.129064 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:36.129043    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1e78d03d-0506-4503-b34e-dbb6d40c1d45-must-gather-output\") pod \"must-gather-xgtf4\" (UID: \"1e78d03d-0506-4503-b34e-dbb6d40c1d45\") " pod="openshift-must-gather-k7rxk/must-gather-xgtf4"
Jul 08 16:59:36.129166 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:36.129076    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zlfv\" (UniqueName: \"kubernetes.io/projected/1e78d03d-0506-4503-b34e-dbb6d40c1d45-kube-api-access-9zlfv\") pod \"must-gather-xgtf4\" (UID: \"1e78d03d-0506-4503-b34e-dbb6d40c1d45\") " pod="openshift-must-gather-k7rxk/must-gather-xgtf4"
Jul 08 16:59:36.230053 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:36.230015    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1e78d03d-0506-4503-b34e-dbb6d40c1d45-must-gather-output\") pod \"must-gather-xgtf4\" (UID: \"1e78d03d-0506-4503-b34e-dbb6d40c1d45\") " pod="openshift-must-gather-k7rxk/must-gather-xgtf4"
Jul 08 16:59:36.230053 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:36.230053    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zlfv\" (UniqueName: \"kubernetes.io/projected/1e78d03d-0506-4503-b34e-dbb6d40c1d45-kube-api-access-9zlfv\") pod \"must-gather-xgtf4\" (UID: \"1e78d03d-0506-4503-b34e-dbb6d40c1d45\") " pod="openshift-must-gather-k7rxk/must-gather-xgtf4"
Jul 08 16:59:36.230382 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:36.230365    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1e78d03d-0506-4503-b34e-dbb6d40c1d45-must-gather-output\") pod \"must-gather-xgtf4\" (UID: \"1e78d03d-0506-4503-b34e-dbb6d40c1d45\") " pod="openshift-must-gather-k7rxk/must-gather-xgtf4"
Jul 08 16:59:36.238995 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:36.238969    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zlfv\" (UniqueName: \"kubernetes.io/projected/1e78d03d-0506-4503-b34e-dbb6d40c1d45-kube-api-access-9zlfv\") pod \"must-gather-xgtf4\" (UID: \"1e78d03d-0506-4503-b34e-dbb6d40c1d45\") " pod="openshift-must-gather-k7rxk/must-gather-xgtf4"
Jul 08 16:59:36.437625 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:36.437594    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k7rxk/must-gather-xgtf4"
Jul 08 16:59:36.572142 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:36.572110    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k7rxk/must-gather-xgtf4"]
Jul 08 16:59:36.573575 ip-10-0-139-237 kubenswrapper[2582]: W0708 16:59:36.573542    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e78d03d_0506_4503_b34e_dbb6d40c1d45.slice/crio-3621702e519c142b73d3a227a7ce30e83b1996d85c0d40ee0f824671c6011a28 WatchSource:0}: Error finding container 3621702e519c142b73d3a227a7ce30e83b1996d85c0d40ee0f824671c6011a28: Status 404 returned error can't find the container with id 3621702e519c142b73d3a227a7ce30e83b1996d85c0d40ee0f824671c6011a28
Jul 08 16:59:36.575476 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:36.575456    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 16:59:37.156792 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:37.156757    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k7rxk/must-gather-xgtf4" event={"ID":"1e78d03d-0506-4503-b34e-dbb6d40c1d45","Type":"ContainerStarted","Data":"3621702e519c142b73d3a227a7ce30e83b1996d85c0d40ee0f824671c6011a28"}
Jul 08 16:59:42.799211 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:42.799185    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kvhpr_19df7751-4b62-4ae5-92a6-0bbda11fec85/ovn-acl-logging/0.log"
Jul 08 16:59:42.799665 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:42.799235    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kvhpr_19df7751-4b62-4ae5-92a6-0bbda11fec85/ovn-acl-logging/0.log"
Jul 08 16:59:42.804409 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:59:42.804375    2582 log.go:32] "ImageFsInfo from image service failed" err="rpc error: code = Unknown desc = get image fs info unable to get usage for /var/lib/containers/storage/overlay-images: get disk usage for path /var/lib/containers/storage/overlay-images: lstat /var/lib/containers/storage/overlay-images/.tmp-images.json4135480177: no such file or directory"
Jul 08 16:59:42.804546 ip-10-0-139-237 kubenswrapper[2582]: E0708 16:59:42.804422    2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="missing image stats: <nil>"
Jul 08 16:59:43.190329 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:43.190253    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k7rxk/must-gather-xgtf4" event={"ID":"1e78d03d-0506-4503-b34e-dbb6d40c1d45","Type":"ContainerStarted","Data":"e8ee22651fb7f90b3984c414beb1b85a9946e5b5c141c2b19159c72aa9be17b4"}
Jul 08 16:59:43.190527 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:43.190349    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k7rxk/must-gather-xgtf4" event={"ID":"1e78d03d-0506-4503-b34e-dbb6d40c1d45","Type":"ContainerStarted","Data":"42cec3a2b4bf423a83eac21ed63a1f3407ab80c8862186cf13f15b32e584166b"}
Jul 08 16:59:43.208167 ip-10-0-139-237 kubenswrapper[2582]: I0708 16:59:43.208107    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-k7rxk/must-gather-xgtf4" podStartSLOduration=0.935983913 podStartE2EDuration="7.208091667s" totalImagesPullingTime="6.272107754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 16:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 16:59:43.206682074 +0000 UTC m=+1502.547647152" watchObservedRunningTime="2026-07-08 16:59:43.208091667 +0000 UTC m=+1502.549056743"
Jul 08 17:00:04.296526 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:04.296491    2582 generic.go:356] "Generic (PLEG): container finished" podID="1e78d03d-0506-4503-b34e-dbb6d40c1d45" containerID="42cec3a2b4bf423a83eac21ed63a1f3407ab80c8862186cf13f15b32e584166b" exitCode=0
Jul 08 17:00:04.296951 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:04.296565    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k7rxk/must-gather-xgtf4" event={"ID":"1e78d03d-0506-4503-b34e-dbb6d40c1d45","Type":"ContainerDied","Data":"42cec3a2b4bf423a83eac21ed63a1f3407ab80c8862186cf13f15b32e584166b"}
Jul 08 17:00:04.296951 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:04.296904    2582 scope.go:117] "RemoveContainer" containerID="42cec3a2b4bf423a83eac21ed63a1f3407ab80c8862186cf13f15b32e584166b"
Jul 08 17:00:08.754483 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:08.754447    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-k7rxk_must-gather-xgtf4_1e78d03d-0506-4503-b34e-dbb6d40c1d45/gather/0.log"
Jul 08 17:00:10.311994 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:10.311961    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-72rhc/must-gather-qkf4z"]
Jul 08 17:00:10.314435 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:10.314416    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-72rhc/must-gather-qkf4z"
Jul 08 17:00:10.316818 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:10.316795    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-72rhc\"/\"openshift-service-ca.crt\""
Jul 08 17:00:10.317682 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:10.317664    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-72rhc\"/\"default-dockercfg-rrf2d\""
Jul 08 17:00:10.317783 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:10.317726    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-72rhc\"/\"kube-root-ca.crt\""
Jul 08 17:00:10.324340 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:10.324301    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-72rhc/must-gather-qkf4z"]
Jul 08 17:00:10.455699 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:10.455664    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d3fc28de-1f27-4dac-b4a7-6da4ea70d2a6-must-gather-output\") pod \"must-gather-qkf4z\" (UID: \"d3fc28de-1f27-4dac-b4a7-6da4ea70d2a6\") " pod="openshift-must-gather-72rhc/must-gather-qkf4z"
Jul 08 17:00:10.455699 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:10.455706    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vsxt\" (UniqueName: \"kubernetes.io/projected/d3fc28de-1f27-4dac-b4a7-6da4ea70d2a6-kube-api-access-4vsxt\") pod \"must-gather-qkf4z\" (UID: \"d3fc28de-1f27-4dac-b4a7-6da4ea70d2a6\") " pod="openshift-must-gather-72rhc/must-gather-qkf4z"
Jul 08 17:00:10.556450 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:10.556414    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d3fc28de-1f27-4dac-b4a7-6da4ea70d2a6-must-gather-output\") pod \"must-gather-qkf4z\" (UID: \"d3fc28de-1f27-4dac-b4a7-6da4ea70d2a6\") " pod="openshift-must-gather-72rhc/must-gather-qkf4z"
Jul 08 17:00:10.556450 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:10.556456    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vsxt\" (UniqueName: \"kubernetes.io/projected/d3fc28de-1f27-4dac-b4a7-6da4ea70d2a6-kube-api-access-4vsxt\") pod \"must-gather-qkf4z\" (UID: \"d3fc28de-1f27-4dac-b4a7-6da4ea70d2a6\") " pod="openshift-must-gather-72rhc/must-gather-qkf4z"
Jul 08 17:00:10.556788 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:10.556770    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d3fc28de-1f27-4dac-b4a7-6da4ea70d2a6-must-gather-output\") pod \"must-gather-qkf4z\" (UID: \"d3fc28de-1f27-4dac-b4a7-6da4ea70d2a6\") " pod="openshift-must-gather-72rhc/must-gather-qkf4z"
Jul 08 17:00:10.565256 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:10.565193    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vsxt\" (UniqueName: \"kubernetes.io/projected/d3fc28de-1f27-4dac-b4a7-6da4ea70d2a6-kube-api-access-4vsxt\") pod \"must-gather-qkf4z\" (UID: \"d3fc28de-1f27-4dac-b4a7-6da4ea70d2a6\") " pod="openshift-must-gather-72rhc/must-gather-qkf4z"
Jul 08 17:00:10.624701 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:10.624664    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-72rhc/must-gather-qkf4z"
Jul 08 17:00:10.760597 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:10.760566    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-72rhc/must-gather-qkf4z"]
Jul 08 17:00:10.762740 ip-10-0-139-237 kubenswrapper[2582]: W0708 17:00:10.762707    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3fc28de_1f27_4dac_b4a7_6da4ea70d2a6.slice/crio-37f2232e974c29007f1addce0ac6af26406f82b60997d88808a285f0fa785548 WatchSource:0}: Error finding container 37f2232e974c29007f1addce0ac6af26406f82b60997d88808a285f0fa785548: Status 404 returned error can't find the container with id 37f2232e974c29007f1addce0ac6af26406f82b60997d88808a285f0fa785548
Jul 08 17:00:11.325803 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:11.325768    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-72rhc/must-gather-qkf4z" event={"ID":"d3fc28de-1f27-4dac-b4a7-6da4ea70d2a6","Type":"ContainerStarted","Data":"37f2232e974c29007f1addce0ac6af26406f82b60997d88808a285f0fa785548"}
Jul 08 17:00:12.332572 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:12.332535    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-72rhc/must-gather-qkf4z" event={"ID":"d3fc28de-1f27-4dac-b4a7-6da4ea70d2a6","Type":"ContainerStarted","Data":"9e2fa2a01e2800ad67c9ef301250d6c950f648e0352e7d0278e6b2eee4a98f8f"}
Jul 08 17:00:12.333067 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:12.332588    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-72rhc/must-gather-qkf4z" event={"ID":"d3fc28de-1f27-4dac-b4a7-6da4ea70d2a6","Type":"ContainerStarted","Data":"2f78ff22b811a213c8cdd0dc3d8cbfb0598773980f144a306b4994b4f0391da8"}
Jul 08 17:00:12.349662 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:12.349607    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-72rhc/must-gather-qkf4z" podStartSLOduration=1.272350508 podStartE2EDuration="2.349592314s" totalImagesPullingTime="1.077241806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:00:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:00:12.348514858 +0000 UTC m=+1531.689479963" watchObservedRunningTime="2026-07-08 17:00:12.349592314 +0000 UTC m=+1531.690557396"
Jul 08 17:00:13.278434 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:13.278408    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4vtkl_8eb2abc0-b610-4f3b-b6dc-02131ad29e35/global-pull-secret-syncer/0.log"
Jul 08 17:00:13.450457 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:13.450418    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-957gg_f877b6b1-282c-4c50-95d6-530371d41a15/konnectivity-agent/0.log"
Jul 08 17:00:13.527977 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:13.527947    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-237.ec2.internal_81aa7add2915dbb7aae72d3a67b06d72/haproxy/0.log"
Jul 08 17:00:15.150993 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:15.150936    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-k7rxk/must-gather-xgtf4"]
Jul 08 17:00:15.151519 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:15.151206    2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-must-gather-k7rxk/must-gather-xgtf4" podUID="1e78d03d-0506-4503-b34e-dbb6d40c1d45" containerName="copy" containerID="cri-o://e8ee22651fb7f90b3984c414beb1b85a9946e5b5c141c2b19159c72aa9be17b4" gracePeriod=2
Jul 08 17:00:15.153049 ip-10-0-139-237 kubenswrapper[2582]: E0708 17:00:15.153019    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-xgtf4\" is forbidden: User \"system:node:ip-10-0-139-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-k7rxk\": no relationship found between node 'ip-10-0-139-237.ec2.internal' and this object" podUID="1e78d03d-0506-4503-b34e-dbb6d40c1d45" pod="openshift-must-gather-k7rxk/must-gather-xgtf4"
Jul 08 17:00:15.154782 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:15.154613    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-k7rxk/must-gather-xgtf4"]
Jul 08 17:00:15.353801 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:15.352332    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-k7rxk_must-gather-xgtf4_1e78d03d-0506-4503-b34e-dbb6d40c1d45/copy/0.log"
Jul 08 17:00:15.353801 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:15.352750    2582 generic.go:356] "Generic (PLEG): container finished" podID="1e78d03d-0506-4503-b34e-dbb6d40c1d45" containerID="e8ee22651fb7f90b3984c414beb1b85a9946e5b5c141c2b19159c72aa9be17b4" exitCode=143
Jul 08 17:00:15.570417 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:15.568777    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-k7rxk_must-gather-xgtf4_1e78d03d-0506-4503-b34e-dbb6d40c1d45/copy/0.log"
Jul 08 17:00:15.576186 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:15.573797    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k7rxk/must-gather-xgtf4"
Jul 08 17:00:15.614509 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:15.614456    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1e78d03d-0506-4503-b34e-dbb6d40c1d45-must-gather-output\") pod \"1e78d03d-0506-4503-b34e-dbb6d40c1d45\" (UID: \"1e78d03d-0506-4503-b34e-dbb6d40c1d45\") "
Jul 08 17:00:15.614708 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:15.614518    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zlfv\" (UniqueName: \"kubernetes.io/projected/1e78d03d-0506-4503-b34e-dbb6d40c1d45-kube-api-access-9zlfv\") pod \"1e78d03d-0506-4503-b34e-dbb6d40c1d45\" (UID: \"1e78d03d-0506-4503-b34e-dbb6d40c1d45\") "
Jul 08 17:00:15.617695 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:15.617140    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e78d03d-0506-4503-b34e-dbb6d40c1d45-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "1e78d03d-0506-4503-b34e-dbb6d40c1d45" (UID: "1e78d03d-0506-4503-b34e-dbb6d40c1d45"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 17:00:15.620730 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:15.620683    2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e78d03d-0506-4503-b34e-dbb6d40c1d45-kube-api-access-9zlfv" (OuterVolumeSpecName: "kube-api-access-9zlfv") pod "1e78d03d-0506-4503-b34e-dbb6d40c1d45" (UID: "1e78d03d-0506-4503-b34e-dbb6d40c1d45"). InnerVolumeSpecName "kube-api-access-9zlfv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 17:00:15.715935 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:15.715890    2582 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1e78d03d-0506-4503-b34e-dbb6d40c1d45-must-gather-output\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 17:00:15.715935 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:15.715935    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zlfv\" (UniqueName: \"kubernetes.io/projected/1e78d03d-0506-4503-b34e-dbb6d40c1d45-kube-api-access-9zlfv\") on node \"ip-10-0-139-237.ec2.internal\" DevicePath \"\""
Jul 08 17:00:16.369202 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:16.368200    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-k7rxk_must-gather-xgtf4_1e78d03d-0506-4503-b34e-dbb6d40c1d45/copy/0.log"
Jul 08 17:00:16.369202 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:16.368852    2582 scope.go:117] "RemoveContainer" containerID="e8ee22651fb7f90b3984c414beb1b85a9946e5b5c141c2b19159c72aa9be17b4"
Jul 08 17:00:16.369202 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:16.369046    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k7rxk/must-gather-xgtf4"
Jul 08 17:00:16.409133 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:16.408703    2582 scope.go:117] "RemoveContainer" containerID="42cec3a2b4bf423a83eac21ed63a1f3407ab80c8862186cf13f15b32e584166b"
Jul 08 17:00:17.221230 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:17.221189    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e78d03d-0506-4503-b34e-dbb6d40c1d45" path="/var/lib/kubelet/pods/1e78d03d-0506-4503-b34e-dbb6d40c1d45/volumes"
Jul 08 17:00:17.474833 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:17.474749    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_45190ca2-bc75-41c0-8d71-d01d5728f856/alertmanager/0.log"
Jul 08 17:00:17.504123 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:17.504084    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_45190ca2-bc75-41c0-8d71-d01d5728f856/config-reloader/0.log"
Jul 08 17:00:17.539009 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:17.538974    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_45190ca2-bc75-41c0-8d71-d01d5728f856/kube-rbac-proxy-web/0.log"
Jul 08 17:00:17.563791 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:17.563761    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_45190ca2-bc75-41c0-8d71-d01d5728f856/kube-rbac-proxy/0.log"
Jul 08 17:00:17.594084 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:17.594021    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_45190ca2-bc75-41c0-8d71-d01d5728f856/kube-rbac-proxy-metric/0.log"
Jul 08 17:00:17.626327 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:17.626280    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_45190ca2-bc75-41c0-8d71-d01d5728f856/prom-label-proxy/0.log"
Jul 08 17:00:17.668779 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:17.668745    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_45190ca2-bc75-41c0-8d71-d01d5728f856/init-config-reloader/0.log"
Jul 08 17:00:17.748779 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:17.748689    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-ckxc5_ade5e891-8f50-4385-9388-6ebfad50927d/kube-state-metrics/0.log"
Jul 08 17:00:17.788285 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:17.788252    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-ckxc5_ade5e891-8f50-4385-9388-6ebfad50927d/kube-rbac-proxy-main/0.log"
Jul 08 17:00:17.823730 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:17.823673    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-ckxc5_ade5e891-8f50-4385-9388-6ebfad50927d/kube-rbac-proxy-self/0.log"
Jul 08 17:00:17.871830 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:17.871797    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6f8c55cbdf-96278_a77c65f3-98f6-49b6-bbb9-b64e0906f14e/metrics-server/0.log"
Jul 08 17:00:17.946934 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:17.946902    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-btd5p_3f03d25a-b3a9-4497-9b40-22e97ae5fcf6/monitoring-plugin/0.log"
Jul 08 17:00:18.120884 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:18.120837    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fd6f4_8f30902a-26b7-48d4-92f1-d0ca067dab66/node-exporter/0.log"
Jul 08 17:00:18.146362 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:18.146303    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fd6f4_8f30902a-26b7-48d4-92f1-d0ca067dab66/kube-rbac-proxy/0.log"
Jul 08 17:00:18.178773 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:18.178737    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fd6f4_8f30902a-26b7-48d4-92f1-d0ca067dab66/init-textfile/0.log"
Jul 08 17:00:18.302639 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:18.302610    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-fsp5w_3ed65374-16ab-405d-96b6-69d3318cc2e7/kube-rbac-proxy-main/0.log"
Jul 08 17:00:18.332075 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:18.331994    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-fsp5w_3ed65374-16ab-405d-96b6-69d3318cc2e7/kube-rbac-proxy-self/0.log"
Jul 08 17:00:18.359591 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:18.359543    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-fsp5w_3ed65374-16ab-405d-96b6-69d3318cc2e7/openshift-state-metrics/0.log"
Jul 08 17:00:18.407298 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:18.407247    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bd303057-abe0-43bc-a865-5e735e00ba8d/prometheus/0.log"
Jul 08 17:00:18.430439 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:18.430407    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bd303057-abe0-43bc-a865-5e735e00ba8d/config-reloader/0.log"
Jul 08 17:00:18.455549 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:18.455518    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bd303057-abe0-43bc-a865-5e735e00ba8d/thanos-sidecar/0.log"
Jul 08 17:00:18.485845 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:18.485813    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bd303057-abe0-43bc-a865-5e735e00ba8d/kube-rbac-proxy-web/0.log"
Jul 08 17:00:18.525286 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:18.525250    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bd303057-abe0-43bc-a865-5e735e00ba8d/kube-rbac-proxy/0.log"
Jul 08 17:00:18.576126 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:18.576078    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bd303057-abe0-43bc-a865-5e735e00ba8d/kube-rbac-proxy-thanos/0.log"
Jul 08 17:00:18.617086 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:18.616975    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_bd303057-abe0-43bc-a865-5e735e00ba8d/init-config-reloader/0.log"
Jul 08 17:00:18.651723 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:18.651692    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-x927t_6eae6020-c953-45de-a2bb-32e708038bcb/prometheus-operator/0.log"
Jul 08 17:00:18.681409 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:18.681376    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-x927t_6eae6020-c953-45de-a2bb-32e708038bcb/kube-rbac-proxy/0.log"
Jul 08 17:00:18.714007 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:18.713964    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-wlph5_cd61eaed-e188-4eaa-88b6-387b5333f6c9/prometheus-operator-admission-webhook/0.log"
Jul 08 17:00:18.756461 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:18.756431    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8df798f9d-kkfjw_9c773b06-163f-4e08-874e-a607cdf4cb3a/telemeter-client/0.log"
Jul 08 17:00:18.787083 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:18.787049    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8df798f9d-kkfjw_9c773b06-163f-4e08-874e-a607cdf4cb3a/reload/0.log"
Jul 08 17:00:18.812809 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:18.812779    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8df798f9d-kkfjw_9c773b06-163f-4e08-874e-a607cdf4cb3a/kube-rbac-proxy/0.log"
Jul 08 17:00:18.844765 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:18.844740    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-89b9f5c4c-4nd2s_7cdd4d5f-4ac1-4dc0-8839-47f474392834/thanos-query/0.log"
Jul 08 17:00:18.869728 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:18.869646    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-89b9f5c4c-4nd2s_7cdd4d5f-4ac1-4dc0-8839-47f474392834/kube-rbac-proxy-web/0.log"
Jul 08 17:00:18.894727 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:18.894693    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-89b9f5c4c-4nd2s_7cdd4d5f-4ac1-4dc0-8839-47f474392834/kube-rbac-proxy/0.log"
Jul 08 17:00:18.918993 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:18.918964    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-89b9f5c4c-4nd2s_7cdd4d5f-4ac1-4dc0-8839-47f474392834/prom-label-proxy/0.log"
Jul 08 17:00:18.943817 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:18.943785    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-89b9f5c4c-4nd2s_7cdd4d5f-4ac1-4dc0-8839-47f474392834/kube-rbac-proxy-rules/0.log"
Jul 08 17:00:18.970970 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:18.970927    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-89b9f5c4c-4nd2s_7cdd4d5f-4ac1-4dc0-8839-47f474392834/kube-rbac-proxy-metrics/0.log"
Jul 08 17:00:22.407862 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:22.407829    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-72rhc/perf-node-gather-daemonset-hhsq7"]
Jul 08 17:00:22.408898 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:22.408873    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e78d03d-0506-4503-b34e-dbb6d40c1d45" containerName="gather"
Jul 08 17:00:22.409054 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:22.409043    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e78d03d-0506-4503-b34e-dbb6d40c1d45" containerName="gather"
Jul 08 17:00:22.409172 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:22.409156    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e78d03d-0506-4503-b34e-dbb6d40c1d45" containerName="copy"
Jul 08 17:00:22.409172 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:22.409171    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e78d03d-0506-4503-b34e-dbb6d40c1d45" containerName="copy"
Jul 08 17:00:22.409334 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:22.409299    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e78d03d-0506-4503-b34e-dbb6d40c1d45" containerName="copy"
Jul 08 17:00:22.409405 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:22.409339    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e78d03d-0506-4503-b34e-dbb6d40c1d45" containerName="gather"
Jul 08 17:00:22.415024 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:22.415004    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-72rhc/perf-node-gather-daemonset-hhsq7"
Jul 08 17:00:22.421097 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:22.421071    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-72rhc/perf-node-gather-daemonset-hhsq7"]
Jul 08 17:00:22.493569 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:22.493526    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/251c4033-051d-4e48-be01-d3247c66e984-podres\") pod \"perf-node-gather-daemonset-hhsq7\" (UID: \"251c4033-051d-4e48-be01-d3247c66e984\") " pod="openshift-must-gather-72rhc/perf-node-gather-daemonset-hhsq7"
Jul 08 17:00:22.493799 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:22.493650    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb5wm\" (UniqueName: \"kubernetes.io/projected/251c4033-051d-4e48-be01-d3247c66e984-kube-api-access-kb5wm\") pod \"perf-node-gather-daemonset-hhsq7\" (UID: \"251c4033-051d-4e48-be01-d3247c66e984\") " pod="openshift-must-gather-72rhc/perf-node-gather-daemonset-hhsq7"
Jul 08 17:00:22.493799 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:22.493712    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/251c4033-051d-4e48-be01-d3247c66e984-proc\") pod \"perf-node-gather-daemonset-hhsq7\" (UID: \"251c4033-051d-4e48-be01-d3247c66e984\") " pod="openshift-must-gather-72rhc/perf-node-gather-daemonset-hhsq7"
Jul 08 17:00:22.493925 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:22.493798    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/251c4033-051d-4e48-be01-d3247c66e984-lib-modules\") pod \"perf-node-gather-daemonset-hhsq7\" (UID: \"251c4033-051d-4e48-be01-d3247c66e984\") " pod="openshift-must-gather-72rhc/perf-node-gather-daemonset-hhsq7"
Jul 08 17:00:22.493925 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:22.493861    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/251c4033-051d-4e48-be01-d3247c66e984-sys\") pod \"perf-node-gather-daemonset-hhsq7\" (UID: \"251c4033-051d-4e48-be01-d3247c66e984\") " pod="openshift-must-gather-72rhc/perf-node-gather-daemonset-hhsq7"
Jul 08 17:00:22.594640 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:22.594602    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/251c4033-051d-4e48-be01-d3247c66e984-sys\") pod \"perf-node-gather-daemonset-hhsq7\" (UID: \"251c4033-051d-4e48-be01-d3247c66e984\") " pod="openshift-must-gather-72rhc/perf-node-gather-daemonset-hhsq7"
Jul 08 17:00:22.594842 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:22.594809    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/251c4033-051d-4e48-be01-d3247c66e984-sys\") pod \"perf-node-gather-daemonset-hhsq7\" (UID: \"251c4033-051d-4e48-be01-d3247c66e984\") " pod="openshift-must-gather-72rhc/perf-node-gather-daemonset-hhsq7"
Jul 08 17:00:22.594842 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:22.594819    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/251c4033-051d-4e48-be01-d3247c66e984-podres\") pod \"perf-node-gather-daemonset-hhsq7\" (UID: \"251c4033-051d-4e48-be01-d3247c66e984\") " pod="openshift-must-gather-72rhc/perf-node-gather-daemonset-hhsq7"
Jul 08 17:00:22.594967 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:22.594906    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb5wm\" (UniqueName: \"kubernetes.io/projected/251c4033-051d-4e48-be01-d3247c66e984-kube-api-access-kb5wm\") pod \"perf-node-gather-daemonset-hhsq7\" (UID: \"251c4033-051d-4e48-be01-d3247c66e984\") " pod="openshift-must-gather-72rhc/perf-node-gather-daemonset-hhsq7"
Jul 08 17:00:22.594967 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:22.594929    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/251c4033-051d-4e48-be01-d3247c66e984-proc\") pod \"perf-node-gather-daemonset-hhsq7\" (UID: \"251c4033-051d-4e48-be01-d3247c66e984\") " pod="openshift-must-gather-72rhc/perf-node-gather-daemonset-hhsq7"
Jul 08 17:00:22.594967 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:22.594947    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/251c4033-051d-4e48-be01-d3247c66e984-lib-modules\") pod \"perf-node-gather-daemonset-hhsq7\" (UID: \"251c4033-051d-4e48-be01-d3247c66e984\") " pod="openshift-must-gather-72rhc/perf-node-gather-daemonset-hhsq7"
Jul 08 17:00:22.595119 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:22.594945    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/251c4033-051d-4e48-be01-d3247c66e984-podres\") pod \"perf-node-gather-daemonset-hhsq7\" (UID: \"251c4033-051d-4e48-be01-d3247c66e984\") " pod="openshift-must-gather-72rhc/perf-node-gather-daemonset-hhsq7"
Jul 08 17:00:22.595119 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:22.594991    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/251c4033-051d-4e48-be01-d3247c66e984-proc\") pod \"perf-node-gather-daemonset-hhsq7\" (UID: \"251c4033-051d-4e48-be01-d3247c66e984\") " pod="openshift-must-gather-72rhc/perf-node-gather-daemonset-hhsq7"
Jul 08 17:00:22.595119 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:22.595061    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/251c4033-051d-4e48-be01-d3247c66e984-lib-modules\") pod \"perf-node-gather-daemonset-hhsq7\" (UID: \"251c4033-051d-4e48-be01-d3247c66e984\") " pod="openshift-must-gather-72rhc/perf-node-gather-daemonset-hhsq7"
Jul 08 17:00:22.604511 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:22.604479    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb5wm\" (UniqueName: \"kubernetes.io/projected/251c4033-051d-4e48-be01-d3247c66e984-kube-api-access-kb5wm\") pod \"perf-node-gather-daemonset-hhsq7\" (UID: \"251c4033-051d-4e48-be01-d3247c66e984\") " pod="openshift-must-gather-72rhc/perf-node-gather-daemonset-hhsq7"
Jul 08 17:00:22.728751 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:22.728705    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-72rhc/perf-node-gather-daemonset-hhsq7"
Jul 08 17:00:22.903054 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:22.903028    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-72rhc/perf-node-gather-daemonset-hhsq7"]
Jul 08 17:00:22.905534 ip-10-0-139-237 kubenswrapper[2582]: W0708 17:00:22.905492    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod251c4033_051d_4e48_be01_d3247c66e984.slice/crio-214c32cd28f67646206fe3d9477495d95575623d553e2eca6e9030841ea197c4 WatchSource:0}: Error finding container 214c32cd28f67646206fe3d9477495d95575623d553e2eca6e9030841ea197c4: Status 404 returned error can't find the container with id 214c32cd28f67646206fe3d9477495d95575623d553e2eca6e9030841ea197c4
Jul 08 17:00:23.000467 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:23.000431    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tpllq_d9c7d9d9-4854-413e-8eec-d3608f5333f1/dns/0.log"
Jul 08 17:00:23.026001 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:23.025972    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tpllq_d9c7d9d9-4854-413e-8eec-d3608f5333f1/kube-rbac-proxy/0.log"
Jul 08 17:00:23.078924 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:23.078896    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pq4sj_ecc650f2-27aa-4300-b61a-8328a41d599d/dns-node-resolver/0.log"
Jul 08 17:00:23.415080 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:23.415049    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-72rhc/perf-node-gather-daemonset-hhsq7" event={"ID":"251c4033-051d-4e48-be01-d3247c66e984","Type":"ContainerStarted","Data":"58d5694606f04909966c891233fcda4c7a88e483dd3afe58dbe24a844956ef5e"}
Jul 08 17:00:23.415566 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:23.415107    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-72rhc/perf-node-gather-daemonset-hhsq7" event={"ID":"251c4033-051d-4e48-be01-d3247c66e984","Type":"ContainerStarted","Data":"214c32cd28f67646206fe3d9477495d95575623d553e2eca6e9030841ea197c4"}
Jul 08 17:00:23.415566 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:23.415182    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-72rhc/perf-node-gather-daemonset-hhsq7"
Jul 08 17:00:23.430952 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:23.430910    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-72rhc/perf-node-gather-daemonset-hhsq7" podStartSLOduration=1.430893599 podStartE2EDuration="1.430893599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:00:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:00:23.430005489 +0000 UTC m=+1542.770970568" watchObservedRunningTime="2026-07-08 17:00:23.430893599 +0000 UTC m=+1542.771858677"
Jul 08 17:00:23.660688 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:23.660659    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bpx78_d45b9a7f-3c63-4f52-863f-5d128fd07d93/node-ca/0.log"
Jul 08 17:00:24.932111 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:24.932082    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wcgpv_9f68ab1b-b463-44bb-87f3-1341b827b5ec/serve-healthcheck-canary/0.log"
Jul 08 17:00:25.487144 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:25.487118    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w69bj_f96e3712-16d4-4a1d-9d73-1b3214dda98c/kube-rbac-proxy/0.log"
Jul 08 17:00:25.510563 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:25.510538    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w69bj_f96e3712-16d4-4a1d-9d73-1b3214dda98c/exporter/0.log"
Jul 08 17:00:25.535027 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:25.534993    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w69bj_f96e3712-16d4-4a1d-9d73-1b3214dda98c/extractor/0.log"
Jul 08 17:00:27.829482 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:27.829458    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-8jv9l_e805780f-a7fa-4a63-83b7-a6b60acaa1dd/s3-init/0.log"
Jul 08 17:00:27.847534 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:27.847508    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-8jv9l_e805780f-a7fa-4a63-83b7-a6b60acaa1dd/download-hf-model/0.log"
Jul 08 17:00:27.882047 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:27.882021    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-tls-init-custom-srvgm_0d190cf7-84ec-4cad-8a3c-3e65a78b48d8/s3-tls-init-custom/0.log"
Jul 08 17:00:27.901814 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:27.901792    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-tls-init-custom-srvgm_0d190cf7-84ec-4cad-8a3c-3e65a78b48d8/download-hf-model/0.log"
Jul 08 17:00:27.933439 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:27.933412    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-tls-init-serving-dq8rv_4aeec037-1338-42d6-8f4e-cb80fa564a3a/s3-tls-init-serving/0.log"
Jul 08 17:00:27.953841 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:27.953819    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-tls-init-serving-dq8rv_4aeec037-1338-42d6-8f4e-cb80fa564a3a/download-hf-model/0.log"
Jul 08 17:00:27.985016 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:27.984994    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-64568bcd49-tn4bw_61a8704c-1d52-4394-b894-22384e72201a/seaweedfs/0.log"
Jul 08 17:00:28.040523 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:28.040493    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-tls-serving-67b7477c6d-n6q6c_4c49f2e6-1f07-482a-a7c6-3124391420df/seaweedfs-tls-serving/0.log"
Jul 08 17:00:29.431669 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:29.431643    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-72rhc/perf-node-gather-daemonset-hhsq7"
Jul 08 17:00:34.156222 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:34.156192    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mw6h7_d9ddf360-c5ae-4ef7-a947-59405bf4e8db/kube-multus-additional-cni-plugins/0.log"
Jul 08 17:00:34.182982 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:34.182907    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mw6h7_d9ddf360-c5ae-4ef7-a947-59405bf4e8db/egress-router-binary-copy/0.log"
Jul 08 17:00:34.207876 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:34.207852    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mw6h7_d9ddf360-c5ae-4ef7-a947-59405bf4e8db/cni-plugins/0.log"
Jul 08 17:00:34.235121 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:34.235094    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mw6h7_d9ddf360-c5ae-4ef7-a947-59405bf4e8db/bond-cni-plugin/0.log"
Jul 08 17:00:34.260784 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:34.260758    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mw6h7_d9ddf360-c5ae-4ef7-a947-59405bf4e8db/routeoverride-cni/0.log"
Jul 08 17:00:34.286176 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:34.286140    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mw6h7_d9ddf360-c5ae-4ef7-a947-59405bf4e8db/whereabouts-cni-bincopy/0.log"
Jul 08 17:00:34.312891 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:34.312856    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mw6h7_d9ddf360-c5ae-4ef7-a947-59405bf4e8db/whereabouts-cni/0.log"
Jul 08 17:00:34.346818 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:34.346770    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-jgp2d_a2b65945-693a-42f8-9955-f484a3f96d10/kube-multus/0.log"
Jul 08 17:00:34.493972 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:34.493871    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-z82nm_2796ef11-ffb4-4bff-b73b-38d727eb45f2/network-metrics-daemon/0.log"
Jul 08 17:00:34.522075 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:34.522045    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-z82nm_2796ef11-ffb4-4bff-b73b-38d727eb45f2/kube-rbac-proxy/0.log"
Jul 08 17:00:35.382182 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:35.382153    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kvhpr_19df7751-4b62-4ae5-92a6-0bbda11fec85/ovn-controller/0.log"
Jul 08 17:00:35.463605 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:35.463567    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kvhpr_19df7751-4b62-4ae5-92a6-0bbda11fec85/ovn-acl-logging/0.log"
Jul 08 17:00:35.468629 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:35.468610    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kvhpr_19df7751-4b62-4ae5-92a6-0bbda11fec85/ovn-acl-logging/1.log"
Jul 08 17:00:35.516559 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:35.516536    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kvhpr_19df7751-4b62-4ae5-92a6-0bbda11fec85/kube-rbac-proxy-node/0.log"
Jul 08 17:00:35.551182 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:35.551142    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kvhpr_19df7751-4b62-4ae5-92a6-0bbda11fec85/kube-rbac-proxy-ovn-metrics/0.log"
Jul 08 17:00:35.580296 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:35.580247    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kvhpr_19df7751-4b62-4ae5-92a6-0bbda11fec85/northd/0.log"
Jul 08 17:00:35.603020 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:35.602996    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kvhpr_19df7751-4b62-4ae5-92a6-0bbda11fec85/nbdb/0.log"
Jul 08 17:00:35.636457 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:35.636393    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kvhpr_19df7751-4b62-4ae5-92a6-0bbda11fec85/sbdb/0.log"
Jul 08 17:00:35.694697 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:35.694672    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kvhpr_19df7751-4b62-4ae5-92a6-0bbda11fec85/ovnkube-controller/0.log"
Jul 08 17:00:37.327218 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:37.327194    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-bkqjt_c98cc144-3322-4f83-b571-9772b5a63d49/network-check-target-container/0.log"
Jul 08 17:00:38.306513 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:38.306483    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-8hh26_a12a9aeb-4a05-41b1-9f81-dc6dc476fda8/iptables-alerter/0.log"
Jul 08 17:00:39.027271 ip-10-0-139-237 kubenswrapper[2582]: I0708 17:00:39.027194    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-p46xh_782cdc3e-e8c4-4d37-a22e-832fac199082/tuned/0.log"