Sep 16 18:44:45.838040 ip-10-0-130-26 systemd[1]: Starting Kubernetes Kubelet...
Sep 16 18:44:46.235402 ip-10-0-130-26 kubenswrapper[2557]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 18:44:46.235402 ip-10-0-130-26 kubenswrapper[2557]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 16 18:44:46.235402 ip-10-0-130-26 kubenswrapper[2557]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 18:44:46.235402 ip-10-0-130-26 kubenswrapper[2557]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 16 18:44:46.235402 ip-10-0-130-26 kubenswrapper[2557]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 18:44:46.237416 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.237326    2557 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 16 18:44:46.243012 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.242997    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:44:46.243012 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243012    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:44:46.243074 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243016    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:44:46.243074 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243020    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:44:46.243074 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243023    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:44:46.243074 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243026    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:44:46.243074 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243029    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:44:46.243074 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243032    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:44:46.243074 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243035    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:44:46.243074 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243037    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:44:46.243074 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243041    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:44:46.243074 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243043    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:44:46.243074 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243046    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:44:46.243074 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243048    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:44:46.243074 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243051    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:44:46.243074 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243054    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:44:46.243074 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243057    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:44:46.243074 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243063    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:44:46.243074 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243066    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:44:46.243074 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243069    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:44:46.243074 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243071    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:44:46.243074 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243074    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:44:46.243557 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243077    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:44:46.243557 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243080    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:44:46.243557 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243082    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:44:46.243557 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243085    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:44:46.243557 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243088    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:44:46.243557 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243091    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:44:46.243557 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243093    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:44:46.243557 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243096    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:44:46.243557 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243099    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:44:46.243557 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243102    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:44:46.243557 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243104    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:44:46.243557 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243109    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:44:46.243557 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243112    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:44:46.243557 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243115    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:44:46.243557 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243118    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:44:46.243557 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243120    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:44:46.243557 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243123    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:44:46.243557 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243125    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:44:46.243557 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243128    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:44:46.244169 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243130    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:44:46.244169 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243133    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:44:46.244169 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243135    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:44:46.244169 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243138    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:44:46.244169 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243140    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:44:46.244169 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243143    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:44:46.244169 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243145    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:44:46.244169 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243148    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:44:46.244169 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243150    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:44:46.244169 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243153    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:44:46.244169 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243155    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:44:46.244169 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243158    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:44:46.244169 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243160    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:44:46.244169 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243163    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:44:46.244169 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243166    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:44:46.244169 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243169    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:44:46.244169 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243171    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:44:46.244169 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243174    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:44:46.244169 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243176    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:44:46.244169 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243179    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:44:46.244986 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243182    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:44:46.244986 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243185    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:44:46.244986 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243188    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:44:46.244986 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243190    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:44:46.244986 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243193    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:44:46.244986 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243196    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:44:46.244986 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243198    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:44:46.244986 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243201    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:44:46.244986 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243205    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:44:46.244986 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243208    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:44:46.244986 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243211    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:44:46.244986 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243214    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:44:46.244986 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243217    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:44:46.244986 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243220    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:44:46.244986 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243223    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:44:46.244986 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243227    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:44:46.244986 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243230    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:44:46.244986 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243233    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:44:46.244986 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243236    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:44:46.245452 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243239    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:44:46.245452 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243242    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:44:46.245452 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243245    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:44:46.245452 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243247    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:44:46.245452 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243250    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:44:46.245452 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243253    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:44:46.245452 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243256    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:44:46.245452 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243260    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:44:46.245452 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243262    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:44:46.245452 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243265    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:44:46.245452 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243267    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:44:46.245452 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243270    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:44:46.245452 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243273    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:44:46.245452 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243275    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:44:46.245452 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243278    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:44:46.245452 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243280    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:44:46.245452 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243283    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:44:46.245452 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243285    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:44:46.245452 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243288    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:44:46.245452 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243983    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:44:46.245949 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243991    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:44:46.245949 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243995    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:44:46.245949 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.243997    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:44:46.245949 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244000    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:44:46.245949 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244003    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:44:46.245949 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244006    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:44:46.245949 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244009    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:44:46.245949 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244011    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:44:46.245949 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244014    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:44:46.245949 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244016    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:44:46.245949 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244019    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:44:46.245949 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244022    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:44:46.245949 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244024    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:44:46.245949 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244027    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:44:46.245949 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244030    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:44:46.245949 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244032    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:44:46.245949 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244035    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:44:46.245949 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244038    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:44:46.245949 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244040    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:44:46.245949 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244043    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:44:46.246456 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244046    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:44:46.246456 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244049    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:44:46.246456 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244051    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:44:46.246456 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244053    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:44:46.246456 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244056    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:44:46.246456 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244059    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:44:46.246456 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244062    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:44:46.246456 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244064    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:44:46.246456 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244066    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:44:46.246456 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244069    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:44:46.246456 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244072    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:44:46.246456 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244076    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:44:46.246456 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244080    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:44:46.246456 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244083    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:44:46.246456 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244085    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:44:46.246456 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244088    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:44:46.246456 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244091    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:44:46.246456 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244093    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:44:46.246456 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244096    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:44:46.246943 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244099    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:44:46.246943 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244101    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:44:46.246943 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244103    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:44:46.246943 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244106    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:44:46.246943 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244109    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:44:46.246943 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244112    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:44:46.246943 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244116    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:44:46.246943 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244119    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:44:46.246943 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244122    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:44:46.246943 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244125    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:44:46.246943 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244127    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:44:46.246943 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244130    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:44:46.246943 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244134    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:44:46.246943 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244136    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:44:46.246943 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244139    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:44:46.246943 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244142    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:44:46.246943 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244144    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:44:46.246943 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244147    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:44:46.246943 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244149    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:44:46.246943 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244152    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:44:46.247444 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244155    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:44:46.247444 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244158    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:44:46.247444 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244160    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:44:46.247444 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244164    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:44:46.247444 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244168    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:44:46.247444 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244171    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:44:46.247444 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244175    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:44:46.247444 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244180    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:44:46.247444 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244183    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:44:46.247444 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244186    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:44:46.247444 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244188    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:44:46.247444 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244191    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:44:46.247444 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244194    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:44:46.247444 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244197    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:44:46.247444 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244199    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:44:46.247444 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244202    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:44:46.247444 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244205    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:44:46.247444 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244208    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:44:46.247898 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244211    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:44:46.247898 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244214    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:44:46.247898 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244216    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:44:46.247898 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244219    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:44:46.247898 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244222    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:44:46.247898 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244224    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:44:46.247898 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244227    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:44:46.247898 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244230    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:44:46.247898 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244232    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:44:46.247898 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244235    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:44:46.247898 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244237    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:44:46.247898 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244240    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:44:46.247898 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244242    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:44:46.247898 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244245    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:44:46.247898 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244247    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:44:46.247898 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244250    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:44:46.247898 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244252    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:44:46.247898 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244255    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:44:46.247898 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244257    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:44:46.247898 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244260    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:44:46.248386 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.244263    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:44:46.248386 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245323    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:44:46.248386 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245338    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:44:46.248386 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245343    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:44:46.248386 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245358    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:44:46.248386 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245365    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:44:46.248386 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245370    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:44:46.248386 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245375    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:44:46.248386 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245380    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:44:46.248386 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245384    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:44:46.248386 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245388    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:44:46.248386 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245391    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:44:46.248386 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245396    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:44:46.248386 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245400    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:44:46.248386 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245404    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:44:46.248386 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245408    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:44:46.248386 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245415    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:44:46.248386 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245427    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:44:46.248386 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245432    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:44:46.248886 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245436    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:44:46.248886 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245440    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:44:46.248886 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245444    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:44:46.248886 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245448    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:44:46.248886 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245453    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:44:46.248886 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245458    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:44:46.248886 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245462    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:44:46.248886 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245466    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:44:46.248886 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245470    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:44:46.248886 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245475    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:44:46.248886 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245484    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:44:46.248886 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245488    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:44:46.248886 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245492    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:44:46.248886 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245496    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:44:46.248886 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245500    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:44:46.248886 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245506    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:44:46.248886 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245510    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:44:46.248886 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245515    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:44:46.248886 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245519    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:44:46.248886 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245523    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:44:46.249374 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245528    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:44:46.249374 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245532    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:44:46.249374 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245536    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:44:46.249374 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245545    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:44:46.249374 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245549    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:44:46.249374 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245553    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:44:46.249374 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245558    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:44:46.249374 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245562    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:44:46.249374 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245569    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:44:46.249374 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245573    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:44:46.249374 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245577    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:44:46.249374 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245582    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:44:46.249374 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245586    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:44:46.249374 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245590    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:44:46.249374 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245595    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:44:46.249374 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245604    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:44:46.249374 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245608    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:44:46.249374 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245612    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:44:46.249374 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245616    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:44:46.249374 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245621    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:44:46.249887 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245625    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:44:46.249887 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245629    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:44:46.249887 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245635    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:44:46.249887 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245640    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:44:46.249887 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245646    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:44:46.249887 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245650    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:44:46.249887 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245655    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:44:46.249887 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245665    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:44:46.249887 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245671    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:44:46.249887 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245676    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:44:46.249887 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245681    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:44:46.249887 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245685    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:44:46.249887 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245689    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:44:46.249887 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245693    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:44:46.249887 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245697    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:44:46.249887 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245704    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:44:46.249887 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245708    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:44:46.249887 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245713    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:44:46.249887 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245721    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:44:46.250379 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245726    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:44:46.250379 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245730    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:44:46.250379 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245734    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:44:46.250379 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245738    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:44:46.250379 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245742    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:44:46.250379 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245746    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:44:46.250379 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245750    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:44:46.250379 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245754    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:44:46.250379 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245758    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:44:46.250379 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245762    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:44:46.250379 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245766    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:44:46.250379 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245786    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:44:46.250379 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245796    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:44:46.250379 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245800    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:44:46.250379 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245804    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:44:46.250379 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245808    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:44:46.250379 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245813    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:44:46.250379 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245817    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:44:46.250379 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245822    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:44:46.250379 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245825    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:44:46.250892 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245834    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:44:46.250892 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.245838    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:44:46.250892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246040    2557 flags.go:64] FLAG: --address="0.0.0.0"
Sep 16 18:44:46.250892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246248    2557 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 16 18:44:46.250892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246261    2557 flags.go:64] FLAG: --anonymous-auth="true"
Sep 16 18:44:46.250892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246266    2557 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 16 18:44:46.250892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246270    2557 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 16 18:44:46.250892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246273    2557 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 16 18:44:46.250892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246278    2557 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 16 18:44:46.250892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246282    2557 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 16 18:44:46.250892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246286    2557 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 16 18:44:46.250892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246289    2557 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 16 18:44:46.250892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246293    2557 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 16 18:44:46.250892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246296    2557 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 16 18:44:46.250892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246300    2557 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 16 18:44:46.250892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246303    2557 flags.go:64] FLAG: --cgroup-root=""
Sep 16 18:44:46.250892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246306    2557 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 16 18:44:46.250892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246309    2557 flags.go:64] FLAG: --client-ca-file=""
Sep 16 18:44:46.250892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246312    2557 flags.go:64] FLAG: --cloud-provider="external"
Sep 16 18:44:46.250892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246315    2557 flags.go:64] FLAG: --cluster-dns="[]"
Sep 16 18:44:46.250892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246320    2557 flags.go:64] FLAG: --cluster-domain=""
Sep 16 18:44:46.250892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246323    2557 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 16 18:44:46.250892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246326    2557 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 16 18:44:46.250892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246329    2557 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 16 18:44:46.251474 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246333    2557 flags.go:64] FLAG: --container-log-max-files="5"
Sep 16 18:44:46.251474 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246337    2557 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 16 18:44:46.251474 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246340    2557 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 16 18:44:46.251474 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246343    2557 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 16 18:44:46.251474 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246347    2557 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 16 18:44:46.251474 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246350    2557 flags.go:64] FLAG: --contention-profiling="false"
Sep 16 18:44:46.251474 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246353    2557 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 16 18:44:46.251474 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246356    2557 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 16 18:44:46.251474 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246359    2557 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 16 18:44:46.251474 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246362    2557 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 16 18:44:46.251474 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246367    2557 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 16 18:44:46.251474 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246371    2557 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 16 18:44:46.251474 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246374    2557 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 16 18:44:46.251474 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246377    2557 flags.go:64] FLAG: --enable-load-reader="false"
Sep 16 18:44:46.251474 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246380    2557 flags.go:64] FLAG: --enable-server="true"
Sep 16 18:44:46.251474 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246383    2557 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 16 18:44:46.251474 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246389    2557 flags.go:64] FLAG: --event-burst="100"
Sep 16 18:44:46.251474 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246392    2557 flags.go:64] FLAG: --event-qps="50"
Sep 16 18:44:46.251474 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246395    2557 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 16 18:44:46.251474 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246398    2557 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 16 18:44:46.251474 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246401    2557 flags.go:64] FLAG: --eviction-hard=""
Sep 16 18:44:46.251474 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246405    2557 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 16 18:44:46.251474 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246408    2557 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 16 18:44:46.251474 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246411    2557 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 16 18:44:46.251474 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246414    2557 flags.go:64] FLAG: --eviction-soft=""
Sep 16 18:44:46.252115 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246417    2557 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 16 18:44:46.252115 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246420    2557 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 16 18:44:46.252115 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246423    2557 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 16 18:44:46.252115 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246426    2557 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 16 18:44:46.252115 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246430    2557 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 16 18:44:46.252115 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246433    2557 flags.go:64] FLAG: --fail-swap-on="true"
Sep 16 18:44:46.252115 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246436    2557 flags.go:64] FLAG: --feature-gates=""
Sep 16 18:44:46.252115 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246440    2557 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 16 18:44:46.252115 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246443    2557 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 16 18:44:46.252115 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246446    2557 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 16 18:44:46.252115 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246449    2557 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 16 18:44:46.252115 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246452    2557 flags.go:64] FLAG: --healthz-port="10248"
Sep 16 18:44:46.252115 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246455    2557 flags.go:64] FLAG: --help="false"
Sep 16 18:44:46.252115 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246458    2557 flags.go:64] FLAG: --hostname-override="ip-10-0-130-26.ec2.internal"
Sep 16 18:44:46.252115 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246461    2557 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 16 18:44:46.252115 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246465    2557 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 16 18:44:46.252115 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246468    2557 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 16 18:44:46.252115 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246473    2557 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 16 18:44:46.252115 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246476    2557 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 16 18:44:46.252115 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246480    2557 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 16 18:44:46.252115 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246483    2557 flags.go:64] FLAG: --image-service-endpoint=""
Sep 16 18:44:46.252115 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246485    2557 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 16 18:44:46.252115 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246489    2557 flags.go:64] FLAG: --kube-api-burst="100"
Sep 16 18:44:46.252717 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246492    2557 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 16 18:44:46.252717 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246495    2557 flags.go:64] FLAG: --kube-api-qps="50"
Sep 16 18:44:46.252717 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246498    2557 flags.go:64] FLAG: --kube-reserved=""
Sep 16 18:44:46.252717 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246501    2557 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 16 18:44:46.252717 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246504    2557 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 16 18:44:46.252717 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246507    2557 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 16 18:44:46.252717 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246510    2557 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 16 18:44:46.252717 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246513    2557 flags.go:64] FLAG: --lock-file=""
Sep 16 18:44:46.252717 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246516    2557 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 16 18:44:46.252717 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246518    2557 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 16 18:44:46.252717 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246521    2557 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 16 18:44:46.252717 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246527    2557 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 16 18:44:46.252717 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246530    2557 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 16 18:44:46.252717 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246533    2557 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 16 18:44:46.252717 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246536    2557 flags.go:64] FLAG: --logging-format="text"
Sep 16 18:44:46.252717 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246540    2557 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 16 18:44:46.252717 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246543    2557 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 16 18:44:46.252717 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246546    2557 flags.go:64] FLAG: --manifest-url=""
Sep 16 18:44:46.252717 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246549    2557 flags.go:64] FLAG: --manifest-url-header=""
Sep 16 18:44:46.252717 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246554    2557 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 16 18:44:46.252717 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246557    2557 flags.go:64] FLAG: --max-open-files="1000000"
Sep 16 18:44:46.252717 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246561    2557 flags.go:64] FLAG: --max-pods="110"
Sep 16 18:44:46.252717 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246565    2557 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 16 18:44:46.252717 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246568    2557 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 16 18:44:46.252717 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246571    2557 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 16 18:44:46.253341 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246574    2557 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 16 18:44:46.253341 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246577    2557 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 16 18:44:46.253341 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246580    2557 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 16 18:44:46.253341 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246583    2557 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 16 18:44:46.253341 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246591    2557 flags.go:64] FLAG: --node-status-max-images="50"
Sep 16 18:44:46.253341 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246594    2557 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 16 18:44:46.253341 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246597    2557 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 16 18:44:46.253341 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246600    2557 flags.go:64] FLAG: --pod-cidr=""
Sep 16 18:44:46.253341 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246603    2557 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 16 18:44:46.253341 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246609    2557 flags.go:64] FLAG: --pod-manifest-path=""
Sep 16 18:44:46.253341 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246615    2557 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 16 18:44:46.253341 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246618    2557 flags.go:64] FLAG: --pods-per-core="0"
Sep 16 18:44:46.253341 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246621    2557 flags.go:64] FLAG: --port="10250"
Sep 16 18:44:46.253341 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246624    2557 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 16 18:44:46.253341 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246627    2557 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08838e7a6eb2050bd"
Sep 16 18:44:46.253341 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246630    2557 flags.go:64] FLAG: --qos-reserved=""
Sep 16 18:44:46.253341 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246633    2557 flags.go:64] FLAG: --read-only-port="10255"
Sep 16 18:44:46.253341 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246636    2557 flags.go:64] FLAG: --register-node="true"
Sep 16 18:44:46.253341 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246639    2557 flags.go:64] FLAG: --register-with-taints=""
Sep 16 18:44:46.253341 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246643    2557 flags.go:64] FLAG: --registry-burst="10"
Sep 16 18:44:46.253341 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246646    2557 flags.go:64] FLAG: --registry-qps="5"
Sep 16 18:44:46.253341 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246649    2557 flags.go:64] FLAG: --reserved-cpus=""
Sep 16 18:44:46.253341 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246652    2557 flags.go:64] FLAG: --reserved-memory=""
Sep 16 18:44:46.253341 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246658    2557 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 16 18:44:46.253341 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246661    2557 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 16 18:44:46.254039 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246664    2557 flags.go:64] FLAG: --rotate-certificates="false"
Sep 16 18:44:46.254039 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246667    2557 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 16 18:44:46.254039 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246670    2557 flags.go:64] FLAG: --runonce="false"
Sep 16 18:44:46.254039 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246673    2557 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 16 18:44:46.254039 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246677    2557 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 16 18:44:46.254039 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246680    2557 flags.go:64] FLAG: --seccomp-default="false"
Sep 16 18:44:46.254039 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246683    2557 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 16 18:44:46.254039 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246686    2557 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 16 18:44:46.254039 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246689    2557 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 16 18:44:46.254039 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246692    2557 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 16 18:44:46.254039 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246695    2557 flags.go:64] FLAG: --storage-driver-password="root"
Sep 16 18:44:46.254039 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246698    2557 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 16 18:44:46.254039 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246701    2557 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 16 18:44:46.254039 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246704    2557 flags.go:64] FLAG: --storage-driver-user="root"
Sep 16 18:44:46.254039 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246707    2557 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 16 18:44:46.254039 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246710    2557 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 16 18:44:46.254039 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246713    2557 flags.go:64] FLAG: --system-cgroups=""
Sep 16 18:44:46.254039 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246717    2557 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 16 18:44:46.254039 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246724    2557 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 16 18:44:46.254039 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246727    2557 flags.go:64] FLAG: --tls-cert-file=""
Sep 16 18:44:46.254039 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246730    2557 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 16 18:44:46.254039 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246733    2557 flags.go:64] FLAG: --tls-min-version=""
Sep 16 18:44:46.254039 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246736    2557 flags.go:64] FLAG: --tls-private-key-file=""
Sep 16 18:44:46.254039 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246739    2557 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 16 18:44:46.254039 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246742    2557 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 16 18:44:46.255094 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246745    2557 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 16 18:44:46.255094 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246748    2557 flags.go:64] FLAG: --v="2"
Sep 16 18:44:46.255094 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246752    2557 flags.go:64] FLAG: --version="false"
Sep 16 18:44:46.255094 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246756    2557 flags.go:64] FLAG: --vmodule=""
Sep 16 18:44:46.255094 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246760    2557 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 16 18:44:46.255094 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.246764    2557 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 16 18:44:46.255094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246892    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:44:46.255094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246897    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:44:46.255094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246900    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:44:46.255094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246904    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:44:46.255094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246907    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:44:46.255094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246910    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:44:46.255094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246913    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:44:46.255094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246916    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:44:46.255094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246918    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:44:46.255094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246921    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:44:46.255094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246923    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:44:46.255094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246926    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:44:46.255094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246928    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:44:46.255094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246931    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:44:46.255094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246933    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:44:46.256011 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246936    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:44:46.256011 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246938    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:44:46.256011 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246941    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:44:46.256011 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246945    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:44:46.256011 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246947    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:44:46.256011 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246950    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:44:46.256011 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246953    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:44:46.256011 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246956    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:44:46.256011 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246958    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:44:46.256011 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246961    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:44:46.256011 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246964    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:44:46.256011 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246966    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:44:46.256011 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246969    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:44:46.256011 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246971    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:44:46.256011 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246974    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:44:46.256011 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246977    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:44:46.256011 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246980    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:44:46.256011 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246982    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:44:46.256011 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246985    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:44:46.256011 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246987    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:44:46.256679 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246990    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:44:46.256679 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246993    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:44:46.256679 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246996    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:44:46.256679 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.246998    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:44:46.256679 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247001    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:44:46.256679 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247003    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:44:46.256679 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247006    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:44:46.256679 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247008    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:44:46.256679 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247011    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:44:46.256679 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247014    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:44:46.256679 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247020    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:44:46.256679 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247026    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:44:46.256679 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247029    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:44:46.256679 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247032    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:44:46.256679 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247035    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:44:46.256679 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247039    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:44:46.256679 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247042    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:44:46.256679 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247045    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:44:46.256679 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247047    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:44:46.257268 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247050    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:44:46.257268 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247053    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:44:46.257268 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247056    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:44:46.257268 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247059    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:44:46.257268 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247061    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:44:46.257268 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247064    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:44:46.257268 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247066    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:44:46.257268 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247069    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:44:46.257268 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247071    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:44:46.257268 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247075    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:44:46.257268 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247080    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:44:46.257268 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247084    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:44:46.257268 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247087    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:44:46.257268 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247093    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:44:46.257268 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247095    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:44:46.257268 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247098    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:44:46.257268 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247101    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:44:46.257268 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247104    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:44:46.257268 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247106    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:44:46.257268 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247109    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:44:46.258145 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247111    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:44:46.258145 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247114    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:44:46.258145 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247117    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:44:46.258145 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247119    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:44:46.258145 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247122    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:44:46.258145 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247125    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:44:46.258145 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247127    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:44:46.258145 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247130    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:44:46.258145 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247133    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:44:46.258145 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247135    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:44:46.258145 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247138    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:44:46.258145 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247140    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:44:46.258145 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247143    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:44:46.258145 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247147    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:44:46.258145 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247151    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:44:46.258145 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247154    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:44:46.258145 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247157    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:44:46.258145 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247159    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:44:46.258145 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247162    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:44:46.258913 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247164    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:44:46.258913 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247167    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:44:46.258913 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247170    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:44:46.258913 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247174    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:44:46.258913 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247178    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:44:46.258913 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.247181    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:44:46.258913 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.247889    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 18:44:46.258913 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.256311    2557 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 16 18:44:46.258913 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.256331    2557 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 16 18:44:46.258913 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.257252    2557 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 16 18:44:46.258913 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.257267    2557 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 16 18:44:46.258913 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257371    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:44:46.258913 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257381    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:44:46.258913 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257387    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:44:46.259258 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257391    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:44:46.259258 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257396    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:44:46.259258 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257400    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:44:46.259258 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257405    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:44:46.259258 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257409    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:44:46.259258 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257413    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:44:46.259258 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257418    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:44:46.259258 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257422    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:44:46.259258 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257429    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:44:46.259258 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257435    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:44:46.259258 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257440    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:44:46.259258 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257445    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:44:46.259258 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257449    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:44:46.259258 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257453    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:44:46.259258 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257457    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:44:46.259258 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257462    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:44:46.259258 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257466    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:44:46.259258 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257470    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:44:46.259258 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257475    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:44:46.259934 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257479    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:44:46.259934 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257483    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:44:46.259934 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257487    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:44:46.259934 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257491    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:44:46.259934 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257495    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:44:46.259934 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257499    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:44:46.259934 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257504    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:44:46.259934 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257510    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:44:46.259934 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257517    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:44:46.259934 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257523    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:44:46.259934 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257529    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:44:46.259934 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257534    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:44:46.259934 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257539    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:44:46.259934 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257544    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:44:46.259934 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257550    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:44:46.259934 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257554    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:44:46.259934 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257559    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:44:46.259934 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257563    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:44:46.259934 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257567    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:44:46.260600 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257571    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:44:46.260600 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257576    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:44:46.260600 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257580    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:44:46.260600 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257584    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:44:46.260600 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257588    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:44:46.260600 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257592    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:44:46.260600 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257596    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:44:46.260600 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257601    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:44:46.260600 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257605    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:44:46.260600 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257609    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:44:46.260600 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257613    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:44:46.260600 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257617    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:44:46.260600 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257622    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:44:46.260600 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257626    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:44:46.260600 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257630    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:44:46.260600 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257634    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:44:46.260600 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257638    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:44:46.260600 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257642    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:44:46.260600 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257647    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:44:46.261109 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257651    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:44:46.261109 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257656    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:44:46.261109 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257661    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:44:46.261109 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257666    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:44:46.261109 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257670    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:44:46.261109 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257674    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:44:46.261109 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257679    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:44:46.261109 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257683    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:44:46.261109 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257687    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:44:46.261109 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257691    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:44:46.261109 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257695    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:44:46.261109 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257699    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:44:46.261109 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257703    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:44:46.261109 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257707    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:44:46.261109 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257712    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:44:46.261109 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257717    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:44:46.261109 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257721    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:44:46.261109 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257725    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:44:46.261109 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257729    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:44:46.261109 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257734    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:44:46.261611 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257738    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:44:46.261611 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257742    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:44:46.261611 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257746    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:44:46.261611 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257750    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:44:46.261611 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257754    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:44:46.261611 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257759    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:44:46.261611 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257763    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:44:46.261611 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257783    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:44:46.261611 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257788    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:44:46.261611 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257792    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:44:46.261611 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257796    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:44:46.261611 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257801    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:44:46.261611 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257805    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:44:46.261611 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257809    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:44:46.261611 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257813    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:44:46.261611 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257822    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:44:46.261611 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257827    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:44:46.261611 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257832    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:44:46.261611 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.257837    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:44:46.262125 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.257845    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 18:44:46.262125 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258046    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:44:46.262125 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258055    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:44:46.262125 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258060    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:44:46.262125 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258065    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:44:46.262125 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258071    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:44:46.262125 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258075    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:44:46.262125 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258080    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:44:46.262125 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258084    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:44:46.262125 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258088    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:44:46.262125 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258092    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:44:46.262125 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258096    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:44:46.262125 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258103    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:44:46.262125 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258109    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:44:46.262125 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258114    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:44:46.262518 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258118    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:44:46.262518 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258123    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:44:46.262518 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258128    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:44:46.262518 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258132    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:44:46.262518 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258136    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:44:46.262518 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258141    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:44:46.262518 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258145    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:44:46.262518 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258149    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:44:46.262518 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258153    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:44:46.262518 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258157    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:44:46.262518 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258161    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:44:46.262518 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258165    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:44:46.262518 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258170    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:44:46.262518 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258174    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:44:46.262518 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258179    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:44:46.262518 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258184    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:44:46.262518 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258188    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:44:46.262518 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258192    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:44:46.262518 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258197    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:44:46.262518 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258201    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:44:46.263094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258205    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:44:46.263094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258209    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:44:46.263094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258213    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:44:46.263094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258217    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:44:46.263094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258221    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:44:46.263094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258225    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:44:46.263094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258229    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:44:46.263094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258234    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:44:46.263094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258238    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:44:46.263094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258242    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:44:46.263094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258246    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:44:46.263094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258252    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:44:46.263094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258258    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:44:46.263094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258262    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:44:46.263094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258266    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:44:46.263094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258271    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:44:46.263094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258275    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:44:46.263094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258279    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:44:46.263094 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258284    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:44:46.263753 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258288    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:44:46.263753 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258294    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:44:46.263753 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258300    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:44:46.263753 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258305    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:44:46.263753 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258310    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:44:46.263753 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258315    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:44:46.263753 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258320    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:44:46.263753 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258324    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:44:46.263753 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258329    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:44:46.263753 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258334    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:44:46.263753 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258339    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:44:46.263753 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258344    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:44:46.263753 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258348    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:44:46.263753 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258353    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:44:46.263753 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258357    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:44:46.263753 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258361    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:44:46.263753 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258366    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:44:46.263753 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258370    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:44:46.263753 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258374    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:44:46.263753 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258378    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:44:46.264293 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258382    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:44:46.264293 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258386    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:44:46.264293 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258390    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:44:46.264293 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258395    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:44:46.264293 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258399    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:44:46.264293 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258403    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:44:46.264293 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258407    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:44:46.264293 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258411    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:44:46.264293 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258415    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:44:46.264293 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258419    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:44:46.264293 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258423    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:44:46.264293 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258428    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:44:46.264293 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258432    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:44:46.264293 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258436    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:44:46.264293 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258440    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:44:46.264293 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258444    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:44:46.264293 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258449    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:44:46.264293 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258454    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:44:46.264293 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258458    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:44:46.264293 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258462    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:44:46.264900 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258466    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:44:46.264900 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258471    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:44:46.264900 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258475    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:44:46.264900 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258480    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:44:46.264900 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258484    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:44:46.264900 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:46.258488    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:44:46.264900 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.258496    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 18:44:46.264900 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.258643    2557 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 16 18:44:46.264900 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.264333    2557 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 16 18:44:46.265517 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.265504    2557 server.go:1019] "Starting client certificate rotation"
Sep 16 18:44:46.265627 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.265609    2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 18:44:46.265660 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.265648    2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 18:44:46.268316 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.268299    2557 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 18:44:46.269425 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.269405    2557 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 18:44:46.278270 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.278255    2557 log.go:25] "Validated CRI v1 runtime API"
Sep 16 18:44:46.281555 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.281538    2557 log.go:25] "Validated CRI v1 image API"
Sep 16 18:44:46.282394 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.282377    2557 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 16 18:44:46.288488 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.288471    2557 fs.go:135] Filesystem UUIDs: map[2a752d62-4333-4306-81b4-8599a92bce79:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 86be40c9-415d-4e82-96f8-32f1ee5d4dc9:/dev/nvme0n1p4]
Sep 16 18:44:46.288548 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.288487    2557 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 16 18:44:46.294791 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.294661    2557 manager.go:216] Machine: {Timestamp:2026-09-16 18:44:46.293244226 +0000 UTC m=+0.370349077 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3105592 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec284e85a5761826589c39dc65cca44f SystemUUID:ec284e85-a576-1826-589c-39dc65cca44f BootID:52b86641-fa22-4e59-bd98-1632903772e1 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:c0:7b:46:77 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:c0:7b:46:77 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:06:72:ea:ff:77:d9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 16 18:44:46.294791 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.294783    2557 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 16 18:44:46.294919 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.294858    2557 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 16 18:44:46.297939 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.297923    2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 18:44:46.298651 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.298626    2557 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 16 18:44:46.298800 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.298657    2557 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-26.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 16 18:44:46.299519 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.299509    2557 topology_manager.go:138] "Creating topology manager with none policy"
Sep 16 18:44:46.299550 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.299522    2557 container_manager_linux.go:309] "Creating device plugin manager"
Sep 16 18:44:46.299550 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.299530    2557 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 18:44:46.299550 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.299543    2557 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 18:44:46.300861 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.300851    2557 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 16 18:44:46.305654 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.305643    2557 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 18:44:46.305763 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.305754    2557 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 16 18:44:46.305809 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.305795    2557 kubelet.go:486] "Attempting to sync node with API server"
Sep 16 18:44:46.305809 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.305805    2557 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 16 18:44:46.305865 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.305821    2557 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 16 18:44:46.305865 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.305831    2557 kubelet.go:398] "Adding apiserver pod source"
Sep 16 18:44:46.305865 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.305840    2557 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 16 18:44:46.306201 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.306189    2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 18:44:46.306241 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.306213    2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 18:44:46.308479 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.308448    2557 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 16 18:44:46.311435 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.311420    2557 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 16 18:44:46.311503 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.311450    2557 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 16 18:44:46.313467 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.313455    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 16 18:44:46.313514 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.313471    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 16 18:44:46.313514 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.313477    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 16 18:44:46.313514 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.313482    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 16 18:44:46.313514 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.313491    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 16 18:44:46.313514 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.313496    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 16 18:44:46.313514 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.313509    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 16 18:44:46.313514 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.313514    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 16 18:44:46.313696 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.313521    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 16 18:44:46.313696 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.313526    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 16 18:44:46.313696 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.313535    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 16 18:44:46.313696 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.313540    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 16 18:44:46.315350 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.315335    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 16 18:44:46.315350 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.315352    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 16 18:44:46.316619 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:46.316589    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 16 18:44:46.316691 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:46.316626    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-26.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 16 18:44:46.319133 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.319120    2557 server.go:1268] "Started kubelet"
Sep 16 18:44:46.319208 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.319184    2557 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 16 18:44:46.319311 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.319277    2557 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 16 18:44:46.319370 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.319347    2557 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 16 18:44:46.319933 ip-10-0-130-26 systemd[1]: Started Kubernetes Kubelet.
Sep 16 18:44:46.320682 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.320666    2557 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 16 18:44:46.321855 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.321840    2557 server.go:310] "Adding debug handlers to kubelet server"
Sep 16 18:44:46.327038 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:46.327004    2557 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 16 18:44:46.328382 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.328362    2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nqcq2"
Sep 16 18:44:46.331502 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.331483    2557 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-26.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 16 18:44:46.332604 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:46.331508    2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-26.ec2.internal.18d5e16d41abad48  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-26.ec2.internal,UID:ip-10-0-130-26.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-26.ec2.internal,},FirstTimestamp:2026-09-16 18:44:46.319095112 +0000 UTC m=+0.396199963,LastTimestamp:2026-09-16 18:44:46.319095112 +0000 UTC m=+0.396199963,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-26.ec2.internal,}"
Sep 16 18:44:46.333750 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.333735    2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 16 18:44:46.334465 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.334436    2557 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 16 18:44:46.334593 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:46.334570    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-26.ec2.internal\" not found"
Sep 16 18:44:46.334761 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.334723    2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nqcq2"
Sep 16 18:44:46.336211 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.335522    2557 factory.go:55] Registering systemd factory
Sep 16 18:44:46.336211 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.335541    2557 factory.go:223] Registration of the systemd container factory successfully
Sep 16 18:44:46.336211 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.335719    2557 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 16 18:44:46.336211 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.335796    2557 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 16 18:44:46.336211 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.335813    2557 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 16 18:44:46.336211 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.335976    2557 reconstruct.go:97] "Volume reconstruction finished"
Sep 16 18:44:46.336211 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.335989    2557 reconciler.go:29] "Reconciler: start to sync state"
Sep 16 18:44:46.336711 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.336692    2557 factory.go:153] Registering CRI-O factory
Sep 16 18:44:46.336711 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.336713    2557 factory.go:223] Registration of the crio container factory successfully
Sep 16 18:44:46.336849 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.336760    2557 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 16 18:44:46.336849 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.336797    2557 factory.go:103] Registering Raw factory
Sep 16 18:44:46.336849 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.336810    2557 manager.go:1196] Started watching for new ooms in manager
Sep 16 18:44:46.337211 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.337198    2557 manager.go:349] Starting recovery of all containers
Sep 16 18:44:46.337374 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:46.337340    2557 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-26.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 16 18:44:46.337374 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:46.337358    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 16 18:44:46.348148 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.347985    2557 manager.go:354] Recovery completed
Sep 16 18:44:46.351124 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:46.351098    2557 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Sep 16 18:44:46.353845 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.353833    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:44:46.355409 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.355396    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:44:46.355465 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.355419    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:44:46.355465 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.355429    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:44:46.355962 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.355946    2557 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 16 18:44:46.355962 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.355960    2557 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 16 18:44:46.356098 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.355967    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:44:46.356098 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.355975    2557 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 18:44:46.356614 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.356600    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:44:46.356655 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.356625    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:44:46.356655 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.356645    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:44:46.358010 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.357996    2557 policy_none.go:49] "None policy: Start"
Sep 16 18:44:46.358065 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.358015    2557 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 16 18:44:46.358065 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.358029    2557 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 16 18:44:46.360045 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.360032    2557 policy_none.go:47] "Start"
Sep 16 18:44:46.393375 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.393358    2557 manager.go:337] "Starting Device Plugin manager"
Sep 16 18:44:46.412223 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:46.393399    2557 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 16 18:44:46.412223 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.393411    2557 server.go:87] "Starting device plugin registration server"
Sep 16 18:44:46.412223 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.393710    2557 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 16 18:44:46.412223 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.393721    2557 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 16 18:44:46.412223 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.393823    2557 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 16 18:44:46.412223 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.393907    2557 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 16 18:44:46.412223 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.393917    2557 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 16 18:44:46.412223 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:46.394353    2557 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 16 18:44:46.412223 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:46.394391    2557 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-26.ec2.internal\" not found"
Sep 16 18:44:46.458987 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.458956    2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 16 18:44:46.460143 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.460127    2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 16 18:44:46.460208 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.460176    2557 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 16 18:44:46.460250 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.460208    2557 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 16 18:44:46.460293 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:46.460254    2557 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 16 18:44:46.462617 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.462600    2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:44:46.494649 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.494577    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:44:46.495519 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.495501    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:44:46.495610 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.495535    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:44:46.495610 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.495551    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:44:46.495610 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.495580    2557 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-26.ec2.internal"
Sep 16 18:44:46.503284 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.503269    2557 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-26.ec2.internal"
Sep 16 18:44:46.503357 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:46.503291    2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-26.ec2.internal\": node \"ip-10-0-130-26.ec2.internal\" not found"
Sep 16 18:44:46.522116 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:46.522095    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-26.ec2.internal\" not found"
Sep 16 18:44:46.561172 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.561152    2557 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-26.ec2.internal"]
Sep 16 18:44:46.561241 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.561209    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:44:46.562385 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.562371    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:44:46.562446 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.562398    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:44:46.562446 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.562410    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:44:46.564698 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.564686    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:44:46.564845 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.564831    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal"
Sep 16 18:44:46.564895 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.564858    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:44:46.565395 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.565378    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:44:46.565395 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.565386    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:44:46.565481 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.565403    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:44:46.565481 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.565407    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:44:46.565481 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.565414    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:44:46.565481 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.565418    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:44:46.567677 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.567660    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-26.ec2.internal"
Sep 16 18:44:46.567784 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.567683    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:44:46.568303 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.568288    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:44:46.568365 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.568318    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:44:46.568365 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.568334    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:44:46.594277 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:46.594254    2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-26.ec2.internal\" not found" node="ip-10-0-130-26.ec2.internal"
Sep 16 18:44:46.598474 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:46.598455    2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-26.ec2.internal\" not found" node="ip-10-0-130-26.ec2.internal"
Sep 16 18:44:46.622530 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:46.622510    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-26.ec2.internal\" not found"
Sep 16 18:44:46.637164 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.637143    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ceb312c45138c203c60454ce1db5dde-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal\" (UID: \"9ceb312c45138c203c60454ce1db5dde\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal"
Sep 16 18:44:46.637217 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.637171    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/26c1c5215cb05aa10e9d3bcd9724a09a-config\") pod \"kube-apiserver-proxy-ip-10-0-130-26.ec2.internal\" (UID: \"26c1c5215cb05aa10e9d3bcd9724a09a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-26.ec2.internal"
Sep 16 18:44:46.637217 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.637189    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9ceb312c45138c203c60454ce1db5dde-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal\" (UID: \"9ceb312c45138c203c60454ce1db5dde\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal"
Sep 16 18:44:46.723019 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:46.722998    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-26.ec2.internal\" not found"
Sep 16 18:44:46.737792 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.737758    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9ceb312c45138c203c60454ce1db5dde-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal\" (UID: \"9ceb312c45138c203c60454ce1db5dde\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal"
Sep 16 18:44:46.737850 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.737798    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ceb312c45138c203c60454ce1db5dde-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal\" (UID: \"9ceb312c45138c203c60454ce1db5dde\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal"
Sep 16 18:44:46.737850 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.737816    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/26c1c5215cb05aa10e9d3bcd9724a09a-config\") pod \"kube-apiserver-proxy-ip-10-0-130-26.ec2.internal\" (UID: \"26c1c5215cb05aa10e9d3bcd9724a09a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-26.ec2.internal"
Sep 16 18:44:46.737913 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.737856    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ceb312c45138c203c60454ce1db5dde-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal\" (UID: \"9ceb312c45138c203c60454ce1db5dde\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal"
Sep 16 18:44:46.737913 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.737861    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9ceb312c45138c203c60454ce1db5dde-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal\" (UID: \"9ceb312c45138c203c60454ce1db5dde\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal"
Sep 16 18:44:46.737913 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.737908    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/26c1c5215cb05aa10e9d3bcd9724a09a-config\") pod \"kube-apiserver-proxy-ip-10-0-130-26.ec2.internal\" (UID: \"26c1c5215cb05aa10e9d3bcd9724a09a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-26.ec2.internal"
Sep 16 18:44:46.823239 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:46.823211    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-26.ec2.internal\" not found"
Sep 16 18:44:46.896732 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.896701    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal"
Sep 16 18:44:46.901662 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:46.901648    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-26.ec2.internal"
Sep 16 18:44:46.924294 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:46.924276    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-26.ec2.internal\" not found"
Sep 16 18:44:47.025436 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:47.025404    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-26.ec2.internal\" not found"
Sep 16 18:44:47.132057 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:47.131973    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-26.ec2.internal\" not found"
Sep 16 18:44:47.234274 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:47.234242    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-26.ec2.internal\" not found"
Sep 16 18:44:47.255070 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:47.255050    2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:44:47.265737 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:47.265718    2557 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 16 18:44:47.265870 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:47.265847    2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 18:44:47.265915 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:47.265862    2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 18:44:47.334886 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:47.334862    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-26.ec2.internal\" not found"
Sep 16 18:44:47.342439 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:47.342404    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-15 18:39:47 +0000 UTC" deadline="2028-03-12 02:01:23.659865071 +0000 UTC"
Sep 16 18:44:47.342498 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:47.342443    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13015h16m36.317427876s"
Sep 16 18:44:47.435360 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:47.435298    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-26.ec2.internal\" not found"
Sep 16 18:44:47.536405 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:47.536369    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-26.ec2.internal\" not found"
Sep 16 18:44:47.605844 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:47.605808    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ceb312c45138c203c60454ce1db5dde.slice/crio-16fc081ea16c5c74e5c2810178f32965c519560eb71403b2f093f7d8d660f4f4 WatchSource:0}: Error finding container 16fc081ea16c5c74e5c2810178f32965c519560eb71403b2f093f7d8d660f4f4: Status 404 returned error can't find the container with id 16fc081ea16c5c74e5c2810178f32965c519560eb71403b2f093f7d8d660f4f4
Sep 16 18:44:47.608875 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:47.608821    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26c1c5215cb05aa10e9d3bcd9724a09a.slice/crio-7d7daf987f350b8046466922f85034cf4e0e8c97847f717e408a1e854b2fb3a0 WatchSource:0}: Error finding container 7d7daf987f350b8046466922f85034cf4e0e8c97847f717e408a1e854b2fb3a0: Status 404 returned error can't find the container with id 7d7daf987f350b8046466922f85034cf4e0e8c97847f717e408a1e854b2fb3a0
Sep 16 18:44:47.611420 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:47.611400    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:44:47.636589 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:47.636567    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-26.ec2.internal\" not found"
Sep 16 18:44:47.741589 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:47.741522    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-26.ec2.internal\" not found"
Sep 16 18:44:47.744577 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:47.744558    2557 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:44:47.747378 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:47.747359    2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:44:47.836130 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:47.836087    2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal"
Sep 16 18:44:47.857781 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:47.857755    2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 18:44:47.858630 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:47.858614    2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-26.ec2.internal"
Sep 16 18:44:47.876428 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:47.876408    2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 18:44:48.309328 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.309298    2557 apiserver.go:52] "Watching apiserver"
Sep 16 18:44:48.318316 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.318289    2557 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 16 18:44:48.319272 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.319239    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-tvwkw","openshift-ovn-kubernetes/ovnkube-node-mgl4t","kube-system/global-pull-secret-syncer-c2j6m","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wfkgg","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal","openshift-multus/multus-6g9b9","openshift-multus/multus-additional-cni-plugins-shx66","openshift-multus/network-metrics-daemon-4wnsv","openshift-network-diagnostics/network-check-target-gtj9s","kube-system/konnectivity-agent-r9g58","kube-system/kube-apiserver-proxy-ip-10-0-130-26.ec2.internal","openshift-cluster-node-tuning-operator/tuned-zklcc","openshift-dns/node-resolver-4rzjs","openshift-image-registry/node-ca-q6g22"]
Sep 16 18:44:48.321990 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.321954    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-shx66"
Sep 16 18:44:48.335147 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.335114    2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 16 18:44:48.335260 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.335163    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tvwkw"
Sep 16 18:44:48.337090 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.335486    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wfkgg"
Sep 16 18:44:48.337090 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.335899    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:44:48.337090 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.336000    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 16 18:44:48.337090 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:48.336000    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2j6m" podUID="b7d1f1fc-4f58-4499-a42b-1f9325efa292"
Sep 16 18:44:48.337090 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.336088    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 16 18:44:48.337090 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.336623    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.337090 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.336991    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 16 18:44:48.337504 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.337103    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:44:48.337504 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:48.337213    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wnsv" podUID="c19f514e-c3a8-41d2-9de1-ad2d86fce986"
Sep 16 18:44:48.337504 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.337394    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 16 18:44:48.337667 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.337645    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 16 18:44:48.337667 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.337661    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:44:48.337850 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:48.337742    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtj9s" podUID="4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf"
Sep 16 18:44:48.337910 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.337845    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-lx79v\""
Sep 16 18:44:48.338630 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.338606    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:44:48.340773 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.339152    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 16 18:44:48.340773 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.339426    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 16 18:44:48.340773 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.339953    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 16 18:44:48.340773 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.340281    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mr2vk\""
Sep 16 18:44:48.340773 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.340621    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-x7tp8\""
Sep 16 18:44:48.342746 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.341573    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 16 18:44:48.342746 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.341599    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4rzjs"
Sep 16 18:44:48.342746 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.341885    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 16 18:44:48.342746 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.342501    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 16 18:44:48.342746 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.342574    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rfnnz\""
Sep 16 18:44:48.345932 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.343365    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.345932 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.343998    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-etc-kubernetes\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.345932 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344038    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0307bb60-33a4-4e04-a80e-6e280e7217df-sys-fs\") pod \"aws-ebs-csi-driver-node-wfkgg\" (UID: \"0307bb60-33a4-4e04-a80e-6e280e7217df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wfkgg"
Sep 16 18:44:48.345932 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344071    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-cnibin\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.345932 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344108    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-host-run-netns\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.345932 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344139    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-host-var-lib-cni-bin\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.345932 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344170    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlkwb\" (UniqueName: \"kubernetes.io/projected/87795909-0ae4-4163-aef4-bb1e143885da-kube-api-access-tlkwb\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.345932 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344201    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl62r\" (UniqueName: \"kubernetes.io/projected/4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf-kube-api-access-pl62r\") pod \"network-check-target-gtj9s\" (UID: \"4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf\") " pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:44:48.345932 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344241    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-host-var-lib-cni-multus\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.345932 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344277    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/837f7ec9-d8aa-45b3-bebc-2eb825742375-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-shx66\" (UID: \"837f7ec9-d8aa-45b3-bebc-2eb825742375\") " pod="openshift-multus/multus-additional-cni-plugins-shx66"
Sep 16 18:44:48.345932 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344309    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/030fd847-6677-48f0-8e6b-e30f2e63d1a8-iptables-alerter-script\") pod \"iptables-alerter-tvwkw\" (UID: \"030fd847-6677-48f0-8e6b-e30f2e63d1a8\") " pod="openshift-network-operator/iptables-alerter-tvwkw"
Sep 16 18:44:48.345932 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344345    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7d1f1fc-4f58-4499-a42b-1f9325efa292-original-pull-secret\") pod \"global-pull-secret-syncer-c2j6m\" (UID: \"b7d1f1fc-4f58-4499-a42b-1f9325efa292\") " pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:44:48.345932 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344378    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0307bb60-33a4-4e04-a80e-6e280e7217df-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wfkgg\" (UID: \"0307bb60-33a4-4e04-a80e-6e280e7217df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wfkgg"
Sep 16 18:44:48.345932 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344413    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/87795909-0ae4-4163-aef4-bb1e143885da-cni-binary-copy\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.345932 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344440    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/87795909-0ae4-4163-aef4-bb1e143885da-multus-daemon-config\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.345932 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344474    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/837f7ec9-d8aa-45b3-bebc-2eb825742375-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-shx66\" (UID: \"837f7ec9-d8aa-45b3-bebc-2eb825742375\") " pod="openshift-multus/multus-additional-cni-plugins-shx66"
Sep 16 18:44:48.345932 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344513    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-592rk\" (UniqueName: \"kubernetes.io/projected/837f7ec9-d8aa-45b3-bebc-2eb825742375-kube-api-access-592rk\") pod \"multus-additional-cni-plugins-shx66\" (UID: \"837f7ec9-d8aa-45b3-bebc-2eb825742375\") " pod="openshift-multus/multus-additional-cni-plugins-shx66"
Sep 16 18:44:48.347083 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344547    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r92fg\" (UniqueName: \"kubernetes.io/projected/030fd847-6677-48f0-8e6b-e30f2e63d1a8-kube-api-access-r92fg\") pod \"iptables-alerter-tvwkw\" (UID: \"030fd847-6677-48f0-8e6b-e30f2e63d1a8\") " pod="openshift-network-operator/iptables-alerter-tvwkw"
Sep 16 18:44:48.347083 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344584    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b7d1f1fc-4f58-4499-a42b-1f9325efa292-kubelet-config\") pod \"global-pull-secret-syncer-c2j6m\" (UID: \"b7d1f1fc-4f58-4499-a42b-1f9325efa292\") " pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:44:48.347083 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344618    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6fxs\" (UniqueName: \"kubernetes.io/projected/0307bb60-33a4-4e04-a80e-6e280e7217df-kube-api-access-j6fxs\") pod \"aws-ebs-csi-driver-node-wfkgg\" (UID: \"0307bb60-33a4-4e04-a80e-6e280e7217df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wfkgg"
Sep 16 18:44:48.347083 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344645    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-system-cni-dir\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.347083 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344674    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-hostroot\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.347083 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344701    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-host-run-multus-certs\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.347083 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344728    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/837f7ec9-d8aa-45b3-bebc-2eb825742375-cni-binary-copy\") pod \"multus-additional-cni-plugins-shx66\" (UID: \"837f7ec9-d8aa-45b3-bebc-2eb825742375\") " pod="openshift-multus/multus-additional-cni-plugins-shx66"
Sep 16 18:44:48.347083 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344757    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0307bb60-33a4-4e04-a80e-6e280e7217df-socket-dir\") pod \"aws-ebs-csi-driver-node-wfkgg\" (UID: \"0307bb60-33a4-4e04-a80e-6e280e7217df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wfkgg"
Sep 16 18:44:48.347083 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344781    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0307bb60-33a4-4e04-a80e-6e280e7217df-device-dir\") pod \"aws-ebs-csi-driver-node-wfkgg\" (UID: \"0307bb60-33a4-4e04-a80e-6e280e7217df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wfkgg"
Sep 16 18:44:48.347083 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344805    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-multus-cni-dir\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.347083 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344831    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-multus-socket-dir-parent\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.347083 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344876    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-host-var-lib-kubelet\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.347083 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344901    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c19f514e-c3a8-41d2-9de1-ad2d86fce986-metrics-certs\") pod \"network-metrics-daemon-4wnsv\" (UID: \"c19f514e-c3a8-41d2-9de1-ad2d86fce986\") " pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:44:48.347083 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344929    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0307bb60-33a4-4e04-a80e-6e280e7217df-etc-selinux\") pod \"aws-ebs-csi-driver-node-wfkgg\" (UID: \"0307bb60-33a4-4e04-a80e-6e280e7217df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wfkgg"
Sep 16 18:44:48.347083 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344960    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-host-run-k8s-cni-cncf-io\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.347083 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.344995    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/837f7ec9-d8aa-45b3-bebc-2eb825742375-system-cni-dir\") pod \"multus-additional-cni-plugins-shx66\" (UID: \"837f7ec9-d8aa-45b3-bebc-2eb825742375\") " pod="openshift-multus/multus-additional-cni-plugins-shx66"
Sep 16 18:44:48.347900 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.345026    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/837f7ec9-d8aa-45b3-bebc-2eb825742375-tuning-conf-dir\") pod \"multus-additional-cni-plugins-shx66\" (UID: \"837f7ec9-d8aa-45b3-bebc-2eb825742375\") " pod="openshift-multus/multus-additional-cni-plugins-shx66"
Sep 16 18:44:48.347900 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.345053    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-multus-conf-dir\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.347900 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.345093    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/837f7ec9-d8aa-45b3-bebc-2eb825742375-cnibin\") pod \"multus-additional-cni-plugins-shx66\" (UID: \"837f7ec9-d8aa-45b3-bebc-2eb825742375\") " pod="openshift-multus/multus-additional-cni-plugins-shx66"
Sep 16 18:44:48.347900 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.345130    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7qhk\" (UniqueName: \"kubernetes.io/projected/c19f514e-c3a8-41d2-9de1-ad2d86fce986-kube-api-access-q7qhk\") pod \"network-metrics-daemon-4wnsv\" (UID: \"c19f514e-c3a8-41d2-9de1-ad2d86fce986\") " pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:44:48.347900 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.345163    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/030fd847-6677-48f0-8e6b-e30f2e63d1a8-host-slash\") pod \"iptables-alerter-tvwkw\" (UID: \"030fd847-6677-48f0-8e6b-e30f2e63d1a8\") " pod="openshift-network-operator/iptables-alerter-tvwkw"
Sep 16 18:44:48.347900 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.345194    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b7d1f1fc-4f58-4499-a42b-1f9325efa292-dbus\") pod \"global-pull-secret-syncer-c2j6m\" (UID: \"b7d1f1fc-4f58-4499-a42b-1f9325efa292\") " pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:44:48.347900 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.345234    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0307bb60-33a4-4e04-a80e-6e280e7217df-registration-dir\") pod \"aws-ebs-csi-driver-node-wfkgg\" (UID: \"0307bb60-33a4-4e04-a80e-6e280e7217df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wfkgg"
Sep 16 18:44:48.347900 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.346875    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-r9g58"
Sep 16 18:44:48.347900 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.347121    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mvcs8\""
Sep 16 18:44:48.347900 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.347122    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 16 18:44:48.348379 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.347936    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-r75fm\""
Sep 16 18:44:48.348379 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.348107    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 16 18:44:48.349133 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.349110    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 16 18:44:48.349336 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.349318    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 16 18:44:48.350231 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.350008    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 16 18:44:48.350231 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.350037    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 16 18:44:48.351002 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.350425    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-q6g22"
Sep 16 18:44:48.351002 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.350426    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 16 18:44:48.352510 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.352489    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.352972 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.352952    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 16 18:44:48.353062 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.353046    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 16 18:44:48.353129 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.353036    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-j6z7d\""
Sep 16 18:44:48.353286 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.353269    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 16 18:44:48.353350 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.353269    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 16 18:44:48.354414 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.354393    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vgkvj\""
Sep 16 18:44:48.354753 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.354604    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 16 18:44:48.354898 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.354879    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 16 18:44:48.354990 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.354968    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:44:48.355110 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.355079    2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 18:44:48.357310 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.357290    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 16 18:44:48.357646 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.357620    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-46hks\""
Sep 16 18:44:48.376839 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.376794    2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6fb6g"
Sep 16 18:44:48.385119 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.385094    2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6fb6g"
Sep 16 18:44:48.426226 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.426205    2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:44:48.437095 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.437072    2557 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 16 18:44:48.445789 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.445764    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0307bb60-33a4-4e04-a80e-6e280e7217df-device-dir\") pod \"aws-ebs-csi-driver-node-wfkgg\" (UID: \"0307bb60-33a4-4e04-a80e-6e280e7217df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wfkgg"
Sep 16 18:44:48.445937 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.445821    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-multus-cni-dir\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.445937 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.445846    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-multus-socket-dir-parent\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.445937 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.445863    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-host-var-lib-kubelet\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.445937 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.445884    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c19f514e-c3a8-41d2-9de1-ad2d86fce986-metrics-certs\") pod \"network-metrics-daemon-4wnsv\" (UID: \"c19f514e-c3a8-41d2-9de1-ad2d86fce986\") " pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:44:48.445937 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.445916    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.445937 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.445943    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-sys\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.446236 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.445961    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-multus-socket-dir-parent\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.446236 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.445967    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0307bb60-33a4-4e04-a80e-6e280e7217df-device-dir\") pod \"aws-ebs-csi-driver-node-wfkgg\" (UID: \"0307bb60-33a4-4e04-a80e-6e280e7217df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wfkgg"
Sep 16 18:44:48.446236 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.445974    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0307bb60-33a4-4e04-a80e-6e280e7217df-etc-selinux\") pod \"aws-ebs-csi-driver-node-wfkgg\" (UID: \"0307bb60-33a4-4e04-a80e-6e280e7217df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wfkgg"
Sep 16 18:44:48.446236 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446009    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/837f7ec9-d8aa-45b3-bebc-2eb825742375-system-cni-dir\") pod \"multus-additional-cni-plugins-shx66\" (UID: \"837f7ec9-d8aa-45b3-bebc-2eb825742375\") " pod="openshift-multus/multus-additional-cni-plugins-shx66"
Sep 16 18:44:48.446236 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446053    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/837f7ec9-d8aa-45b3-bebc-2eb825742375-system-cni-dir\") pod \"multus-additional-cni-plugins-shx66\" (UID: \"837f7ec9-d8aa-45b3-bebc-2eb825742375\") " pod="openshift-multus/multus-additional-cni-plugins-shx66"
Sep 16 18:44:48.446236 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446062    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/55642348-e5c2-42ed-af38-d0631bd164aa-hosts-file\") pod \"node-resolver-4rzjs\" (UID: \"55642348-e5c2-42ed-af38-d0631bd164aa\") " pod="openshift-dns/node-resolver-4rzjs"
Sep 16 18:44:48.446236 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446092    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-run-ovn\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.446236 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446104    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-multus-cni-dir\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.446236 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:48.446125    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:44:48.446236 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446142    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-host-var-lib-kubelet\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.446236 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446127    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-host-cni-bin\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.446236 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:48.446204    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c19f514e-c3a8-41d2-9de1-ad2d86fce986-metrics-certs podName:c19f514e-c3a8-41d2-9de1-ad2d86fce986 nodeName:}" failed. No retries permitted until 2026-09-16 18:44:48.946170572 +0000 UTC m=+3.023275426 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c19f514e-c3a8-41d2-9de1-ad2d86fce986-metrics-certs") pod "network-metrics-daemon-4wnsv" (UID: "c19f514e-c3a8-41d2-9de1-ad2d86fce986") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:44:48.446236 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446236    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/837f7ec9-d8aa-45b3-bebc-2eb825742375-cnibin\") pod \"multus-additional-cni-plugins-shx66\" (UID: \"837f7ec9-d8aa-45b3-bebc-2eb825742375\") " pod="openshift-multus/multus-additional-cni-plugins-shx66"
Sep 16 18:44:48.446942 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446265    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/030fd847-6677-48f0-8e6b-e30f2e63d1a8-host-slash\") pod \"iptables-alerter-tvwkw\" (UID: \"030fd847-6677-48f0-8e6b-e30f2e63d1a8\") " pod="openshift-network-operator/iptables-alerter-tvwkw"
Sep 16 18:44:48.446942 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446272    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0307bb60-33a4-4e04-a80e-6e280e7217df-etc-selinux\") pod \"aws-ebs-csi-driver-node-wfkgg\" (UID: \"0307bb60-33a4-4e04-a80e-6e280e7217df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wfkgg"
Sep 16 18:44:48.446942 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446296    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-etc-openvswitch\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.446942 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446318    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/837f7ec9-d8aa-45b3-bebc-2eb825742375-cnibin\") pod \"multus-additional-cni-plugins-shx66\" (UID: \"837f7ec9-d8aa-45b3-bebc-2eb825742375\") " pod="openshift-multus/multus-additional-cni-plugins-shx66"
Sep 16 18:44:48.446942 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446333    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c85031e5-5e03-4f66-b61c-636789b6e44f-ovnkube-config\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.446942 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446359    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b7d1f1fc-4f58-4499-a42b-1f9325efa292-dbus\") pod \"global-pull-secret-syncer-c2j6m\" (UID: \"b7d1f1fc-4f58-4499-a42b-1f9325efa292\") " pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:44:48.446942 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446362    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/030fd847-6677-48f0-8e6b-e30f2e63d1a8-host-slash\") pod \"iptables-alerter-tvwkw\" (UID: \"030fd847-6677-48f0-8e6b-e30f2e63d1a8\") " pod="openshift-network-operator/iptables-alerter-tvwkw"
Sep 16 18:44:48.446942 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446384    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-etc-kubernetes\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.446942 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446408    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55642348-e5c2-42ed-af38-d0631bd164aa-tmp-dir\") pod \"node-resolver-4rzjs\" (UID: \"55642348-e5c2-42ed-af38-d0631bd164aa\") " pod="openshift-dns/node-resolver-4rzjs"
Sep 16 18:44:48.446942 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446434    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-lib-modules\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.446942 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446438    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b7d1f1fc-4f58-4499-a42b-1f9325efa292-dbus\") pod \"global-pull-secret-syncer-c2j6m\" (UID: \"b7d1f1fc-4f58-4499-a42b-1f9325efa292\") " pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:44:48.446942 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446460    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-host-var-lib-cni-bin\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.446942 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446497    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlkwb\" (UniqueName: \"kubernetes.io/projected/87795909-0ae4-4163-aef4-bb1e143885da-kube-api-access-tlkwb\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.446942 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446509    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-etc-kubernetes\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.446942 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446558    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-host-var-lib-cni-bin\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.446942 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446566    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-host-cni-netd\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.446942 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446653    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5c6f6f8b-e6fc-4a16-b084-7e851b3c3cc6-serviceca\") pod \"node-ca-q6g22\" (UID: \"5c6f6f8b-e6fc-4a16-b084-7e851b3c3cc6\") " pod="openshift-image-registry/node-ca-q6g22"
Sep 16 18:44:48.447742 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446680    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-etc-modprobe-d\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.447742 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446710    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-host-var-lib-cni-multus\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.447742 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446741    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-host-var-lib-cni-multus\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.447742 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446773    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/837f7ec9-d8aa-45b3-bebc-2eb825742375-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-shx66\" (UID: \"837f7ec9-d8aa-45b3-bebc-2eb825742375\") " pod="openshift-multus/multus-additional-cni-plugins-shx66"
Sep 16 18:44:48.447742 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446818    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-host-run-netns\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.447742 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446865    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9747l\" (UniqueName: \"kubernetes.io/projected/5c6f6f8b-e6fc-4a16-b084-7e851b3c3cc6-kube-api-access-9747l\") pod \"node-ca-q6g22\" (UID: \"5c6f6f8b-e6fc-4a16-b084-7e851b3c3cc6\") " pod="openshift-image-registry/node-ca-q6g22"
Sep 16 18:44:48.447742 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446923    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-run\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.447742 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446960    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7d1f1fc-4f58-4499-a42b-1f9325efa292-original-pull-secret\") pod \"global-pull-secret-syncer-c2j6m\" (UID: \"b7d1f1fc-4f58-4499-a42b-1f9325efa292\") " pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:44:48.447742 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.446990    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0307bb60-33a4-4e04-a80e-6e280e7217df-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wfkgg\" (UID: \"0307bb60-33a4-4e04-a80e-6e280e7217df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wfkgg"
Sep 16 18:44:48.447742 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447015    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/87795909-0ae4-4163-aef4-bb1e143885da-multus-daemon-config\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.447742 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447056    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-592rk\" (UniqueName: \"kubernetes.io/projected/837f7ec9-d8aa-45b3-bebc-2eb825742375-kube-api-access-592rk\") pod \"multus-additional-cni-plugins-shx66\" (UID: \"837f7ec9-d8aa-45b3-bebc-2eb825742375\") " pod="openshift-multus/multus-additional-cni-plugins-shx66"
Sep 16 18:44:48.447742 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447084    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-etc-sysconfig\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.447742 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:48.447096    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:44:48.447742 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447114    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6fxs\" (UniqueName: \"kubernetes.io/projected/0307bb60-33a4-4e04-a80e-6e280e7217df-kube-api-access-j6fxs\") pod \"aws-ebs-csi-driver-node-wfkgg\" (UID: \"0307bb60-33a4-4e04-a80e-6e280e7217df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wfkgg"
Sep 16 18:44:48.447742 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447140    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-system-cni-dir\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.447742 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:48.447159    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7d1f1fc-4f58-4499-a42b-1f9325efa292-original-pull-secret podName:b7d1f1fc-4f58-4499-a42b-1f9325efa292 nodeName:}" failed. No retries permitted until 2026-09-16 18:44:48.947141419 +0000 UTC m=+3.024246274 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b7d1f1fc-4f58-4499-a42b-1f9325efa292-original-pull-secret") pod "global-pull-secret-syncer-c2j6m" (UID: "b7d1f1fc-4f58-4499-a42b-1f9325efa292") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:44:48.448658 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447191    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-host-run-multus-certs\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.448658 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447221    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-run-systemd\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.448658 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447243    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-system-cni-dir\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.448658 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447245    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c6f6f8b-e6fc-4a16-b084-7e851b3c3cc6-host\") pod \"node-ca-q6g22\" (UID: \"5c6f6f8b-e6fc-4a16-b084-7e851b3c3cc6\") " pod="openshift-image-registry/node-ca-q6g22"
Sep 16 18:44:48.448658 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447097    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0307bb60-33a4-4e04-a80e-6e280e7217df-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wfkgg\" (UID: \"0307bb60-33a4-4e04-a80e-6e280e7217df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wfkgg"
Sep 16 18:44:48.448658 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447285    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jt67\" (UniqueName: \"kubernetes.io/projected/f03fb6fa-7cba-4886-958a-81e902f11566-kube-api-access-7jt67\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.448658 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447316    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-host-run-multus-certs\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.448658 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447333    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0307bb60-33a4-4e04-a80e-6e280e7217df-socket-dir\") pod \"aws-ebs-csi-driver-node-wfkgg\" (UID: \"0307bb60-33a4-4e04-a80e-6e280e7217df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wfkgg"
Sep 16 18:44:48.448658 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447359    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c85031e5-5e03-4f66-b61c-636789b6e44f-ovn-node-metrics-cert\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.448658 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447391    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-host-run-k8s-cni-cncf-io\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.448658 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447410    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/837f7ec9-d8aa-45b3-bebc-2eb825742375-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-shx66\" (UID: \"837f7ec9-d8aa-45b3-bebc-2eb825742375\") " pod="openshift-multus/multus-additional-cni-plugins-shx66"
Sep 16 18:44:48.448658 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447419    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/837f7ec9-d8aa-45b3-bebc-2eb825742375-tuning-conf-dir\") pod \"multus-additional-cni-plugins-shx66\" (UID: \"837f7ec9-d8aa-45b3-bebc-2eb825742375\") " pod="openshift-multus/multus-additional-cni-plugins-shx66"
Sep 16 18:44:48.448658 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447450    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-multus-conf-dir\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.448658 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447486    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7qhk\" (UniqueName: \"kubernetes.io/projected/c19f514e-c3a8-41d2-9de1-ad2d86fce986-kube-api-access-q7qhk\") pod \"network-metrics-daemon-4wnsv\" (UID: \"c19f514e-c3a8-41d2-9de1-ad2d86fce986\") " pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:44:48.448658 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447512    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-var-lib-openvswitch\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.448658 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447538    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c85031e5-5e03-4f66-b61c-636789b6e44f-ovnkube-script-lib\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.448658 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447533    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0307bb60-33a4-4e04-a80e-6e280e7217df-socket-dir\") pod \"aws-ebs-csi-driver-node-wfkgg\" (UID: \"0307bb60-33a4-4e04-a80e-6e280e7217df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wfkgg"
Sep 16 18:44:48.449632 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447569    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-etc-kubernetes\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.449632 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447593    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/87795909-0ae4-4163-aef4-bb1e143885da-multus-daemon-config\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.449632 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447609    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-etc-systemd\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.449632 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447640    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0307bb60-33a4-4e04-a80e-6e280e7217df-registration-dir\") pod \"aws-ebs-csi-driver-node-wfkgg\" (UID: \"0307bb60-33a4-4e04-a80e-6e280e7217df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wfkgg"
Sep 16 18:44:48.449632 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447674    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-host-run-k8s-cni-cncf-io\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.449632 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447687    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh42n\" (UniqueName: \"kubernetes.io/projected/55642348-e5c2-42ed-af38-d0631bd164aa-kube-api-access-zh42n\") pod \"node-resolver-4rzjs\" (UID: \"55642348-e5c2-42ed-af38-d0631bd164aa\") " pod="openshift-dns/node-resolver-4rzjs"
Sep 16 18:44:48.449632 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447718    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-host-kubelet\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.449632 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447742    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/837f7ec9-d8aa-45b3-bebc-2eb825742375-tuning-conf-dir\") pod \"multus-additional-cni-plugins-shx66\" (UID: \"837f7ec9-d8aa-45b3-bebc-2eb825742375\") " pod="openshift-multus/multus-additional-cni-plugins-shx66"
Sep 16 18:44:48.449632 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447746    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c85031e5-5e03-4f66-b61c-636789b6e44f-env-overrides\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.449632 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447777    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-multus-conf-dir\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.449632 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447802    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0307bb60-33a4-4e04-a80e-6e280e7217df-registration-dir\") pod \"aws-ebs-csi-driver-node-wfkgg\" (UID: \"0307bb60-33a4-4e04-a80e-6e280e7217df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wfkgg"
Sep 16 18:44:48.449632 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447781    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0307bb60-33a4-4e04-a80e-6e280e7217df-sys-fs\") pod \"aws-ebs-csi-driver-node-wfkgg\" (UID: \"0307bb60-33a4-4e04-a80e-6e280e7217df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wfkgg"
Sep 16 18:44:48.449632 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447853    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0307bb60-33a4-4e04-a80e-6e280e7217df-sys-fs\") pod \"aws-ebs-csi-driver-node-wfkgg\" (UID: \"0307bb60-33a4-4e04-a80e-6e280e7217df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wfkgg"
Sep 16 18:44:48.449632 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447866    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-cnibin\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.449632 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447898    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-host-run-netns\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.449632 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447929    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-cnibin\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.449632 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447929    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl62r\" (UniqueName: \"kubernetes.io/projected/4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf-kube-api-access-pl62r\") pod \"network-check-target-gtj9s\" (UID: \"4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf\") " pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:44:48.450168 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447964    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-host-run-netns\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.450168 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.447977    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-systemd-units\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.450168 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.448003    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4c1c55c5-0ed8-46eb-a61d-381273852b77-agent-certs\") pod \"konnectivity-agent-r9g58\" (UID: \"4c1c55c5-0ed8-46eb-a61d-381273852b77\") " pod="kube-system/konnectivity-agent-r9g58"
Sep 16 18:44:48.450168 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.448027    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f03fb6fa-7cba-4886-958a-81e902f11566-etc-tuned\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.450168 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.448054    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/030fd847-6677-48f0-8e6b-e30f2e63d1a8-iptables-alerter-script\") pod \"iptables-alerter-tvwkw\" (UID: \"030fd847-6677-48f0-8e6b-e30f2e63d1a8\") " pod="openshift-network-operator/iptables-alerter-tvwkw"
Sep 16 18:44:48.450168 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.448081    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-host-slash\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.450168 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.448204    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-log-socket\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.450168 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.448235    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-etc-sysctl-d\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.450168 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.448270    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-etc-sysctl-conf\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.450168 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.448299    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/87795909-0ae4-4163-aef4-bb1e143885da-cni-binary-copy\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.450168 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.448327    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/837f7ec9-d8aa-45b3-bebc-2eb825742375-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-shx66\" (UID: \"837f7ec9-d8aa-45b3-bebc-2eb825742375\") " pod="openshift-multus/multus-additional-cni-plugins-shx66"
Sep 16 18:44:48.450168 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.448358    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r92fg\" (UniqueName: \"kubernetes.io/projected/030fd847-6677-48f0-8e6b-e30f2e63d1a8-kube-api-access-r92fg\") pod \"iptables-alerter-tvwkw\" (UID: \"030fd847-6677-48f0-8e6b-e30f2e63d1a8\") " pod="openshift-network-operator/iptables-alerter-tvwkw"
Sep 16 18:44:48.450168 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.448386    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-run-openvswitch\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.450168 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.448412    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-node-log\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.450168 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.448458    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55htk\" (UniqueName: \"kubernetes.io/projected/c85031e5-5e03-4f66-b61c-636789b6e44f-kube-api-access-55htk\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.450168 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.448485    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4c1c55c5-0ed8-46eb-a61d-381273852b77-konnectivity-ca\") pod \"konnectivity-agent-r9g58\" (UID: \"4c1c55c5-0ed8-46eb-a61d-381273852b77\") " pod="kube-system/konnectivity-agent-r9g58"
Sep 16 18:44:48.450168 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.448513    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-var-lib-kubelet\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.451016 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.448539    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b7d1f1fc-4f58-4499-a42b-1f9325efa292-kubelet-config\") pod \"global-pull-secret-syncer-c2j6m\" (UID: \"b7d1f1fc-4f58-4499-a42b-1f9325efa292\") " pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:44:48.451016 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.448567    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-hostroot\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.451016 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.448592    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/837f7ec9-d8aa-45b3-bebc-2eb825742375-cni-binary-copy\") pod \"multus-additional-cni-plugins-shx66\" (UID: \"837f7ec9-d8aa-45b3-bebc-2eb825742375\") " pod="openshift-multus/multus-additional-cni-plugins-shx66"
Sep 16 18:44:48.451016 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.448619    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-host-run-ovn-kubernetes\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.451016 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.448653    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-host\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.451016 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.448663    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/030fd847-6677-48f0-8e6b-e30f2e63d1a8-iptables-alerter-script\") pod \"iptables-alerter-tvwkw\" (UID: \"030fd847-6677-48f0-8e6b-e30f2e63d1a8\") " pod="openshift-network-operator/iptables-alerter-tvwkw"
Sep 16 18:44:48.451016 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.448677    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f03fb6fa-7cba-4886-958a-81e902f11566-tmp\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.451016 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.448788    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/87795909-0ae4-4163-aef4-bb1e143885da-cni-binary-copy\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.451016 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.448943    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b7d1f1fc-4f58-4499-a42b-1f9325efa292-kubelet-config\") pod \"global-pull-secret-syncer-c2j6m\" (UID: \"b7d1f1fc-4f58-4499-a42b-1f9325efa292\") " pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:44:48.451016 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.448984    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/87795909-0ae4-4163-aef4-bb1e143885da-hostroot\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.451016 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.449464    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/837f7ec9-d8aa-45b3-bebc-2eb825742375-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-shx66\" (UID: \"837f7ec9-d8aa-45b3-bebc-2eb825742375\") " pod="openshift-multus/multus-additional-cni-plugins-shx66"
Sep 16 18:44:48.451016 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.449469    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/837f7ec9-d8aa-45b3-bebc-2eb825742375-cni-binary-copy\") pod \"multus-additional-cni-plugins-shx66\" (UID: \"837f7ec9-d8aa-45b3-bebc-2eb825742375\") " pod="openshift-multus/multus-additional-cni-plugins-shx66"
Sep 16 18:44:48.456362 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.456340    2557 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 16 18:44:48.458203 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:48.458178    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:44:48.458300 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:48.458207    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:44:48.458300 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:48.458222    2557 projected.go:196] Error preparing data for projected volume kube-api-access-pl62r for pod openshift-network-diagnostics/network-check-target-gtj9s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:44:48.458300 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:48.458284    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf-kube-api-access-pl62r podName:4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf nodeName:}" failed. No retries permitted until 2026-09-16 18:44:48.958268199 +0000 UTC m=+3.035373061 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pl62r" (UniqueName: "kubernetes.io/projected/4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf-kube-api-access-pl62r") pod "network-check-target-gtj9s" (UID: "4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:44:48.461088 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.461007    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlkwb\" (UniqueName: \"kubernetes.io/projected/87795909-0ae4-4163-aef4-bb1e143885da-kube-api-access-tlkwb\") pod \"multus-6g9b9\" (UID: \"87795909-0ae4-4163-aef4-bb1e143885da\") " pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.461088 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.461028    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6fxs\" (UniqueName: \"kubernetes.io/projected/0307bb60-33a4-4e04-a80e-6e280e7217df-kube-api-access-j6fxs\") pod \"aws-ebs-csi-driver-node-wfkgg\" (UID: \"0307bb60-33a4-4e04-a80e-6e280e7217df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wfkgg"
Sep 16 18:44:48.461896 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.461825    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r92fg\" (UniqueName: \"kubernetes.io/projected/030fd847-6677-48f0-8e6b-e30f2e63d1a8-kube-api-access-r92fg\") pod \"iptables-alerter-tvwkw\" (UID: \"030fd847-6677-48f0-8e6b-e30f2e63d1a8\") " pod="openshift-network-operator/iptables-alerter-tvwkw"
Sep 16 18:44:48.463091 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.463048    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-592rk\" (UniqueName: \"kubernetes.io/projected/837f7ec9-d8aa-45b3-bebc-2eb825742375-kube-api-access-592rk\") pod \"multus-additional-cni-plugins-shx66\" (UID: \"837f7ec9-d8aa-45b3-bebc-2eb825742375\") " pod="openshift-multus/multus-additional-cni-plugins-shx66"
Sep 16 18:44:48.467465 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.466388    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7qhk\" (UniqueName: \"kubernetes.io/projected/c19f514e-c3a8-41d2-9de1-ad2d86fce986-kube-api-access-q7qhk\") pod \"network-metrics-daemon-4wnsv\" (UID: \"c19f514e-c3a8-41d2-9de1-ad2d86fce986\") " pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:44:48.467465 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.467071    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-26.ec2.internal" event={"ID":"26c1c5215cb05aa10e9d3bcd9724a09a","Type":"ContainerStarted","Data":"7d7daf987f350b8046466922f85034cf4e0e8c97847f717e408a1e854b2fb3a0"}
Sep 16 18:44:48.468663 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.468630    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal" event={"ID":"9ceb312c45138c203c60454ce1db5dde","Type":"ContainerStarted","Data":"16fc081ea16c5c74e5c2810178f32965c519560eb71403b2f093f7d8d660f4f4"}
Sep 16 18:44:48.549489 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.549446    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-host-cni-netd\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.549489 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.549495    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5c6f6f8b-e6fc-4a16-b084-7e851b3c3cc6-serviceca\") pod \"node-ca-q6g22\" (UID: \"5c6f6f8b-e6fc-4a16-b084-7e851b3c3cc6\") " pod="openshift-image-registry/node-ca-q6g22"
Sep 16 18:44:48.549729 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.549523    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-etc-modprobe-d\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.549729 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.549550    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-host-run-netns\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.549729 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.549576    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9747l\" (UniqueName: \"kubernetes.io/projected/5c6f6f8b-e6fc-4a16-b084-7e851b3c3cc6-kube-api-access-9747l\") pod \"node-ca-q6g22\" (UID: \"5c6f6f8b-e6fc-4a16-b084-7e851b3c3cc6\") " pod="openshift-image-registry/node-ca-q6g22"
Sep 16 18:44:48.549729 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.549590    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-host-cni-netd\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.549729 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.549602    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-run\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.549729 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.549662    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-run\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.549729 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.549676    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-etc-sysconfig\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.549729 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.549705    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-run-systemd\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.549729 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.549731    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c6f6f8b-e6fc-4a16-b084-7e851b3c3cc6-host\") pod \"node-ca-q6g22\" (UID: \"5c6f6f8b-e6fc-4a16-b084-7e851b3c3cc6\") " pod="openshift-image-registry/node-ca-q6g22"
Sep 16 18:44:48.550266 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.549758    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jt67\" (UniqueName: \"kubernetes.io/projected/f03fb6fa-7cba-4886-958a-81e902f11566-kube-api-access-7jt67\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.550266 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.549786    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-etc-modprobe-d\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.550266 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.549805    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c85031e5-5e03-4f66-b61c-636789b6e44f-ovn-node-metrics-cert\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.550266 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.549839    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-host-run-netns\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.550266 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.549841    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-var-lib-openvswitch\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.550266 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.549868    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c85031e5-5e03-4f66-b61c-636789b6e44f-ovnkube-script-lib\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.550266 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.549896    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-etc-kubernetes\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.550266 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.549919    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-etc-systemd\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.550266 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.549945    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh42n\" (UniqueName: \"kubernetes.io/projected/55642348-e5c2-42ed-af38-d0631bd164aa-kube-api-access-zh42n\") pod \"node-resolver-4rzjs\" (UID: \"55642348-e5c2-42ed-af38-d0631bd164aa\") " pod="openshift-dns/node-resolver-4rzjs"
Sep 16 18:44:48.550266 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.549969    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-host-kubelet\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.550266 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.549993    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c85031e5-5e03-4f66-b61c-636789b6e44f-env-overrides\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.550266 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.550033    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-systemd-units\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.550266 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.550046    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-var-lib-openvswitch\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.550266 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.550050    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5c6f6f8b-e6fc-4a16-b084-7e851b3c3cc6-serviceca\") pod \"node-ca-q6g22\" (UID: \"5c6f6f8b-e6fc-4a16-b084-7e851b3c3cc6\") " pod="openshift-image-registry/node-ca-q6g22"
Sep 16 18:44:48.550266 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.550059    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4c1c55c5-0ed8-46eb-a61d-381273852b77-agent-certs\") pod \"konnectivity-agent-r9g58\" (UID: \"4c1c55c5-0ed8-46eb-a61d-381273852b77\") " pod="kube-system/konnectivity-agent-r9g58"
Sep 16 18:44:48.550266 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.550081    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f03fb6fa-7cba-4886-958a-81e902f11566-etc-tuned\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.550266 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.550107    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-host-slash\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.551024 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.550134    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-log-socket\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.551024 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.550142    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-run-systemd\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.551024 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.550158    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-etc-sysctl-d\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.551024 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.550182    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-etc-sysctl-conf\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.551024 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.550215    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-run-openvswitch\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.551024 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.550215    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-etc-kubernetes\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.551024 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.550109    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c6f6f8b-e6fc-4a16-b084-7e851b3c3cc6-host\") pod \"node-ca-q6g22\" (UID: \"5c6f6f8b-e6fc-4a16-b084-7e851b3c3cc6\") " pod="openshift-image-registry/node-ca-q6g22"
Sep 16 18:44:48.551024 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.550252    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-host-kubelet\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.551024 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.550268    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-systemd-units\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.551024 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.550284    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-node-log\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.551024 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.550084    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-etc-sysconfig\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.551024 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.550673    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c85031e5-5e03-4f66-b61c-636789b6e44f-env-overrides\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.551024 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.550698    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-etc-systemd\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.551024 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.550768    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-host-slash\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.551024 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.550781    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-etc-sysctl-d\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.551024 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.550840    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-log-socket\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.551024 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.550906    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-run-openvswitch\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.551024 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.550943    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-etc-sysctl-conf\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.551895 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.551012    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55htk\" (UniqueName: \"kubernetes.io/projected/c85031e5-5e03-4f66-b61c-636789b6e44f-kube-api-access-55htk\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.551895 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.551046    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4c1c55c5-0ed8-46eb-a61d-381273852b77-konnectivity-ca\") pod \"konnectivity-agent-r9g58\" (UID: \"4c1c55c5-0ed8-46eb-a61d-381273852b77\") " pod="kube-system/konnectivity-agent-r9g58"
Sep 16 18:44:48.551895 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.551073    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-var-lib-kubelet\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.551895 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.551103    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-host-run-ovn-kubernetes\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.551895 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.551129    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-host\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.551895 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.551133    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-node-log\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.551895 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.551153    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f03fb6fa-7cba-4886-958a-81e902f11566-tmp\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.551895 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.551190    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-var-lib-kubelet\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.551895 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.551200    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.551895 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.551206    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c85031e5-5e03-4f66-b61c-636789b6e44f-ovnkube-script-lib\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.551895 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.551228    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-sys\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.551895 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.551258    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/55642348-e5c2-42ed-af38-d0631bd164aa-hosts-file\") pod \"node-resolver-4rzjs\" (UID: \"55642348-e5c2-42ed-af38-d0631bd164aa\") " pod="openshift-dns/node-resolver-4rzjs"
Sep 16 18:44:48.551895 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.551282    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-run-ovn\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.551895 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.551306    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-host-cni-bin\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.551895 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.551334    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-etc-openvswitch\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.551895 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.551358    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c85031e5-5e03-4f66-b61c-636789b6e44f-ovnkube-config\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.551895 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.551385    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55642348-e5c2-42ed-af38-d0631bd164aa-tmp-dir\") pod \"node-resolver-4rzjs\" (UID: \"55642348-e5c2-42ed-af38-d0631bd164aa\") " pod="openshift-dns/node-resolver-4rzjs"
Sep 16 18:44:48.551895 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.551410    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-lib-modules\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.552800 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.551478    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/55642348-e5c2-42ed-af38-d0631bd164aa-hosts-file\") pod \"node-resolver-4rzjs\" (UID: \"55642348-e5c2-42ed-af38-d0631bd164aa\") " pod="openshift-dns/node-resolver-4rzjs"
Sep 16 18:44:48.552800 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.551524    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-host-run-ovn-kubernetes\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.552800 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.551541    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-lib-modules\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.552800 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.551585    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-run-ovn\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.552800 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.551589    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-host\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.552800 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.551627    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-host-cni-bin\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.552800 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.551628    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-etc-openvswitch\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.552800 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.551681    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c85031e5-5e03-4f66-b61c-636789b6e44f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.552800 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.551726    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4c1c55c5-0ed8-46eb-a61d-381273852b77-konnectivity-ca\") pod \"konnectivity-agent-r9g58\" (UID: \"4c1c55c5-0ed8-46eb-a61d-381273852b77\") " pod="kube-system/konnectivity-agent-r9g58"
Sep 16 18:44:48.552800 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.551945    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f03fb6fa-7cba-4886-958a-81e902f11566-sys\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.552800 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.552011    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55642348-e5c2-42ed-af38-d0631bd164aa-tmp-dir\") pod \"node-resolver-4rzjs\" (UID: \"55642348-e5c2-42ed-af38-d0631bd164aa\") " pod="openshift-dns/node-resolver-4rzjs"
Sep 16 18:44:48.552800 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.552258    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c85031e5-5e03-4f66-b61c-636789b6e44f-ovnkube-config\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.554645 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.554301    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f03fb6fa-7cba-4886-958a-81e902f11566-etc-tuned\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.554645 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.554334    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f03fb6fa-7cba-4886-958a-81e902f11566-tmp\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.554875 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.554848    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c85031e5-5e03-4f66-b61c-636789b6e44f-ovn-node-metrics-cert\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.556095 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.556071    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4c1c55c5-0ed8-46eb-a61d-381273852b77-agent-certs\") pod \"konnectivity-agent-r9g58\" (UID: \"4c1c55c5-0ed8-46eb-a61d-381273852b77\") " pod="kube-system/konnectivity-agent-r9g58"
Sep 16 18:44:48.561241 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.561144    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jt67\" (UniqueName: \"kubernetes.io/projected/f03fb6fa-7cba-4886-958a-81e902f11566-kube-api-access-7jt67\") pod \"tuned-zklcc\" (UID: \"f03fb6fa-7cba-4886-958a-81e902f11566\") " pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.561366 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.561336    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh42n\" (UniqueName: \"kubernetes.io/projected/55642348-e5c2-42ed-af38-d0631bd164aa-kube-api-access-zh42n\") pod \"node-resolver-4rzjs\" (UID: \"55642348-e5c2-42ed-af38-d0631bd164aa\") " pod="openshift-dns/node-resolver-4rzjs"
Sep 16 18:44:48.561366 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.561352    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9747l\" (UniqueName: \"kubernetes.io/projected/5c6f6f8b-e6fc-4a16-b084-7e851b3c3cc6-kube-api-access-9747l\") pod \"node-ca-q6g22\" (UID: \"5c6f6f8b-e6fc-4a16-b084-7e851b3c3cc6\") " pod="openshift-image-registry/node-ca-q6g22"
Sep 16 18:44:48.562809 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.562787    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55htk\" (UniqueName: \"kubernetes.io/projected/c85031e5-5e03-4f66-b61c-636789b6e44f-kube-api-access-55htk\") pod \"ovnkube-node-mgl4t\" (UID: \"c85031e5-5e03-4f66-b61c-636789b6e44f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.651564 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.651506    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-shx66"
Sep 16 18:44:48.664370 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.664337    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tvwkw"
Sep 16 18:44:48.674352 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.674332    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wfkgg"
Sep 16 18:44:48.681203 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.681170    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6g9b9"
Sep 16 18:44:48.690985 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.690967    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4rzjs"
Sep 16 18:44:48.698759 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.698737    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:44:48.706564 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.706542    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-r9g58"
Sep 16 18:44:48.714375 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.714357    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-q6g22"
Sep 16 18:44:48.727158 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.727137    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zklcc"
Sep 16 18:44:48.954143 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.954054    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c19f514e-c3a8-41d2-9de1-ad2d86fce986-metrics-certs\") pod \"network-metrics-daemon-4wnsv\" (UID: \"c19f514e-c3a8-41d2-9de1-ad2d86fce986\") " pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:44:48.954143 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:48.954117    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7d1f1fc-4f58-4499-a42b-1f9325efa292-original-pull-secret\") pod \"global-pull-secret-syncer-c2j6m\" (UID: \"b7d1f1fc-4f58-4499-a42b-1f9325efa292\") " pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:44:48.954310 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:48.954226    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:44:48.954310 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:48.954266    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:44:48.954310 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:48.954300    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c19f514e-c3a8-41d2-9de1-ad2d86fce986-metrics-certs podName:c19f514e-c3a8-41d2-9de1-ad2d86fce986 nodeName:}" failed. No retries permitted until 2026-09-16 18:44:49.954282666 +0000 UTC m=+4.031387523 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c19f514e-c3a8-41d2-9de1-ad2d86fce986-metrics-certs") pod "network-metrics-daemon-4wnsv" (UID: "c19f514e-c3a8-41d2-9de1-ad2d86fce986") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:44:48.954455 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:48.954318    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7d1f1fc-4f58-4499-a42b-1f9325efa292-original-pull-secret podName:b7d1f1fc-4f58-4499-a42b-1f9325efa292 nodeName:}" failed. No retries permitted until 2026-09-16 18:44:49.954309246 +0000 UTC m=+4.031414096 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b7d1f1fc-4f58-4499-a42b-1f9325efa292-original-pull-secret") pod "global-pull-secret-syncer-c2j6m" (UID: "b7d1f1fc-4f58-4499-a42b-1f9325efa292") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:44:49.054723 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:49.054683    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl62r\" (UniqueName: \"kubernetes.io/projected/4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf-kube-api-access-pl62r\") pod \"network-check-target-gtj9s\" (UID: \"4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf\") " pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:44:49.054904 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:49.054858    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:44:49.054904 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:49.054891    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:44:49.054991 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:49.054906    2557 projected.go:196] Error preparing data for projected volume kube-api-access-pl62r for pod openshift-network-diagnostics/network-check-target-gtj9s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:44:49.054991 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:49.054977    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf-kube-api-access-pl62r podName:4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf nodeName:}" failed. No retries permitted until 2026-09-16 18:44:50.054956878 +0000 UTC m=+4.132061741 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pl62r" (UniqueName: "kubernetes.io/projected/4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf-kube-api-access-pl62r") pod "network-check-target-gtj9s" (UID: "4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:44:49.387959 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:49.387914    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 18:39:49 +0000 UTC" deadline="2028-02-12 10:13:49.799373837 +0000 UTC"
Sep 16 18:44:49.387959 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:49.387951    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12327h29m0.411427076s"
Sep 16 18:44:49.460916 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:49.460878    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:44:49.461086 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:49.461022    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2j6m" podUID="b7d1f1fc-4f58-4499-a42b-1f9325efa292"
Sep 16 18:44:49.554132 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:49.553913    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c6f6f8b_e6fc_4a16_b084_7e851b3c3cc6.slice/crio-3aeee5f5921f8a8947f6cf5011dbcd299cf15ef4e1ce78c33520b372ee173eb7 WatchSource:0}: Error finding container 3aeee5f5921f8a8947f6cf5011dbcd299cf15ef4e1ce78c33520b372ee173eb7: Status 404 returned error can't find the container with id 3aeee5f5921f8a8947f6cf5011dbcd299cf15ef4e1ce78c33520b372ee173eb7
Sep 16 18:44:49.556896 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:44:49.556689    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf03fb6fa_7cba_4886_958a_81e902f11566.slice/crio-299e569047b2b674fe8b0ded122bbe9bf4de0a10cd89901ee6b6ae745d329d0e WatchSource:0}: Error finding container 299e569047b2b674fe8b0ded122bbe9bf4de0a10cd89901ee6b6ae745d329d0e: Status 404 returned error can't find the container with id 299e569047b2b674fe8b0ded122bbe9bf4de0a10cd89901ee6b6ae745d329d0e
Sep 16 18:44:49.962186 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:49.962091    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c19f514e-c3a8-41d2-9de1-ad2d86fce986-metrics-certs\") pod \"network-metrics-daemon-4wnsv\" (UID: \"c19f514e-c3a8-41d2-9de1-ad2d86fce986\") " pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:44:49.962186 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:49.962157    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7d1f1fc-4f58-4499-a42b-1f9325efa292-original-pull-secret\") pod \"global-pull-secret-syncer-c2j6m\" (UID: \"b7d1f1fc-4f58-4499-a42b-1f9325efa292\") " pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:44:49.962415 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:49.962283    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:44:49.962415 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:49.962328    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:44:49.962415 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:49.962353    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7d1f1fc-4f58-4499-a42b-1f9325efa292-original-pull-secret podName:b7d1f1fc-4f58-4499-a42b-1f9325efa292 nodeName:}" failed. No retries permitted until 2026-09-16 18:44:51.962332869 +0000 UTC m=+6.039437716 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b7d1f1fc-4f58-4499-a42b-1f9325efa292-original-pull-secret") pod "global-pull-secret-syncer-c2j6m" (UID: "b7d1f1fc-4f58-4499-a42b-1f9325efa292") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:44:49.962415 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:49.962396    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c19f514e-c3a8-41d2-9de1-ad2d86fce986-metrics-certs podName:c19f514e-c3a8-41d2-9de1-ad2d86fce986 nodeName:}" failed. No retries permitted until 2026-09-16 18:44:51.962377978 +0000 UTC m=+6.039482819 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c19f514e-c3a8-41d2-9de1-ad2d86fce986-metrics-certs") pod "network-metrics-daemon-4wnsv" (UID: "c19f514e-c3a8-41d2-9de1-ad2d86fce986") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:44:50.063812 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:50.063164    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl62r\" (UniqueName: \"kubernetes.io/projected/4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf-kube-api-access-pl62r\") pod \"network-check-target-gtj9s\" (UID: \"4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf\") " pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:44:50.063812 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:50.063346    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:44:50.063812 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:50.063371    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:44:50.063812 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:50.063383    2557 projected.go:196] Error preparing data for projected volume kube-api-access-pl62r for pod openshift-network-diagnostics/network-check-target-gtj9s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:44:50.063812 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:50.063445    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf-kube-api-access-pl62r podName:4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf nodeName:}" failed. No retries permitted until 2026-09-16 18:44:52.063425276 +0000 UTC m=+6.140530133 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pl62r" (UniqueName: "kubernetes.io/projected/4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf-kube-api-access-pl62r") pod "network-check-target-gtj9s" (UID: "4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:44:50.173226 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:50.173191    2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:44:50.388721 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:50.388653    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 18:39:49 +0000 UTC" deadline="2028-02-28 04:53:00.852167237 +0000 UTC"
Sep 16 18:44:50.388721 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:50.388720    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12706h8m10.463453238s"
Sep 16 18:44:50.469350 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:50.469317    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:44:50.469527 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:50.469450    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtj9s" podUID="4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf"
Sep 16 18:44:50.469909 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:50.469886    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:44:50.470031 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:50.470012    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wnsv" podUID="c19f514e-c3a8-41d2-9de1-ad2d86fce986"
Sep 16 18:44:50.483825 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:50.483790    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tvwkw" event={"ID":"030fd847-6677-48f0-8e6b-e30f2e63d1a8","Type":"ContainerStarted","Data":"3abc37720d5c5b9f02aa17811525f715571ecd9587cd82a934c4fd92e4308433"}
Sep 16 18:44:50.490252 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:50.490225    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6g9b9" event={"ID":"87795909-0ae4-4163-aef4-bb1e143885da","Type":"ContainerStarted","Data":"99b294fbecc31a87f84f889b6c50334a16ee6ec4d925a8f87db311459508d281"}
Sep 16 18:44:50.514484 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:50.514450    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-shx66" event={"ID":"837f7ec9-d8aa-45b3-bebc-2eb825742375","Type":"ContainerStarted","Data":"1074eebe42b4d39e88f2b72ddc8e184ab89811ab47cfc9aeb59f5058e738ac40"}
Sep 16 18:44:50.523532 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:50.522915    2557 generic.go:356] "Generic (PLEG): container finished" podID="9ceb312c45138c203c60454ce1db5dde" containerID="9e6960b7bb2e5e7c6fb66e4083e8b023e1956ee5d5ff3ceb48efcf4d6a437589" exitCode=0
Sep 16 18:44:50.523532 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:50.523200    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal" event={"ID":"9ceb312c45138c203c60454ce1db5dde","Type":"ContainerDied","Data":"9e6960b7bb2e5e7c6fb66e4083e8b023e1956ee5d5ff3ceb48efcf4d6a437589"}
Sep 16 18:44:50.536376 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:50.536350    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4rzjs" event={"ID":"55642348-e5c2-42ed-af38-d0631bd164aa","Type":"ContainerStarted","Data":"710ee9e8e7579d795b42ec0e8dc5d8403eba4d4d610901bdf6095e471d4cfd87"}
Sep 16 18:44:50.552160 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:50.552072    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zklcc" event={"ID":"f03fb6fa-7cba-4886-958a-81e902f11566","Type":"ContainerStarted","Data":"299e569047b2b674fe8b0ded122bbe9bf4de0a10cd89901ee6b6ae745d329d0e"}
Sep 16 18:44:50.569408 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:50.569069    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-q6g22" event={"ID":"5c6f6f8b-e6fc-4a16-b084-7e851b3c3cc6","Type":"ContainerStarted","Data":"3aeee5f5921f8a8947f6cf5011dbcd299cf15ef4e1ce78c33520b372ee173eb7"}
Sep 16 18:44:50.599502 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:50.599476    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-26.ec2.internal" event={"ID":"26c1c5215cb05aa10e9d3bcd9724a09a","Type":"ContainerStarted","Data":"ac889097ed3e5f130514c9316014a908e45d5849743ff10ded2c9ebc30223535"}
Sep 16 18:44:50.614650 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:50.614618    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t" event={"ID":"c85031e5-5e03-4f66-b61c-636789b6e44f","Type":"ContainerStarted","Data":"ef56de63ab1e3a0403b917c2c4939950303d481815291357d28067b8bd20eee9"}
Sep 16 18:44:50.619929 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:50.619181    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-26.ec2.internal" podStartSLOduration=2.619164557 podStartE2EDuration="2.619164557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:44:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:44:50.618799489 +0000 UTC m=+4.695904355" watchObservedRunningTime="2026-09-16 18:44:50.619164557 +0000 UTC m=+4.696269424"
Sep 16 18:44:50.630041 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:50.630012    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-r9g58" event={"ID":"4c1c55c5-0ed8-46eb-a61d-381273852b77","Type":"ContainerStarted","Data":"ab1e2a671d9ea3621175539c81c3dc875053db695e831761ade7a223954e2aef"}
Sep 16 18:44:50.642218 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:50.642149    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wfkgg" event={"ID":"0307bb60-33a4-4e04-a80e-6e280e7217df","Type":"ContainerStarted","Data":"89845e3e0672856c27167b2e08670da00584cfbbcbdedeff6303bb0be7f1bed6"}
Sep 16 18:44:51.461520 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:51.461482    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:44:51.462186 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:51.461632    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2j6m" podUID="b7d1f1fc-4f58-4499-a42b-1f9325efa292"
Sep 16 18:44:51.683986 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:51.683809    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal" event={"ID":"9ceb312c45138c203c60454ce1db5dde","Type":"ContainerStarted","Data":"cea75c93220ac37cf0ccca1ca4611ab202eb907a0246ff16776279407c21a1db"}
Sep 16 18:44:51.981969 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:51.980845    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7d1f1fc-4f58-4499-a42b-1f9325efa292-original-pull-secret\") pod \"global-pull-secret-syncer-c2j6m\" (UID: \"b7d1f1fc-4f58-4499-a42b-1f9325efa292\") " pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:44:51.981969 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:51.981006    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c19f514e-c3a8-41d2-9de1-ad2d86fce986-metrics-certs\") pod \"network-metrics-daemon-4wnsv\" (UID: \"c19f514e-c3a8-41d2-9de1-ad2d86fce986\") " pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:44:51.981969 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:51.981234    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:44:51.981969 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:51.981308    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c19f514e-c3a8-41d2-9de1-ad2d86fce986-metrics-certs podName:c19f514e-c3a8-41d2-9de1-ad2d86fce986 nodeName:}" failed. No retries permitted until 2026-09-16 18:44:55.981286899 +0000 UTC m=+10.058391744 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c19f514e-c3a8-41d2-9de1-ad2d86fce986-metrics-certs") pod "network-metrics-daemon-4wnsv" (UID: "c19f514e-c3a8-41d2-9de1-ad2d86fce986") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:44:51.981969 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:51.981806    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:44:51.981969 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:51.981878    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7d1f1fc-4f58-4499-a42b-1f9325efa292-original-pull-secret podName:b7d1f1fc-4f58-4499-a42b-1f9325efa292 nodeName:}" failed. No retries permitted until 2026-09-16 18:44:55.98184396 +0000 UTC m=+10.058948824 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b7d1f1fc-4f58-4499-a42b-1f9325efa292-original-pull-secret") pod "global-pull-secret-syncer-c2j6m" (UID: "b7d1f1fc-4f58-4499-a42b-1f9325efa292") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:44:52.082038 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:52.081992    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl62r\" (UniqueName: \"kubernetes.io/projected/4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf-kube-api-access-pl62r\") pod \"network-check-target-gtj9s\" (UID: \"4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf\") " pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:44:52.082222 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:52.082177    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:44:52.082222 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:52.082197    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:44:52.082222 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:52.082210    2557 projected.go:196] Error preparing data for projected volume kube-api-access-pl62r for pod openshift-network-diagnostics/network-check-target-gtj9s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:44:52.082422 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:52.082267    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf-kube-api-access-pl62r podName:4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf nodeName:}" failed. No retries permitted until 2026-09-16 18:44:56.082249109 +0000 UTC m=+10.159353954 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pl62r" (UniqueName: "kubernetes.io/projected/4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf-kube-api-access-pl62r") pod "network-check-target-gtj9s" (UID: "4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:44:52.461146 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:52.461114    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:44:52.461311 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:52.461243    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtj9s" podUID="4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf"
Sep 16 18:44:52.461761 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:52.461738    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:44:52.462306 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:52.461921    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wnsv" podUID="c19f514e-c3a8-41d2-9de1-ad2d86fce986"
Sep 16 18:44:53.461152 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:53.461118    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:44:53.461338 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:53.461279    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2j6m" podUID="b7d1f1fc-4f58-4499-a42b-1f9325efa292"
Sep 16 18:44:54.460709 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:54.460671    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:44:54.461169 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:54.460849    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wnsv" podUID="c19f514e-c3a8-41d2-9de1-ad2d86fce986"
Sep 16 18:44:54.461441 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:54.461273    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:44:54.461441 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:54.461382    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtj9s" podUID="4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf"
Sep 16 18:44:55.460927 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:55.460849    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:44:55.461430 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:55.461024    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2j6m" podUID="b7d1f1fc-4f58-4499-a42b-1f9325efa292"
Sep 16 18:44:56.017794 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:56.017747    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c19f514e-c3a8-41d2-9de1-ad2d86fce986-metrics-certs\") pod \"network-metrics-daemon-4wnsv\" (UID: \"c19f514e-c3a8-41d2-9de1-ad2d86fce986\") " pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:44:56.017978 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:56.017829    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7d1f1fc-4f58-4499-a42b-1f9325efa292-original-pull-secret\") pod \"global-pull-secret-syncer-c2j6m\" (UID: \"b7d1f1fc-4f58-4499-a42b-1f9325efa292\") " pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:44:56.017978 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:56.017902    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:44:56.018117 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:56.017985    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c19f514e-c3a8-41d2-9de1-ad2d86fce986-metrics-certs podName:c19f514e-c3a8-41d2-9de1-ad2d86fce986 nodeName:}" failed. No retries permitted until 2026-09-16 18:45:04.017962926 +0000 UTC m=+18.095067771 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c19f514e-c3a8-41d2-9de1-ad2d86fce986-metrics-certs") pod "network-metrics-daemon-4wnsv" (UID: "c19f514e-c3a8-41d2-9de1-ad2d86fce986") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:44:56.018117 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:56.017986    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:44:56.018117 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:56.018043    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7d1f1fc-4f58-4499-a42b-1f9325efa292-original-pull-secret podName:b7d1f1fc-4f58-4499-a42b-1f9325efa292 nodeName:}" failed. No retries permitted until 2026-09-16 18:45:04.018025973 +0000 UTC m=+18.095130830 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b7d1f1fc-4f58-4499-a42b-1f9325efa292-original-pull-secret") pod "global-pull-secret-syncer-c2j6m" (UID: "b7d1f1fc-4f58-4499-a42b-1f9325efa292") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:44:56.119007 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:56.118968    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl62r\" (UniqueName: \"kubernetes.io/projected/4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf-kube-api-access-pl62r\") pod \"network-check-target-gtj9s\" (UID: \"4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf\") " pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:44:56.119194 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:56.119176    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:44:56.119282 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:56.119201    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:44:56.119282 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:56.119214    2557 projected.go:196] Error preparing data for projected volume kube-api-access-pl62r for pod openshift-network-diagnostics/network-check-target-gtj9s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:44:56.119282 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:56.119281    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf-kube-api-access-pl62r podName:4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf nodeName:}" failed. No retries permitted until 2026-09-16 18:45:04.11926234 +0000 UTC m=+18.196367206 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pl62r" (UniqueName: "kubernetes.io/projected/4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf-kube-api-access-pl62r") pod "network-check-target-gtj9s" (UID: "4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:44:56.462478 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:56.461879    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:44:56.462478 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:56.461914    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:44:56.462478 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:56.462008    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wnsv" podUID="c19f514e-c3a8-41d2-9de1-ad2d86fce986"
Sep 16 18:44:56.462478 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:56.462260    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtj9s" podUID="4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf"
Sep 16 18:44:57.460908 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:57.460715    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:44:57.460908 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:57.460830    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2j6m" podUID="b7d1f1fc-4f58-4499-a42b-1f9325efa292"
Sep 16 18:44:58.464383 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:58.464248    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:44:58.465091 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:58.464854    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtj9s" podUID="4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf"
Sep 16 18:44:58.465091 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:58.464954    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:44:58.465091 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:58.465055    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wnsv" podUID="c19f514e-c3a8-41d2-9de1-ad2d86fce986"
Sep 16 18:44:59.460918 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:44:59.460668    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:44:59.461076 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:44:59.461015    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2j6m" podUID="b7d1f1fc-4f58-4499-a42b-1f9325efa292"
Sep 16 18:45:00.461263 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:00.461228    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:45:00.461657 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:00.461347    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wnsv" podUID="c19f514e-c3a8-41d2-9de1-ad2d86fce986"
Sep 16 18:45:00.461657 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:00.461399    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:45:00.461657 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:00.461481    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtj9s" podUID="4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf"
Sep 16 18:45:01.460564 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:01.460524    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:45:01.460713 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:01.460657    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2j6m" podUID="b7d1f1fc-4f58-4499-a42b-1f9325efa292"
Sep 16 18:45:02.460576 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:02.460532    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:45:02.461019 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:02.460667    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wnsv" podUID="c19f514e-c3a8-41d2-9de1-ad2d86fce986"
Sep 16 18:45:02.461019 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:02.460717    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:45:02.461019 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:02.460846    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtj9s" podUID="4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf"
Sep 16 18:45:03.461105 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:03.461073    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:45:03.461585 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:03.461176    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2j6m" podUID="b7d1f1fc-4f58-4499-a42b-1f9325efa292"
Sep 16 18:45:04.084512 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:04.084472    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c19f514e-c3a8-41d2-9de1-ad2d86fce986-metrics-certs\") pod \"network-metrics-daemon-4wnsv\" (UID: \"c19f514e-c3a8-41d2-9de1-ad2d86fce986\") " pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:45:04.084709 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:04.084537    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7d1f1fc-4f58-4499-a42b-1f9325efa292-original-pull-secret\") pod \"global-pull-secret-syncer-c2j6m\" (UID: \"b7d1f1fc-4f58-4499-a42b-1f9325efa292\") " pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:45:04.084709 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:04.084650    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:45:04.084856 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:04.084728    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c19f514e-c3a8-41d2-9de1-ad2d86fce986-metrics-certs podName:c19f514e-c3a8-41d2-9de1-ad2d86fce986 nodeName:}" failed. No retries permitted until 2026-09-16 18:45:20.084709643 +0000 UTC m=+34.161814499 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c19f514e-c3a8-41d2-9de1-ad2d86fce986-metrics-certs") pod "network-metrics-daemon-4wnsv" (UID: "c19f514e-c3a8-41d2-9de1-ad2d86fce986") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:45:04.084856 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:04.084653    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:45:04.084856 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:04.084820    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7d1f1fc-4f58-4499-a42b-1f9325efa292-original-pull-secret podName:b7d1f1fc-4f58-4499-a42b-1f9325efa292 nodeName:}" failed. No retries permitted until 2026-09-16 18:45:20.084802756 +0000 UTC m=+34.161907610 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b7d1f1fc-4f58-4499-a42b-1f9325efa292-original-pull-secret") pod "global-pull-secret-syncer-c2j6m" (UID: "b7d1f1fc-4f58-4499-a42b-1f9325efa292") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:45:04.185193 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:04.185157    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl62r\" (UniqueName: \"kubernetes.io/projected/4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf-kube-api-access-pl62r\") pod \"network-check-target-gtj9s\" (UID: \"4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf\") " pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:45:04.185365 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:04.185303    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:45:04.185365 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:04.185325    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:45:04.185365 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:04.185336    2557 projected.go:196] Error preparing data for projected volume kube-api-access-pl62r for pod openshift-network-diagnostics/network-check-target-gtj9s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:45:04.185529 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:04.185392    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf-kube-api-access-pl62r podName:4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf nodeName:}" failed. No retries permitted until 2026-09-16 18:45:20.185374908 +0000 UTC m=+34.262479763 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pl62r" (UniqueName: "kubernetes.io/projected/4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf-kube-api-access-pl62r") pod "network-check-target-gtj9s" (UID: "4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:45:04.460785 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:04.460681    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:45:04.460785 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:04.460733    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:45:04.460979 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:04.460838    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtj9s" podUID="4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf"
Sep 16 18:45:04.460979 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:04.460957    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wnsv" podUID="c19f514e-c3a8-41d2-9de1-ad2d86fce986"
Sep 16 18:45:05.461000 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:05.460961    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:45:05.461407 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:05.461120    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2j6m" podUID="b7d1f1fc-4f58-4499-a42b-1f9325efa292"
Sep 16 18:45:06.461896 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:06.461856    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:45:06.462359 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:06.461971    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wnsv" podUID="c19f514e-c3a8-41d2-9de1-ad2d86fce986"
Sep 16 18:45:06.462359 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:06.462020    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:45:06.462359 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:06.462130    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtj9s" podUID="4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf"
Sep 16 18:45:07.461388 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:07.461358    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:45:07.461550 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:07.461472    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2j6m" podUID="b7d1f1fc-4f58-4499-a42b-1f9325efa292"
Sep 16 18:45:08.460981 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:08.460827    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:45:08.461612 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:08.460826    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:45:08.461612 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:08.461078    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtj9s" podUID="4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf"
Sep 16 18:45:08.461612 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:08.461123    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wnsv" podUID="c19f514e-c3a8-41d2-9de1-ad2d86fce986"
Sep 16 18:45:08.727716 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:08.727675    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t" event={"ID":"c85031e5-5e03-4f66-b61c-636789b6e44f","Type":"ContainerStarted","Data":"add994aca8e8341d17fe297b12e87981d67ce8ce4518c8166ef6e8b8dc6a9c98"}
Sep 16 18:45:08.727848 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:08.727827    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t" event={"ID":"c85031e5-5e03-4f66-b61c-636789b6e44f","Type":"ContainerStarted","Data":"5bff4ad94a38a66094dee76c5af95513d20faf0d33e0e6cd7362039ad32d1525"}
Sep 16 18:45:08.727911 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:08.727890    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t" event={"ID":"c85031e5-5e03-4f66-b61c-636789b6e44f","Type":"ContainerStarted","Data":"3f6e40663d5aa2f9fc301b2d156963eb1952e3cbbf842e2a7769b3f2597e2edc"}
Sep 16 18:45:08.727962 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:08.727934    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t" event={"ID":"c85031e5-5e03-4f66-b61c-636789b6e44f","Type":"ContainerStarted","Data":"74660dd80a99d098d7cbf69b7d84d3bb37bae7354428c119a3192f7c4e071142"}
Sep 16 18:45:08.728009 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:08.727970    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t" event={"ID":"c85031e5-5e03-4f66-b61c-636789b6e44f","Type":"ContainerStarted","Data":"a550ffb29d450ac9487812dc5c4b96a12168f4a9677e937daab613871f8ae8de"}
Sep 16 18:45:08.729285 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:08.729261    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-r9g58" event={"ID":"4c1c55c5-0ed8-46eb-a61d-381273852b77","Type":"ContainerStarted","Data":"613937c1e6c02a01cdcc87f31db7222edfaf3ce3909990fd8ce115a8fbc72fdf"}
Sep 16 18:45:08.730735 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:08.730713    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wfkgg" event={"ID":"0307bb60-33a4-4e04-a80e-6e280e7217df","Type":"ContainerStarted","Data":"a3bb8366d5f6037268405f08e99158a632bc91ec740e8b2cd5187d1d4ae5203f"}
Sep 16 18:45:08.732186 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:08.732164    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6g9b9" event={"ID":"87795909-0ae4-4163-aef4-bb1e143885da","Type":"ContainerStarted","Data":"541bab66af3c1a4fd8901c41356b328a319a6ded91ab0a77e840de1d34962b54"}
Sep 16 18:45:08.733794 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:08.733755    2557 generic.go:356] "Generic (PLEG): container finished" podID="837f7ec9-d8aa-45b3-bebc-2eb825742375" containerID="19caa4034514d9d74c4274a222427a4fbaefa43d677c77f99a9ca48f1c4163a7" exitCode=0
Sep 16 18:45:08.733900 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:08.733792    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-shx66" event={"ID":"837f7ec9-d8aa-45b3-bebc-2eb825742375","Type":"ContainerDied","Data":"19caa4034514d9d74c4274a222427a4fbaefa43d677c77f99a9ca48f1c4163a7"}
Sep 16 18:45:08.736085 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:08.736066    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4rzjs" event={"ID":"55642348-e5c2-42ed-af38-d0631bd164aa","Type":"ContainerStarted","Data":"556e850671801df0ab82870d4c6f3f9fbe30c2735f57680fa72a8b3714882ae2"}
Sep 16 18:45:08.737562 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:08.737539    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zklcc" event={"ID":"f03fb6fa-7cba-4886-958a-81e902f11566","Type":"ContainerStarted","Data":"410b49b369d73ad3513a39445e6d78f2e19a77ddb1dd33422e1ea5c6a40ed3bf"}
Sep 16 18:45:08.738983 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:08.738962    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-q6g22" event={"ID":"5c6f6f8b-e6fc-4a16-b084-7e851b3c3cc6","Type":"ContainerStarted","Data":"d045b7c4d5e7deccbb41bb48c0a354f113165f1f3fed31f3b723c790dbd211bb"}
Sep 16 18:45:08.759313 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:08.759274    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-26.ec2.internal" podStartSLOduration=20.759262687 podStartE2EDuration="20.759262687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:44:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:44:51.705592366 +0000 UTC m=+5.782697232" watchObservedRunningTime="2026-09-16 18:45:08.759262687 +0000 UTC m=+22.836367545"
Sep 16 18:45:08.759596 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:08.759549    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-r9g58" podStartSLOduration=10.380455775 podStartE2EDuration="21.759541995s" totalImagesPullingTime="11.37908622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:44:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:45:08.759030813 +0000 UTC m=+22.836135672" watchObservedRunningTime="2026-09-16 18:45:08.759541995 +0000 UTC m=+22.836646855"
Sep 16 18:45:08.813717 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:08.813666    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-6g9b9" podStartSLOduration=3.178021621 podStartE2EDuration="21.813653762s" totalImagesPullingTime="18.635632141s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:44:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:45:08.798529884 +0000 UTC m=+22.875634744" watchObservedRunningTime="2026-09-16 18:45:08.813653762 +0000 UTC m=+22.890758664"
Sep 16 18:45:08.904418 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:08.904323    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zklcc" podStartSLOduration=3.307637025 podStartE2EDuration="21.904307248s" totalImagesPullingTime="18.596670223s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:44:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:45:08.854627308 +0000 UTC m=+22.931732168" watchObservedRunningTime="2026-09-16 18:45:08.904307248 +0000 UTC m=+22.981412107"
Sep 16 18:45:08.958176 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:08.958116    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4rzjs" podStartSLOduration=3.385636515 podStartE2EDuration="21.95809608s" totalImagesPullingTime="18.572459565s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:44:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:45:08.904741773 +0000 UTC m=+22.981846634" watchObservedRunningTime="2026-09-16 18:45:08.95809608 +0000 UTC m=+23.035200943"
Sep 16 18:45:08.958332 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:08.958209    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-q6g22" podStartSLOduration=3.370116513 podStartE2EDuration="21.958203798s" totalImagesPullingTime="18.588087285s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:44:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:45:08.958046377 +0000 UTC m=+23.035151239" watchObservedRunningTime="2026-09-16 18:45:08.958203798 +0000 UTC m=+23.035308663"
Sep 16 18:45:09.461349 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:09.461269    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:45:09.461947 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:09.461376    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2j6m" podUID="b7d1f1fc-4f58-4499-a42b-1f9325efa292"
Sep 16 18:45:09.743672 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:09.743578    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tvwkw" event={"ID":"030fd847-6677-48f0-8e6b-e30f2e63d1a8","Type":"ContainerStarted","Data":"8e4e33441310e12ae57651f6b97ba705f3ba082975a2059e727f3040982b11fd"}
Sep 16 18:45:09.747178 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:09.747150    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t" event={"ID":"c85031e5-5e03-4f66-b61c-636789b6e44f","Type":"ContainerStarted","Data":"9d63b428d7ab18e9380b184ab7d42a14d7bcc82f9e156318ea9ae6659e906703"}
Sep 16 18:45:09.929986 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:09.929951    2557 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 16 18:45:10.415207 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:10.415094    2557 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-16T18:45:09.929977141Z","UUID":"80e7a3c2-64d9-4aae-a9aa-9e99c5e5a2a6","Handler":null,"Name":"","Endpoint":""}
Sep 16 18:45:10.416798 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:10.416765    2557 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 16 18:45:10.416918 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:10.416812    2557 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 16 18:45:10.461216 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:10.461187    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:45:10.461395 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:10.461186    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:45:10.461395 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:10.461313    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtj9s" podUID="4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf"
Sep 16 18:45:10.461395 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:10.461368    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wnsv" podUID="c19f514e-c3a8-41d2-9de1-ad2d86fce986"
Sep 16 18:45:10.752186 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:10.752048    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wfkgg" event={"ID":"0307bb60-33a4-4e04-a80e-6e280e7217df","Type":"ContainerStarted","Data":"7c90dbcb7a56047b6168bd568b7f58df616b76a7dc638f1fdfbc2ce6479e1541"}
Sep 16 18:45:11.460609 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:11.460574    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:45:11.460810 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:11.460694    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2j6m" podUID="b7d1f1fc-4f58-4499-a42b-1f9325efa292"
Sep 16 18:45:11.758293 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:11.758212    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t" event={"ID":"c85031e5-5e03-4f66-b61c-636789b6e44f","Type":"ContainerStarted","Data":"8a406b6cb1e47e064d26d03ef68a9050f8d9497870ea98e092aa12ca08de25e6"}
Sep 16 18:45:12.249696 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:12.249660    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-r9g58"
Sep 16 18:45:12.250303 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:12.250277    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-r9g58"
Sep 16 18:45:12.266720 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:12.266672    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tvwkw" podStartSLOduration=6.691135865 podStartE2EDuration="25.266658279s" totalImagesPullingTime="18.575522414s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:44:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:45:09.762725061 +0000 UTC m=+23.839829926" watchObservedRunningTime="2026-09-16 18:45:12.266658279 +0000 UTC m=+26.343763138"
Sep 16 18:45:12.461449 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:12.461419    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:45:12.461595 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:12.461559    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wnsv" podUID="c19f514e-c3a8-41d2-9de1-ad2d86fce986"
Sep 16 18:45:12.461668 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:12.461611    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:45:12.461744 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:12.461724    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtj9s" podUID="4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf"
Sep 16 18:45:12.760049 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:12.760021    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-r9g58"
Sep 16 18:45:12.760491 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:12.760474    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-r9g58"
Sep 16 18:45:13.460814 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:13.460628    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:45:13.460990 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:13.460879    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2j6m" podUID="b7d1f1fc-4f58-4499-a42b-1f9325efa292"
Sep 16 18:45:13.764266 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:13.764186    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wfkgg" event={"ID":"0307bb60-33a4-4e04-a80e-6e280e7217df","Type":"ContainerStarted","Data":"02af79cacb21a1103741e2c14084a9514adc6bf7f64cbc63190693773aa4d374"}
Sep 16 18:45:13.766020 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:13.765997    2557 generic.go:356] "Generic (PLEG): container finished" podID="837f7ec9-d8aa-45b3-bebc-2eb825742375" containerID="a02ecf015bda72e0f09513166950323a6962c52fe83c3de635cb91e336f4aa59" exitCode=0
Sep 16 18:45:13.766124 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:13.766081    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-shx66" event={"ID":"837f7ec9-d8aa-45b3-bebc-2eb825742375","Type":"ContainerDied","Data":"a02ecf015bda72e0f09513166950323a6962c52fe83c3de635cb91e336f4aa59"}
Sep 16 18:45:13.770026 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:13.769999    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t" event={"ID":"c85031e5-5e03-4f66-b61c-636789b6e44f","Type":"ContainerStarted","Data":"e52449d700dc509658367b0dff06ce84fb16e59d8f14c2dca1b2ff8466cdd525"}
Sep 16 18:45:13.788677 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:13.788641    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wfkgg" podStartSLOduration=3.949399965 podStartE2EDuration="26.788630123s" totalImagesPullingTime="22.839230158s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:44:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:45:13.78719611 +0000 UTC m=+27.864300993" watchObservedRunningTime="2026-09-16 18:45:13.788630123 +0000 UTC m=+27.865734979"
Sep 16 18:45:13.854434 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:13.854397    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t" podStartSLOduration=8.224566304 podStartE2EDuration="26.854385131s" totalImagesPullingTime="18.629818827s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:44:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:45:13.830805968 +0000 UTC m=+27.907910827" watchObservedRunningTime="2026-09-16 18:45:13.854385131 +0000 UTC m=+27.931489990"
Sep 16 18:45:14.461433 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:14.461406    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:45:14.461585 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:14.461406    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:45:14.461585 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:14.461554    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wnsv" podUID="c19f514e-c3a8-41d2-9de1-ad2d86fce986"
Sep 16 18:45:14.461748 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:14.461593    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtj9s" podUID="4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf"
Sep 16 18:45:14.774269 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:14.774202    2557 generic.go:356] "Generic (PLEG): container finished" podID="837f7ec9-d8aa-45b3-bebc-2eb825742375" containerID="436dc018d5356d7e8b4b9f25b31378702e8d3ae2f5dd4955a97c64ee45514fcc" exitCode=0
Sep 16 18:45:14.775028 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:14.774307    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-shx66" event={"ID":"837f7ec9-d8aa-45b3-bebc-2eb825742375","Type":"ContainerDied","Data":"436dc018d5356d7e8b4b9f25b31378702e8d3ae2f5dd4955a97c64ee45514fcc"}
Sep 16 18:45:14.775098 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:14.775078    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:45:14.775168 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:14.775156    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:45:14.775227 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:14.775205    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:45:14.791607 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:14.791586    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:45:14.791805 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:14.791793    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:45:15.119905 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:15.119849    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gtj9s"]
Sep 16 18:45:15.120091 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:15.120015    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:45:15.120163 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:15.120126    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtj9s" podUID="4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf"
Sep 16 18:45:15.122607 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:15.122565    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-c2j6m"]
Sep 16 18:45:15.122728 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:15.122711    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:45:15.122875 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:15.122848    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2j6m" podUID="b7d1f1fc-4f58-4499-a42b-1f9325efa292"
Sep 16 18:45:15.123168 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:15.123150    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4wnsv"]
Sep 16 18:45:15.123254 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:15.123246    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:45:15.123361 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:15.123345    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wnsv" podUID="c19f514e-c3a8-41d2-9de1-ad2d86fce986"
Sep 16 18:45:15.779087 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:15.779060    2557 generic.go:356] "Generic (PLEG): container finished" podID="837f7ec9-d8aa-45b3-bebc-2eb825742375" containerID="45e89a84418c1be3f9da25db183caa616eac4888cc275dafc7e441a3e87363e1" exitCode=0
Sep 16 18:45:15.779454 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:15.779112    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-shx66" event={"ID":"837f7ec9-d8aa-45b3-bebc-2eb825742375","Type":"ContainerDied","Data":"45e89a84418c1be3f9da25db183caa616eac4888cc275dafc7e441a3e87363e1"}
Sep 16 18:45:16.461900 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:16.461869    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:45:16.462057 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:16.462001    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wnsv" podUID="c19f514e-c3a8-41d2-9de1-ad2d86fce986"
Sep 16 18:45:16.462150 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:16.462125    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:45:16.462250 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:16.462226    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtj9s" podUID="4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf"
Sep 16 18:45:17.461243 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:17.461166    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:45:17.461571 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:17.461282    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2j6m" podUID="b7d1f1fc-4f58-4499-a42b-1f9325efa292"
Sep 16 18:45:18.460779 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:18.460745    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:45:18.460954 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:18.460882    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtj9s" podUID="4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf"
Sep 16 18:45:18.461033 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:18.460949    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:45:18.461114 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:18.461084    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wnsv" podUID="c19f514e-c3a8-41d2-9de1-ad2d86fce986"
Sep 16 18:45:19.461310 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:19.461279    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:45:19.461806 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:19.461391    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2j6m" podUID="b7d1f1fc-4f58-4499-a42b-1f9325efa292"
Sep 16 18:45:20.098644 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:20.098617    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7d1f1fc-4f58-4499-a42b-1f9325efa292-original-pull-secret\") pod \"global-pull-secret-syncer-c2j6m\" (UID: \"b7d1f1fc-4f58-4499-a42b-1f9325efa292\") " pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:45:20.098759 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:20.098697    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c19f514e-c3a8-41d2-9de1-ad2d86fce986-metrics-certs\") pod \"network-metrics-daemon-4wnsv\" (UID: \"c19f514e-c3a8-41d2-9de1-ad2d86fce986\") " pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:45:20.098759 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:20.098748    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:45:20.098899 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:20.098821    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7d1f1fc-4f58-4499-a42b-1f9325efa292-original-pull-secret podName:b7d1f1fc-4f58-4499-a42b-1f9325efa292 nodeName:}" failed. No retries permitted until 2026-09-16 18:45:52.098806396 +0000 UTC m=+66.175911253 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b7d1f1fc-4f58-4499-a42b-1f9325efa292-original-pull-secret") pod "global-pull-secret-syncer-c2j6m" (UID: "b7d1f1fc-4f58-4499-a42b-1f9325efa292") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:45:20.098899 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:20.098821    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:45:20.098899 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:20.098874    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c19f514e-c3a8-41d2-9de1-ad2d86fce986-metrics-certs podName:c19f514e-c3a8-41d2-9de1-ad2d86fce986 nodeName:}" failed. No retries permitted until 2026-09-16 18:45:52.098858371 +0000 UTC m=+66.175963232 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c19f514e-c3a8-41d2-9de1-ad2d86fce986-metrics-certs") pod "network-metrics-daemon-4wnsv" (UID: "c19f514e-c3a8-41d2-9de1-ad2d86fce986") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:45:20.199984 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:20.199959    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl62r\" (UniqueName: \"kubernetes.io/projected/4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf-kube-api-access-pl62r\") pod \"network-check-target-gtj9s\" (UID: \"4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf\") " pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:45:20.200143 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:20.200125    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:45:20.200184 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:20.200149    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:45:20.200184 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:20.200159    2557 projected.go:196] Error preparing data for projected volume kube-api-access-pl62r for pod openshift-network-diagnostics/network-check-target-gtj9s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:45:20.200249 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:20.200205    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf-kube-api-access-pl62r podName:4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf nodeName:}" failed. No retries permitted until 2026-09-16 18:45:52.200193287 +0000 UTC m=+66.277298125 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-pl62r" (UniqueName: "kubernetes.io/projected/4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf-kube-api-access-pl62r") pod "network-check-target-gtj9s" (UID: "4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:45:20.461272 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:20.461244    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:45:20.461535 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:20.461370    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wnsv" podUID="c19f514e-c3a8-41d2-9de1-ad2d86fce986"
Sep 16 18:45:20.461535 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:20.461435    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:45:20.461535 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:20.461513    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtj9s" podUID="4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf"
Sep 16 18:45:20.792875 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:20.792845    2557 generic.go:356] "Generic (PLEG): container finished" podID="837f7ec9-d8aa-45b3-bebc-2eb825742375" containerID="418021ee666fd8d3b5a81e9cb28a1eeb916c0e2db4c2aaac8d3168733224633b" exitCode=0
Sep 16 18:45:20.793044 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:20.792904    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-shx66" event={"ID":"837f7ec9-d8aa-45b3-bebc-2eb825742375","Type":"ContainerDied","Data":"418021ee666fd8d3b5a81e9cb28a1eeb916c0e2db4c2aaac8d3168733224633b"}
Sep 16 18:45:21.460851 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:21.460752    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:45:21.461014 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:21.460882    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c2j6m" podUID="b7d1f1fc-4f58-4499-a42b-1f9325efa292"
Sep 16 18:45:21.798274 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:21.798090    2557 generic.go:356] "Generic (PLEG): container finished" podID="837f7ec9-d8aa-45b3-bebc-2eb825742375" containerID="d42f676f8ee44f4bfd0d08417402ec8ffadd4cfe83418f9bac6653f7bb8c27c3" exitCode=0
Sep 16 18:45:21.798597 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:21.798153    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-shx66" event={"ID":"837f7ec9-d8aa-45b3-bebc-2eb825742375","Type":"ContainerDied","Data":"d42f676f8ee44f4bfd0d08417402ec8ffadd4cfe83418f9bac6653f7bb8c27c3"}
Sep 16 18:45:22.273058 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.272977    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-26.ec2.internal" event="NodeReady"
Sep 16 18:45:22.273219 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.273125    2557 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 16 18:45:22.352550 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.352514    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-sdw9s"]
Sep 16 18:45:22.375094 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.375053    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-58tb5"]
Sep 16 18:45:22.375237 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.375132    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sdw9s"
Sep 16 18:45:22.377924 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.377901    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 16 18:45:22.378035 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.377907    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-g84ch\""
Sep 16 18:45:22.378352 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.378333    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 16 18:45:22.395029 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.395010    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sdw9s"]
Sep 16 18:45:22.395123 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.395042    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-58tb5"]
Sep 16 18:45:22.395123 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.395110    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-58tb5"
Sep 16 18:45:22.398300 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.398280    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 16 18:45:22.398397 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.398386    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 16 18:45:22.398446 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.398390    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9lmtn\""
Sep 16 18:45:22.398627 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.398612    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 16 18:45:22.460533 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.460493    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:45:22.460708 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.460499    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:45:22.464334 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.464317    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 18:45:22.464528 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.464515    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dgmmm\""
Sep 16 18:45:22.464927 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.464910    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 18:45:22.465031 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.464915    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 18:45:22.465397 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.465381    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pf72n\""
Sep 16 18:45:22.517896 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.517861    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-metrics-tls\") pod \"dns-default-sdw9s\" (UID: \"8ac962d8-4a04-40f2-a02c-ad61a26d1fbd\") " pod="openshift-dns/dns-default-sdw9s"
Sep 16 18:45:22.518052 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.517907    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-tmp-dir\") pod \"dns-default-sdw9s\" (UID: \"8ac962d8-4a04-40f2-a02c-ad61a26d1fbd\") " pod="openshift-dns/dns-default-sdw9s"
Sep 16 18:45:22.518052 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.517926    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbxqd\" (UniqueName: \"kubernetes.io/projected/2c43af6d-a516-4fc1-8f99-9a99fad657e8-kube-api-access-mbxqd\") pod \"ingress-canary-58tb5\" (UID: \"2c43af6d-a516-4fc1-8f99-9a99fad657e8\") " pod="openshift-ingress-canary/ingress-canary-58tb5"
Sep 16 18:45:22.518052 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.517973    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-config-volume\") pod \"dns-default-sdw9s\" (UID: \"8ac962d8-4a04-40f2-a02c-ad61a26d1fbd\") " pod="openshift-dns/dns-default-sdw9s"
Sep 16 18:45:22.518052 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.518021    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5fm4\" (UniqueName: \"kubernetes.io/projected/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-kube-api-access-s5fm4\") pod \"dns-default-sdw9s\" (UID: \"8ac962d8-4a04-40f2-a02c-ad61a26d1fbd\") " pod="openshift-dns/dns-default-sdw9s"
Sep 16 18:45:22.518052 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.518044    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c43af6d-a516-4fc1-8f99-9a99fad657e8-cert\") pod \"ingress-canary-58tb5\" (UID: \"2c43af6d-a516-4fc1-8f99-9a99fad657e8\") " pod="openshift-ingress-canary/ingress-canary-58tb5"
Sep 16 18:45:22.619197 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.619164    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-tmp-dir\") pod \"dns-default-sdw9s\" (UID: \"8ac962d8-4a04-40f2-a02c-ad61a26d1fbd\") " pod="openshift-dns/dns-default-sdw9s"
Sep 16 18:45:22.619197 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.619197    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbxqd\" (UniqueName: \"kubernetes.io/projected/2c43af6d-a516-4fc1-8f99-9a99fad657e8-kube-api-access-mbxqd\") pod \"ingress-canary-58tb5\" (UID: \"2c43af6d-a516-4fc1-8f99-9a99fad657e8\") " pod="openshift-ingress-canary/ingress-canary-58tb5"
Sep 16 18:45:22.619445 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.619227    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-config-volume\") pod \"dns-default-sdw9s\" (UID: \"8ac962d8-4a04-40f2-a02c-ad61a26d1fbd\") " pod="openshift-dns/dns-default-sdw9s"
Sep 16 18:45:22.619445 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.619253    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5fm4\" (UniqueName: \"kubernetes.io/projected/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-kube-api-access-s5fm4\") pod \"dns-default-sdw9s\" (UID: \"8ac962d8-4a04-40f2-a02c-ad61a26d1fbd\") " pod="openshift-dns/dns-default-sdw9s"
Sep 16 18:45:22.619445 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.619269    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c43af6d-a516-4fc1-8f99-9a99fad657e8-cert\") pod \"ingress-canary-58tb5\" (UID: \"2c43af6d-a516-4fc1-8f99-9a99fad657e8\") " pod="openshift-ingress-canary/ingress-canary-58tb5"
Sep 16 18:45:22.619445 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.619318    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-metrics-tls\") pod \"dns-default-sdw9s\" (UID: \"8ac962d8-4a04-40f2-a02c-ad61a26d1fbd\") " pod="openshift-dns/dns-default-sdw9s"
Sep 16 18:45:22.619445 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:22.619417    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:45:22.619656 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:22.619479    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-metrics-tls podName:8ac962d8-4a04-40f2-a02c-ad61a26d1fbd nodeName:}" failed. No retries permitted until 2026-09-16 18:45:23.119461331 +0000 UTC m=+37.196566171 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-metrics-tls") pod "dns-default-sdw9s" (UID: "8ac962d8-4a04-40f2-a02c-ad61a26d1fbd") : secret "dns-default-metrics-tls" not found
Sep 16 18:45:22.619720 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.619699    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-tmp-dir\") pod \"dns-default-sdw9s\" (UID: \"8ac962d8-4a04-40f2-a02c-ad61a26d1fbd\") " pod="openshift-dns/dns-default-sdw9s"
Sep 16 18:45:22.619939 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.619924    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-config-volume\") pod \"dns-default-sdw9s\" (UID: \"8ac962d8-4a04-40f2-a02c-ad61a26d1fbd\") " pod="openshift-dns/dns-default-sdw9s"
Sep 16 18:45:22.623428 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.623408    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c43af6d-a516-4fc1-8f99-9a99fad657e8-cert\") pod \"ingress-canary-58tb5\" (UID: \"2c43af6d-a516-4fc1-8f99-9a99fad657e8\") " pod="openshift-ingress-canary/ingress-canary-58tb5"
Sep 16 18:45:22.640565 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.640532    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbxqd\" (UniqueName: \"kubernetes.io/projected/2c43af6d-a516-4fc1-8f99-9a99fad657e8-kube-api-access-mbxqd\") pod \"ingress-canary-58tb5\" (UID: \"2c43af6d-a516-4fc1-8f99-9a99fad657e8\") " pod="openshift-ingress-canary/ingress-canary-58tb5"
Sep 16 18:45:22.641131 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.641111    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5fm4\" (UniqueName: \"kubernetes.io/projected/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-kube-api-access-s5fm4\") pod \"dns-default-sdw9s\" (UID: \"8ac962d8-4a04-40f2-a02c-ad61a26d1fbd\") " pod="openshift-dns/dns-default-sdw9s"
Sep 16 18:45:22.703075 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.703042    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-58tb5"
Sep 16 18:45:22.805352 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.805314    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-shx66" event={"ID":"837f7ec9-d8aa-45b3-bebc-2eb825742375","Type":"ContainerStarted","Data":"6273405f189a54134039c22beebad8457b2e3ef2f7643e82326ca5d7a05ebf12"}
Sep 16 18:45:22.832129 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:22.832065    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-shx66" podStartSLOduration=7.516322821 podStartE2EDuration="35.83204903s" totalImagesPullingTime="28.315726209s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:44:47 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:45:22.830854122 +0000 UTC m=+36.907958981" watchObservedRunningTime="2026-09-16 18:45:22.83204903 +0000 UTC m=+36.909153889"
Sep 16 18:45:23.028478 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:23.028452    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-58tb5"]
Sep 16 18:45:23.032409 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:45:23.032387    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c43af6d_a516_4fc1_8f99_9a99fad657e8.slice/crio-4e7cb727aa2cffe6e66bb1cf3f4725bb52be84f35aae45952a76fa7a889a678a WatchSource:0}: Error finding container 4e7cb727aa2cffe6e66bb1cf3f4725bb52be84f35aae45952a76fa7a889a678a: Status 404 returned error can't find the container with id 4e7cb727aa2cffe6e66bb1cf3f4725bb52be84f35aae45952a76fa7a889a678a
Sep 16 18:45:23.124487 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:23.124450    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-metrics-tls\") pod \"dns-default-sdw9s\" (UID: \"8ac962d8-4a04-40f2-a02c-ad61a26d1fbd\") " pod="openshift-dns/dns-default-sdw9s"
Sep 16 18:45:23.124648 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:23.124627    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:45:23.124803 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:23.124704    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-metrics-tls podName:8ac962d8-4a04-40f2-a02c-ad61a26d1fbd nodeName:}" failed. No retries permitted until 2026-09-16 18:45:24.124683628 +0000 UTC m=+38.201788466 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-metrics-tls") pod "dns-default-sdw9s" (UID: "8ac962d8-4a04-40f2-a02c-ad61a26d1fbd") : secret "dns-default-metrics-tls" not found
Sep 16 18:45:23.460783 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:23.460691    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:45:23.463523 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:23.463496    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 18:45:23.809283 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:23.809044    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-58tb5" event={"ID":"2c43af6d-a516-4fc1-8f99-9a99fad657e8","Type":"ContainerStarted","Data":"4e7cb727aa2cffe6e66bb1cf3f4725bb52be84f35aae45952a76fa7a889a678a"}
Sep 16 18:45:24.132493 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:24.132401    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-metrics-tls\") pod \"dns-default-sdw9s\" (UID: \"8ac962d8-4a04-40f2-a02c-ad61a26d1fbd\") " pod="openshift-dns/dns-default-sdw9s"
Sep 16 18:45:24.132629 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:24.132570    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:45:24.132677 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:24.132653    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-metrics-tls podName:8ac962d8-4a04-40f2-a02c-ad61a26d1fbd nodeName:}" failed. No retries permitted until 2026-09-16 18:45:26.132628859 +0000 UTC m=+40.209733716 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-metrics-tls") pod "dns-default-sdw9s" (UID: "8ac962d8-4a04-40f2-a02c-ad61a26d1fbd") : secret "dns-default-metrics-tls" not found
Sep 16 18:45:25.815443 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:25.815412    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-58tb5" event={"ID":"2c43af6d-a516-4fc1-8f99-9a99fad657e8","Type":"ContainerStarted","Data":"ae281d7a7156a944a23f74e7c6752eb41677b837a9c45b5166bcae4d0d6a8fc5"}
Sep 16 18:45:25.832093 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:25.832041    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-58tb5" podStartSLOduration=1.251708554 podStartE2EDuration="3.832024375s" totalImagesPullingTime="2.580315821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:45:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:45:25.831471869 +0000 UTC m=+39.908576731" watchObservedRunningTime="2026-09-16 18:45:25.832024375 +0000 UTC m=+39.909129234"
Sep 16 18:45:26.148027 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:26.147994    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-metrics-tls\") pod \"dns-default-sdw9s\" (UID: \"8ac962d8-4a04-40f2-a02c-ad61a26d1fbd\") " pod="openshift-dns/dns-default-sdw9s"
Sep 16 18:45:26.148205 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:26.148164    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:45:26.148297 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:26.148282    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-metrics-tls podName:8ac962d8-4a04-40f2-a02c-ad61a26d1fbd nodeName:}" failed. No retries permitted until 2026-09-16 18:45:30.148241686 +0000 UTC m=+44.225346651 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-metrics-tls") pod "dns-default-sdw9s" (UID: "8ac962d8-4a04-40f2-a02c-ad61a26d1fbd") : secret "dns-default-metrics-tls" not found
Sep 16 18:45:30.175618 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:30.175573    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-metrics-tls\") pod \"dns-default-sdw9s\" (UID: \"8ac962d8-4a04-40f2-a02c-ad61a26d1fbd\") " pod="openshift-dns/dns-default-sdw9s"
Sep 16 18:45:30.176059 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:30.175729    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:45:30.176059 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:30.175825    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-metrics-tls podName:8ac962d8-4a04-40f2-a02c-ad61a26d1fbd nodeName:}" failed. No retries permitted until 2026-09-16 18:45:38.175793438 +0000 UTC m=+52.252898280 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-metrics-tls") pod "dns-default-sdw9s" (UID: "8ac962d8-4a04-40f2-a02c-ad61a26d1fbd") : secret "dns-default-metrics-tls" not found
Sep 16 18:45:38.227440 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:38.227401    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-metrics-tls\") pod \"dns-default-sdw9s\" (UID: \"8ac962d8-4a04-40f2-a02c-ad61a26d1fbd\") " pod="openshift-dns/dns-default-sdw9s"
Sep 16 18:45:38.227939 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:38.227518    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:45:38.227939 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:38.227578    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-metrics-tls podName:8ac962d8-4a04-40f2-a02c-ad61a26d1fbd nodeName:}" failed. No retries permitted until 2026-09-16 18:45:54.22756524 +0000 UTC m=+68.304670077 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-metrics-tls") pod "dns-default-sdw9s" (UID: "8ac962d8-4a04-40f2-a02c-ad61a26d1fbd") : secret "dns-default-metrics-tls" not found
Sep 16 18:45:46.791900 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:46.791871    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mgl4t"
Sep 16 18:45:52.123273 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:52.123208    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c19f514e-c3a8-41d2-9de1-ad2d86fce986-metrics-certs\") pod \"network-metrics-daemon-4wnsv\" (UID: \"c19f514e-c3a8-41d2-9de1-ad2d86fce986\") " pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:45:52.123764 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:52.123293    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7d1f1fc-4f58-4499-a42b-1f9325efa292-original-pull-secret\") pod \"global-pull-secret-syncer-c2j6m\" (UID: \"b7d1f1fc-4f58-4499-a42b-1f9325efa292\") " pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:45:52.126334 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:52.126313    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 18:45:52.126387 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:52.126353    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 18:45:52.133641 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:52.133624    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 16 18:45:52.133697 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:52.133679    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c19f514e-c3a8-41d2-9de1-ad2d86fce986-metrics-certs podName:c19f514e-c3a8-41d2-9de1-ad2d86fce986 nodeName:}" failed. No retries permitted until 2026-09-16 18:46:56.13366098 +0000 UTC m=+130.210765832 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c19f514e-c3a8-41d2-9de1-ad2d86fce986-metrics-certs") pod "network-metrics-daemon-4wnsv" (UID: "c19f514e-c3a8-41d2-9de1-ad2d86fce986") : secret "metrics-daemon-secret" not found
Sep 16 18:45:52.136666 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:52.136640    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b7d1f1fc-4f58-4499-a42b-1f9325efa292-original-pull-secret\") pod \"global-pull-secret-syncer-c2j6m\" (UID: \"b7d1f1fc-4f58-4499-a42b-1f9325efa292\") " pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:45:52.224318 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:52.224266    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl62r\" (UniqueName: \"kubernetes.io/projected/4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf-kube-api-access-pl62r\") pod \"network-check-target-gtj9s\" (UID: \"4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf\") " pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:45:52.227317 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:52.227296    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 18:45:52.237256 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:52.237239    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 18:45:52.247145 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:52.247128    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl62r\" (UniqueName: \"kubernetes.io/projected/4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf-kube-api-access-pl62r\") pod \"network-check-target-gtj9s\" (UID: \"4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf\") " pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:45:52.270145 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:52.270123    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c2j6m"
Sep 16 18:45:52.477054 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:52.476967    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pf72n\""
Sep 16 18:45:52.485113 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:52.485077    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:45:52.540319 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:52.540294    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-c2j6m"]
Sep 16 18:45:52.543509 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:45:52.543484    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7d1f1fc_4f58_4499_a42b_1f9325efa292.slice/crio-fb55fde2f6699bbb080f586243079263bde34288142ac6b8a44821b8be036d24 WatchSource:0}: Error finding container fb55fde2f6699bbb080f586243079263bde34288142ac6b8a44821b8be036d24: Status 404 returned error can't find the container with id fb55fde2f6699bbb080f586243079263bde34288142ac6b8a44821b8be036d24
Sep 16 18:45:52.755960 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:52.755887    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gtj9s"]
Sep 16 18:45:52.759780 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:45:52.759740    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aff5bc4_d936_4bcc_9f5b_cb0f55fedacf.slice/crio-59db029b23835823d7d854f75ebfac31f8d177dcbfb9babf8620eab63ef91d84 WatchSource:0}: Error finding container 59db029b23835823d7d854f75ebfac31f8d177dcbfb9babf8620eab63ef91d84: Status 404 returned error can't find the container with id 59db029b23835823d7d854f75ebfac31f8d177dcbfb9babf8620eab63ef91d84
Sep 16 18:45:52.875678 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:52.875637    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gtj9s" event={"ID":"4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf","Type":"ContainerStarted","Data":"59db029b23835823d7d854f75ebfac31f8d177dcbfb9babf8620eab63ef91d84"}
Sep 16 18:45:52.876696 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:52.876673    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-c2j6m" event={"ID":"b7d1f1fc-4f58-4499-a42b-1f9325efa292","Type":"ContainerStarted","Data":"fb55fde2f6699bbb080f586243079263bde34288142ac6b8a44821b8be036d24"}
Sep 16 18:45:54.238050 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:54.238005    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-metrics-tls\") pod \"dns-default-sdw9s\" (UID: \"8ac962d8-4a04-40f2-a02c-ad61a26d1fbd\") " pod="openshift-dns/dns-default-sdw9s"
Sep 16 18:45:54.238512 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:54.238180    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:45:54.238512 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:45:54.238257    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-metrics-tls podName:8ac962d8-4a04-40f2-a02c-ad61a26d1fbd nodeName:}" failed. No retries permitted until 2026-09-16 18:46:26.238237126 +0000 UTC m=+100.315341978 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-metrics-tls") pod "dns-default-sdw9s" (UID: "8ac962d8-4a04-40f2-a02c-ad61a26d1fbd") : secret "dns-default-metrics-tls" not found
Sep 16 18:45:57.890644 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:57.890590    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-c2j6m" event={"ID":"b7d1f1fc-4f58-4499-a42b-1f9325efa292","Type":"ContainerStarted","Data":"1b18486fee244966bae866101e15de8e67339d88425c6a3530bd92821c9a0e89"}
Sep 16 18:45:57.892049 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:57.892023    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gtj9s" event={"ID":"4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf","Type":"ContainerStarted","Data":"667cf957af1b51d84887d98596e22317dc99685f12eec5f318d55c79a0f99af8"}
Sep 16 18:45:57.892184 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:57.892157    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:45:57.906374 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:57.906330    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-c2j6m" podStartSLOduration=66.198583716 podStartE2EDuration="1m10.906318132s" totalImagesPullingTime="4.707734416s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:44:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:45:57.905618771 +0000 UTC m=+71.982723631" watchObservedRunningTime="2026-09-16 18:45:57.906318132 +0000 UTC m=+71.983423015"
Sep 16 18:45:57.921569 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:45:57.921533    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gtj9s" podStartSLOduration=66.435263627 podStartE2EDuration="1m10.921520702s" totalImagesPullingTime="4.486257075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:44:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:45:57.920757064 +0000 UTC m=+71.997861927" watchObservedRunningTime="2026-09-16 18:45:57.921520702 +0000 UTC m=+71.998625559"
Sep 16 18:46:11.563867 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.563740    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-ftvj4"]
Sep 16 18:46:11.567818 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.567802    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-ftvj4"
Sep 16 18:46:11.569487 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.569462    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6cd8c6bfcd-mctrp"]
Sep 16 18:46:11.570311 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.570289    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 16 18:46:11.571020 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.570997    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 16 18:46:11.571252 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.571236    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-rn8jn\""
Sep 16 18:46:11.571562 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.571548    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 16 18:46:11.571620 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.571590    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 16 18:46:11.572128 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.572106    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:11.575192 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.575172    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 16 18:46:11.589720 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.589685    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 16 18:46:11.589720 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.589703    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 16 18:46:11.589912 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.589734    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 16 18:46:11.589912 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.589703    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 16 18:46:11.589912 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.589685    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 16 18:46:11.590041 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.589965    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-x6q8q\""
Sep 16 18:46:11.592202 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.592181    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-ftvj4"]
Sep 16 18:46:11.595950 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.595932    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 16 18:46:11.600280 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.600258    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6cd8c6bfcd-mctrp"]
Sep 16 18:46:11.649352 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.649314    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb6525d5-7015-41e7-a307-c282e1f6bd79-trusted-ca-bundle\") pod \"insights-operator-5844b856db-ftvj4\" (UID: \"fb6525d5-7015-41e7-a307-c282e1f6bd79\") " pod="openshift-insights/insights-operator-5844b856db-ftvj4"
Sep 16 18:46:11.649535 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.649383    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fb6525d5-7015-41e7-a307-c282e1f6bd79-serving-cert\") pod \"insights-operator-5844b856db-ftvj4\" (UID: \"fb6525d5-7015-41e7-a307-c282e1f6bd79\") " pod="openshift-insights/insights-operator-5844b856db-ftvj4"
Sep 16 18:46:11.649535 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.649402    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c70456fc-7f1b-4b8c-bd77-fb3523016153-metrics-certs\") pod \"router-default-6cd8c6bfcd-mctrp\" (UID: \"c70456fc-7f1b-4b8c-bd77-fb3523016153\") " pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:11.649535 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.649440    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c70456fc-7f1b-4b8c-bd77-fb3523016153-default-certificate\") pod \"router-default-6cd8c6bfcd-mctrp\" (UID: \"c70456fc-7f1b-4b8c-bd77-fb3523016153\") " pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:11.649535 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.649459    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzvj5\" (UniqueName: \"kubernetes.io/projected/c70456fc-7f1b-4b8c-bd77-fb3523016153-kube-api-access-rzvj5\") pod \"router-default-6cd8c6bfcd-mctrp\" (UID: \"c70456fc-7f1b-4b8c-bd77-fb3523016153\") " pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:11.649535 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.649499    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb6525d5-7015-41e7-a307-c282e1f6bd79-tmp\") pod \"insights-operator-5844b856db-ftvj4\" (UID: \"fb6525d5-7015-41e7-a307-c282e1f6bd79\") " pod="openshift-insights/insights-operator-5844b856db-ftvj4"
Sep 16 18:46:11.649535 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.649522    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c70456fc-7f1b-4b8c-bd77-fb3523016153-stats-auth\") pod \"router-default-6cd8c6bfcd-mctrp\" (UID: \"c70456fc-7f1b-4b8c-bd77-fb3523016153\") " pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:11.649952 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.649563    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fb6525d5-7015-41e7-a307-c282e1f6bd79-snapshots\") pod \"insights-operator-5844b856db-ftvj4\" (UID: \"fb6525d5-7015-41e7-a307-c282e1f6bd79\") " pod="openshift-insights/insights-operator-5844b856db-ftvj4"
Sep 16 18:46:11.649952 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.649605    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rftxl\" (UniqueName: \"kubernetes.io/projected/fb6525d5-7015-41e7-a307-c282e1f6bd79-kube-api-access-rftxl\") pod \"insights-operator-5844b856db-ftvj4\" (UID: \"fb6525d5-7015-41e7-a307-c282e1f6bd79\") " pod="openshift-insights/insights-operator-5844b856db-ftvj4"
Sep 16 18:46:11.649952 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.649652    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c70456fc-7f1b-4b8c-bd77-fb3523016153-service-ca-bundle\") pod \"router-default-6cd8c6bfcd-mctrp\" (UID: \"c70456fc-7f1b-4b8c-bd77-fb3523016153\") " pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:11.649952 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.649702    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb6525d5-7015-41e7-a307-c282e1f6bd79-service-ca-bundle\") pod \"insights-operator-5844b856db-ftvj4\" (UID: \"fb6525d5-7015-41e7-a307-c282e1f6bd79\") " pod="openshift-insights/insights-operator-5844b856db-ftvj4"
Sep 16 18:46:11.674263 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.674239    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-7gwzm"]
Sep 16 18:46:11.677010 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.676995    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-7jxmm"]
Sep 16 18:46:11.677126 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.677107    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-7gwzm"
Sep 16 18:46:11.679538 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.679500    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7jxmm"
Sep 16 18:46:11.679978 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.679963    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:46:11.680059 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.679967    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-jhsjh\""
Sep 16 18:46:11.680098 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.680061    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 16 18:46:11.682403 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.682382    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:46:11.682490 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.682449    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-p7mvh\""
Sep 16 18:46:11.682490 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.682476    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 16 18:46:11.682579 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.682499    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 16 18:46:11.682623 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.682599    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 16 18:46:11.692544 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.692520    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rtk6r"]
Sep 16 18:46:11.696874 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.696856    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-7gwzm"]
Sep 16 18:46:11.696962 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.696933    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rtk6r"
Sep 16 18:46:11.697034 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.697015    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-7jxmm"]
Sep 16 18:46:11.700844 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.700813    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 16 18:46:11.700965 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.700815    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-2b6gf\""
Sep 16 18:46:11.700965 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.700850    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:46:11.701169 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.701155    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 16 18:46:11.709816 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.709798    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rtk6r"]
Sep 16 18:46:11.750119 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.750089    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c70456fc-7f1b-4b8c-bd77-fb3523016153-service-ca-bundle\") pod \"router-default-6cd8c6bfcd-mctrp\" (UID: \"c70456fc-7f1b-4b8c-bd77-fb3523016153\") " pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:11.750215 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.750126    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c3c4753f-d91a-4409-b89e-be97ec7b523e-serving-cert\") pod \"service-ca-operator-cf7c776b7-7jxmm\" (UID: \"c3c4753f-d91a-4409-b89e-be97ec7b523e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7jxmm"
Sep 16 18:46:11.750215 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.750147    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb6525d5-7015-41e7-a307-c282e1f6bd79-service-ca-bundle\") pod \"insights-operator-5844b856db-ftvj4\" (UID: \"fb6525d5-7015-41e7-a307-c282e1f6bd79\") " pod="openshift-insights/insights-operator-5844b856db-ftvj4"
Sep 16 18:46:11.750215 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.750175    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c3c4753f-d91a-4409-b89e-be97ec7b523e-config\") pod \"service-ca-operator-cf7c776b7-7jxmm\" (UID: \"c3c4753f-d91a-4409-b89e-be97ec7b523e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7jxmm"
Sep 16 18:46:11.750215 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.750205    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb6525d5-7015-41e7-a307-c282e1f6bd79-trusted-ca-bundle\") pod \"insights-operator-5844b856db-ftvj4\" (UID: \"fb6525d5-7015-41e7-a307-c282e1f6bd79\") " pod="openshift-insights/insights-operator-5844b856db-ftvj4"
Sep 16 18:46:11.750374 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.750223    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fb6525d5-7015-41e7-a307-c282e1f6bd79-serving-cert\") pod \"insights-operator-5844b856db-ftvj4\" (UID: \"fb6525d5-7015-41e7-a307-c282e1f6bd79\") " pod="openshift-insights/insights-operator-5844b856db-ftvj4"
Sep 16 18:46:11.750374 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.750237    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c70456fc-7f1b-4b8c-bd77-fb3523016153-metrics-certs\") pod \"router-default-6cd8c6bfcd-mctrp\" (UID: \"c70456fc-7f1b-4b8c-bd77-fb3523016153\") " pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:11.750374 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:11.750254    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c70456fc-7f1b-4b8c-bd77-fb3523016153-service-ca-bundle podName:c70456fc-7f1b-4b8c-bd77-fb3523016153 nodeName:}" failed. No retries permitted until 2026-09-16 18:46:12.250237262 +0000 UTC m=+86.327342102 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c70456fc-7f1b-4b8c-bd77-fb3523016153-service-ca-bundle") pod "router-default-6cd8c6bfcd-mctrp" (UID: "c70456fc-7f1b-4b8c-bd77-fb3523016153") : configmap references non-existent config key: service-ca.crt
Sep 16 18:46:11.750374 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.750295    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j77v\" (UniqueName: \"kubernetes.io/projected/c3c4753f-d91a-4409-b89e-be97ec7b523e-kube-api-access-2j77v\") pod \"service-ca-operator-cf7c776b7-7jxmm\" (UID: \"c3c4753f-d91a-4409-b89e-be97ec7b523e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7jxmm"
Sep 16 18:46:11.750374 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:11.750314    2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 18:46:11.750374 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.750346    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c4db\" (UniqueName: \"kubernetes.io/projected/6eaae832-99a9-45d5-b648-d212603a44cb-kube-api-access-2c4db\") pod \"volume-data-source-validator-86695b7cc7-7gwzm\" (UID: \"6eaae832-99a9-45d5-b648-d212603a44cb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-7gwzm"
Sep 16 18:46:11.750650 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:11.750390    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c70456fc-7f1b-4b8c-bd77-fb3523016153-metrics-certs podName:c70456fc-7f1b-4b8c-bd77-fb3523016153 nodeName:}" failed. No retries permitted until 2026-09-16 18:46:12.250363959 +0000 UTC m=+86.327468801 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c70456fc-7f1b-4b8c-bd77-fb3523016153-metrics-certs") pod "router-default-6cd8c6bfcd-mctrp" (UID: "c70456fc-7f1b-4b8c-bd77-fb3523016153") : secret "router-metrics-certs-default" not found
Sep 16 18:46:11.750650 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.750442    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c70456fc-7f1b-4b8c-bd77-fb3523016153-default-certificate\") pod \"router-default-6cd8c6bfcd-mctrp\" (UID: \"c70456fc-7f1b-4b8c-bd77-fb3523016153\") " pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:11.750650 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.750472    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzvj5\" (UniqueName: \"kubernetes.io/projected/c70456fc-7f1b-4b8c-bd77-fb3523016153-kube-api-access-rzvj5\") pod \"router-default-6cd8c6bfcd-mctrp\" (UID: \"c70456fc-7f1b-4b8c-bd77-fb3523016153\") " pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:11.750650 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.750511    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cb6831f-9106-490f-b51c-5d8bd08a347a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-rtk6r\" (UID: \"5cb6831f-9106-490f-b51c-5d8bd08a347a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rtk6r"
Sep 16 18:46:11.750650 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.750536    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqlxz\" (UniqueName: \"kubernetes.io/projected/5cb6831f-9106-490f-b51c-5d8bd08a347a-kube-api-access-dqlxz\") pod \"cluster-samples-operator-86499c89c4-rtk6r\" (UID: \"5cb6831f-9106-490f-b51c-5d8bd08a347a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rtk6r"
Sep 16 18:46:11.750650 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.750583    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb6525d5-7015-41e7-a307-c282e1f6bd79-tmp\") pod \"insights-operator-5844b856db-ftvj4\" (UID: \"fb6525d5-7015-41e7-a307-c282e1f6bd79\") " pod="openshift-insights/insights-operator-5844b856db-ftvj4"
Sep 16 18:46:11.750650 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.750630    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c70456fc-7f1b-4b8c-bd77-fb3523016153-stats-auth\") pod \"router-default-6cd8c6bfcd-mctrp\" (UID: \"c70456fc-7f1b-4b8c-bd77-fb3523016153\") " pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:11.751031 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.750704    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fb6525d5-7015-41e7-a307-c282e1f6bd79-snapshots\") pod \"insights-operator-5844b856db-ftvj4\" (UID: \"fb6525d5-7015-41e7-a307-c282e1f6bd79\") " pod="openshift-insights/insights-operator-5844b856db-ftvj4"
Sep 16 18:46:11.751031 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.750746    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rftxl\" (UniqueName: \"kubernetes.io/projected/fb6525d5-7015-41e7-a307-c282e1f6bd79-kube-api-access-rftxl\") pod \"insights-operator-5844b856db-ftvj4\" (UID: \"fb6525d5-7015-41e7-a307-c282e1f6bd79\") " pod="openshift-insights/insights-operator-5844b856db-ftvj4"
Sep 16 18:46:11.751031 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.750760    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb6525d5-7015-41e7-a307-c282e1f6bd79-service-ca-bundle\") pod \"insights-operator-5844b856db-ftvj4\" (UID: \"fb6525d5-7015-41e7-a307-c282e1f6bd79\") " pod="openshift-insights/insights-operator-5844b856db-ftvj4"
Sep 16 18:46:11.751031 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.750957    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb6525d5-7015-41e7-a307-c282e1f6bd79-tmp\") pod \"insights-operator-5844b856db-ftvj4\" (UID: \"fb6525d5-7015-41e7-a307-c282e1f6bd79\") " pod="openshift-insights/insights-operator-5844b856db-ftvj4"
Sep 16 18:46:11.751303 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.751276    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fb6525d5-7015-41e7-a307-c282e1f6bd79-snapshots\") pod \"insights-operator-5844b856db-ftvj4\" (UID: \"fb6525d5-7015-41e7-a307-c282e1f6bd79\") " pod="openshift-insights/insights-operator-5844b856db-ftvj4"
Sep 16 18:46:11.751405 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.751283    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb6525d5-7015-41e7-a307-c282e1f6bd79-trusted-ca-bundle\") pod \"insights-operator-5844b856db-ftvj4\" (UID: \"fb6525d5-7015-41e7-a307-c282e1f6bd79\") " pod="openshift-insights/insights-operator-5844b856db-ftvj4"
Sep 16 18:46:11.752988 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.752960    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c70456fc-7f1b-4b8c-bd77-fb3523016153-default-certificate\") pod \"router-default-6cd8c6bfcd-mctrp\" (UID: \"c70456fc-7f1b-4b8c-bd77-fb3523016153\") " pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:11.753070 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.753000    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fb6525d5-7015-41e7-a307-c282e1f6bd79-serving-cert\") pod \"insights-operator-5844b856db-ftvj4\" (UID: \"fb6525d5-7015-41e7-a307-c282e1f6bd79\") " pod="openshift-insights/insights-operator-5844b856db-ftvj4"
Sep 16 18:46:11.753116 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.753070    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c70456fc-7f1b-4b8c-bd77-fb3523016153-stats-auth\") pod \"router-default-6cd8c6bfcd-mctrp\" (UID: \"c70456fc-7f1b-4b8c-bd77-fb3523016153\") " pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:11.759263 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.759241    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzvj5\" (UniqueName: \"kubernetes.io/projected/c70456fc-7f1b-4b8c-bd77-fb3523016153-kube-api-access-rzvj5\") pod \"router-default-6cd8c6bfcd-mctrp\" (UID: \"c70456fc-7f1b-4b8c-bd77-fb3523016153\") " pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:11.759427 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.759408    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rftxl\" (UniqueName: \"kubernetes.io/projected/fb6525d5-7015-41e7-a307-c282e1f6bd79-kube-api-access-rftxl\") pod \"insights-operator-5844b856db-ftvj4\" (UID: \"fb6525d5-7015-41e7-a307-c282e1f6bd79\") " pod="openshift-insights/insights-operator-5844b856db-ftvj4"
Sep 16 18:46:11.851138 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.851052    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cb6831f-9106-490f-b51c-5d8bd08a347a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-rtk6r\" (UID: \"5cb6831f-9106-490f-b51c-5d8bd08a347a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rtk6r"
Sep 16 18:46:11.851138 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.851094    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqlxz\" (UniqueName: \"kubernetes.io/projected/5cb6831f-9106-490f-b51c-5d8bd08a347a-kube-api-access-dqlxz\") pod \"cluster-samples-operator-86499c89c4-rtk6r\" (UID: \"5cb6831f-9106-490f-b51c-5d8bd08a347a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rtk6r"
Sep 16 18:46:11.851342 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.851145    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c3c4753f-d91a-4409-b89e-be97ec7b523e-serving-cert\") pod \"service-ca-operator-cf7c776b7-7jxmm\" (UID: \"c3c4753f-d91a-4409-b89e-be97ec7b523e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7jxmm"
Sep 16 18:46:11.851342 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.851168    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c3c4753f-d91a-4409-b89e-be97ec7b523e-config\") pod \"service-ca-operator-cf7c776b7-7jxmm\" (UID: \"c3c4753f-d91a-4409-b89e-be97ec7b523e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7jxmm"
Sep 16 18:46:11.851342 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.851198    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j77v\" (UniqueName: \"kubernetes.io/projected/c3c4753f-d91a-4409-b89e-be97ec7b523e-kube-api-access-2j77v\") pod \"service-ca-operator-cf7c776b7-7jxmm\" (UID: \"c3c4753f-d91a-4409-b89e-be97ec7b523e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7jxmm"
Sep 16 18:46:11.851342 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:11.851202    2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 18:46:11.851342 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.851215    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c4db\" (UniqueName: \"kubernetes.io/projected/6eaae832-99a9-45d5-b648-d212603a44cb-kube-api-access-2c4db\") pod \"volume-data-source-validator-86695b7cc7-7gwzm\" (UID: \"6eaae832-99a9-45d5-b648-d212603a44cb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-7gwzm"
Sep 16 18:46:11.851342 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:11.851280    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cb6831f-9106-490f-b51c-5d8bd08a347a-samples-operator-tls podName:5cb6831f-9106-490f-b51c-5d8bd08a347a nodeName:}" failed. No retries permitted until 2026-09-16 18:46:12.351258781 +0000 UTC m=+86.428363620 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5cb6831f-9106-490f-b51c-5d8bd08a347a-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-rtk6r" (UID: "5cb6831f-9106-490f-b51c-5d8bd08a347a") : secret "samples-operator-tls" not found
Sep 16 18:46:11.851886 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.851867    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c3c4753f-d91a-4409-b89e-be97ec7b523e-config\") pod \"service-ca-operator-cf7c776b7-7jxmm\" (UID: \"c3c4753f-d91a-4409-b89e-be97ec7b523e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7jxmm"
Sep 16 18:46:11.853407 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.853384    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c3c4753f-d91a-4409-b89e-be97ec7b523e-serving-cert\") pod \"service-ca-operator-cf7c776b7-7jxmm\" (UID: \"c3c4753f-d91a-4409-b89e-be97ec7b523e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7jxmm"
Sep 16 18:46:11.864659 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.864631    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j77v\" (UniqueName: \"kubernetes.io/projected/c3c4753f-d91a-4409-b89e-be97ec7b523e-kube-api-access-2j77v\") pod \"service-ca-operator-cf7c776b7-7jxmm\" (UID: \"c3c4753f-d91a-4409-b89e-be97ec7b523e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7jxmm"
Sep 16 18:46:11.865028 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.865006    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqlxz\" (UniqueName: \"kubernetes.io/projected/5cb6831f-9106-490f-b51c-5d8bd08a347a-kube-api-access-dqlxz\") pod \"cluster-samples-operator-86499c89c4-rtk6r\" (UID: \"5cb6831f-9106-490f-b51c-5d8bd08a347a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rtk6r"
Sep 16 18:46:11.865330 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.865312    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c4db\" (UniqueName: \"kubernetes.io/projected/6eaae832-99a9-45d5-b648-d212603a44cb-kube-api-access-2c4db\") pod \"volume-data-source-validator-86695b7cc7-7gwzm\" (UID: \"6eaae832-99a9-45d5-b648-d212603a44cb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-7gwzm"
Sep 16 18:46:11.878109 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.878081    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-ftvj4"
Sep 16 18:46:11.986783 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.986753    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-7gwzm"
Sep 16 18:46:11.992714 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:11.992697    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7jxmm"
Sep 16 18:46:12.146686 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:12.146611    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-ftvj4"]
Sep 16 18:46:12.149286 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:46:12.149265    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb6525d5_7015_41e7_a307_c282e1f6bd79.slice/crio-326840f1f134db78e9d7395fbb480d2256205dac3a4540b912334d21063b6848 WatchSource:0}: Error finding container 326840f1f134db78e9d7395fbb480d2256205dac3a4540b912334d21063b6848: Status 404 returned error can't find the container with id 326840f1f134db78e9d7395fbb480d2256205dac3a4540b912334d21063b6848
Sep 16 18:46:12.254500 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:12.254338    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c70456fc-7f1b-4b8c-bd77-fb3523016153-service-ca-bundle\") pod \"router-default-6cd8c6bfcd-mctrp\" (UID: \"c70456fc-7f1b-4b8c-bd77-fb3523016153\") " pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:12.254500 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:12.254402    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c70456fc-7f1b-4b8c-bd77-fb3523016153-metrics-certs\") pod \"router-default-6cd8c6bfcd-mctrp\" (UID: \"c70456fc-7f1b-4b8c-bd77-fb3523016153\") " pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:12.254738 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:12.254648    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c70456fc-7f1b-4b8c-bd77-fb3523016153-service-ca-bundle podName:c70456fc-7f1b-4b8c-bd77-fb3523016153 nodeName:}" failed. No retries permitted until 2026-09-16 18:46:13.254622364 +0000 UTC m=+87.331727218 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c70456fc-7f1b-4b8c-bd77-fb3523016153-service-ca-bundle") pod "router-default-6cd8c6bfcd-mctrp" (UID: "c70456fc-7f1b-4b8c-bd77-fb3523016153") : configmap references non-existent config key: service-ca.crt
Sep 16 18:46:12.254738 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:12.254657    2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 18:46:12.254738 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:12.254707    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c70456fc-7f1b-4b8c-bd77-fb3523016153-metrics-certs podName:c70456fc-7f1b-4b8c-bd77-fb3523016153 nodeName:}" failed. No retries permitted until 2026-09-16 18:46:13.254692015 +0000 UTC m=+87.331796853 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c70456fc-7f1b-4b8c-bd77-fb3523016153-metrics-certs") pod "router-default-6cd8c6bfcd-mctrp" (UID: "c70456fc-7f1b-4b8c-bd77-fb3523016153") : secret "router-metrics-certs-default" not found
Sep 16 18:46:12.264226 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:12.264203    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-7gwzm"]
Sep 16 18:46:12.268519 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:46:12.268495    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6eaae832_99a9_45d5_b648_d212603a44cb.slice/crio-9c5430d9b9bfdc81cf7d742b38addad356e56d92edeffe15f2edbc1d78089f34 WatchSource:0}: Error finding container 9c5430d9b9bfdc81cf7d742b38addad356e56d92edeffe15f2edbc1d78089f34: Status 404 returned error can't find the container with id 9c5430d9b9bfdc81cf7d742b38addad356e56d92edeffe15f2edbc1d78089f34
Sep 16 18:46:12.270512 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:12.270492    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-7jxmm"]
Sep 16 18:46:12.272501 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:46:12.272481    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3c4753f_d91a_4409_b89e_be97ec7b523e.slice/crio-76dc02b2718714963dd295f4a56044d79605e56e5490e48b0f869577d6c394e9 WatchSource:0}: Error finding container 76dc02b2718714963dd295f4a56044d79605e56e5490e48b0f869577d6c394e9: Status 404 returned error can't find the container with id 76dc02b2718714963dd295f4a56044d79605e56e5490e48b0f869577d6c394e9
Sep 16 18:46:12.355026 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:12.355005    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cb6831f-9106-490f-b51c-5d8bd08a347a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-rtk6r\" (UID: \"5cb6831f-9106-490f-b51c-5d8bd08a347a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rtk6r"
Sep 16 18:46:12.355163 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:12.355147    2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 18:46:12.355212 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:12.355199    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cb6831f-9106-490f-b51c-5d8bd08a347a-samples-operator-tls podName:5cb6831f-9106-490f-b51c-5d8bd08a347a nodeName:}" failed. No retries permitted until 2026-09-16 18:46:13.35518577 +0000 UTC m=+87.432290609 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5cb6831f-9106-490f-b51c-5d8bd08a347a-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-rtk6r" (UID: "5cb6831f-9106-490f-b51c-5d8bd08a347a") : secret "samples-operator-tls" not found
Sep 16 18:46:12.932107 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:12.932066    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7jxmm" event={"ID":"c3c4753f-d91a-4409-b89e-be97ec7b523e","Type":"ContainerStarted","Data":"76dc02b2718714963dd295f4a56044d79605e56e5490e48b0f869577d6c394e9"}
Sep 16 18:46:12.933570 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:12.933504    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-ftvj4" event={"ID":"fb6525d5-7015-41e7-a307-c282e1f6bd79","Type":"ContainerStarted","Data":"326840f1f134db78e9d7395fbb480d2256205dac3a4540b912334d21063b6848"}
Sep 16 18:46:12.935217 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:12.935192    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-7gwzm" event={"ID":"6eaae832-99a9-45d5-b648-d212603a44cb","Type":"ContainerStarted","Data":"9c5430d9b9bfdc81cf7d742b38addad356e56d92edeffe15f2edbc1d78089f34"}
Sep 16 18:46:13.263031 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:13.262344    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c70456fc-7f1b-4b8c-bd77-fb3523016153-service-ca-bundle\") pod \"router-default-6cd8c6bfcd-mctrp\" (UID: \"c70456fc-7f1b-4b8c-bd77-fb3523016153\") " pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:13.263031 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:13.262420    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c70456fc-7f1b-4b8c-bd77-fb3523016153-metrics-certs\") pod \"router-default-6cd8c6bfcd-mctrp\" (UID: \"c70456fc-7f1b-4b8c-bd77-fb3523016153\") " pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:13.263031 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:13.262581    2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 18:46:13.263031 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:13.262636    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c70456fc-7f1b-4b8c-bd77-fb3523016153-metrics-certs podName:c70456fc-7f1b-4b8c-bd77-fb3523016153 nodeName:}" failed. No retries permitted until 2026-09-16 18:46:15.262623036 +0000 UTC m=+89.339727874 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c70456fc-7f1b-4b8c-bd77-fb3523016153-metrics-certs") pod "router-default-6cd8c6bfcd-mctrp" (UID: "c70456fc-7f1b-4b8c-bd77-fb3523016153") : secret "router-metrics-certs-default" not found
Sep 16 18:46:13.263031 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:13.262850    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c70456fc-7f1b-4b8c-bd77-fb3523016153-service-ca-bundle podName:c70456fc-7f1b-4b8c-bd77-fb3523016153 nodeName:}" failed. No retries permitted until 2026-09-16 18:46:15.262829377 +0000 UTC m=+89.339934233 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c70456fc-7f1b-4b8c-bd77-fb3523016153-service-ca-bundle") pod "router-default-6cd8c6bfcd-mctrp" (UID: "c70456fc-7f1b-4b8c-bd77-fb3523016153") : configmap references non-existent config key: service-ca.crt
Sep 16 18:46:13.363852 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:13.363815    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cb6831f-9106-490f-b51c-5d8bd08a347a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-rtk6r\" (UID: \"5cb6831f-9106-490f-b51c-5d8bd08a347a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rtk6r"
Sep 16 18:46:13.364033 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:13.363988    2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 18:46:13.364097 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:13.364063    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cb6831f-9106-490f-b51c-5d8bd08a347a-samples-operator-tls podName:5cb6831f-9106-490f-b51c-5d8bd08a347a nodeName:}" failed. No retries permitted until 2026-09-16 18:46:15.364043772 +0000 UTC m=+89.441148630 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5cb6831f-9106-490f-b51c-5d8bd08a347a-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-rtk6r" (UID: "5cb6831f-9106-490f-b51c-5d8bd08a347a") : secret "samples-operator-tls" not found
Sep 16 18:46:13.939230 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:13.939187    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-7gwzm" event={"ID":"6eaae832-99a9-45d5-b648-d212603a44cb","Type":"ContainerStarted","Data":"495981ae946bf1b8fb396c3d7c3112ef22a33a74b078e2fc54d421ad22fd9ba6"}
Sep 16 18:46:13.962180 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:13.961903    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-7gwzm" podStartSLOduration=1.849089661 podStartE2EDuration="2.961884806s" totalImagesPullingTime="1.112795145s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:46:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:46:13.961576374 +0000 UTC m=+88.038681237" watchObservedRunningTime="2026-09-16 18:46:13.961884806 +0000 UTC m=+88.038989665"
Sep 16 18:46:14.945296 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:14.945191    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7jxmm" event={"ID":"c3c4753f-d91a-4409-b89e-be97ec7b523e","Type":"ContainerStarted","Data":"4c7aeac3bebf3b4f20af1f6a84afde2930236ce102c7618e82a475c231042585"}
Sep 16 18:46:14.946821 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:14.946790    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-ftvj4" event={"ID":"fb6525d5-7015-41e7-a307-c282e1f6bd79","Type":"ContainerStarted","Data":"e470340fbf6d75192ca346a84df7d604d6d2cf98666b55a3423aaebb0cdf12f0"}
Sep 16 18:46:14.962667 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:14.962617    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7jxmm" podStartSLOduration=1.628396443 podStartE2EDuration="3.962598805s" totalImagesPullingTime="2.334202362s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:46:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:46:14.962286043 +0000 UTC m=+89.039390930" watchObservedRunningTime="2026-09-16 18:46:14.962598805 +0000 UTC m=+89.039703668"
Sep 16 18:46:14.985538 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:14.985486    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-ftvj4" podStartSLOduration=1.5324699819999998 podStartE2EDuration="3.985467567s" totalImagesPullingTime="2.452997585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:46:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:46:14.984147813 +0000 UTC m=+89.061252673" watchObservedRunningTime="2026-09-16 18:46:14.985467567 +0000 UTC m=+89.062572433"
Sep 16 18:46:15.281912 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:15.281879    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c70456fc-7f1b-4b8c-bd77-fb3523016153-service-ca-bundle\") pod \"router-default-6cd8c6bfcd-mctrp\" (UID: \"c70456fc-7f1b-4b8c-bd77-fb3523016153\") " pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:15.282045 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:15.281928    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c70456fc-7f1b-4b8c-bd77-fb3523016153-metrics-certs\") pod \"router-default-6cd8c6bfcd-mctrp\" (UID: \"c70456fc-7f1b-4b8c-bd77-fb3523016153\") " pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:15.282045 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:15.282038    2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 18:46:15.282126 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:15.282074    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c70456fc-7f1b-4b8c-bd77-fb3523016153-service-ca-bundle podName:c70456fc-7f1b-4b8c-bd77-fb3523016153 nodeName:}" failed. No retries permitted until 2026-09-16 18:46:19.282050399 +0000 UTC m=+93.359155245 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c70456fc-7f1b-4b8c-bd77-fb3523016153-service-ca-bundle") pod "router-default-6cd8c6bfcd-mctrp" (UID: "c70456fc-7f1b-4b8c-bd77-fb3523016153") : configmap references non-existent config key: service-ca.crt
Sep 16 18:46:15.282126 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:15.282121    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c70456fc-7f1b-4b8c-bd77-fb3523016153-metrics-certs podName:c70456fc-7f1b-4b8c-bd77-fb3523016153 nodeName:}" failed. No retries permitted until 2026-09-16 18:46:19.282098289 +0000 UTC m=+93.359203130 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c70456fc-7f1b-4b8c-bd77-fb3523016153-metrics-certs") pod "router-default-6cd8c6bfcd-mctrp" (UID: "c70456fc-7f1b-4b8c-bd77-fb3523016153") : secret "router-metrics-certs-default" not found
Sep 16 18:46:15.382794 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:15.382749    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cb6831f-9106-490f-b51c-5d8bd08a347a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-rtk6r\" (UID: \"5cb6831f-9106-490f-b51c-5d8bd08a347a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rtk6r"
Sep 16 18:46:15.382973 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:15.382896    2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 18:46:15.382973 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:15.382955    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cb6831f-9106-490f-b51c-5d8bd08a347a-samples-operator-tls podName:5cb6831f-9106-490f-b51c-5d8bd08a347a nodeName:}" failed. No retries permitted until 2026-09-16 18:46:19.382941413 +0000 UTC m=+93.460046251 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5cb6831f-9106-490f-b51c-5d8bd08a347a-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-rtk6r" (UID: "5cb6831f-9106-490f-b51c-5d8bd08a347a") : secret "samples-operator-tls" not found
Sep 16 18:46:15.881293 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:15.881259    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-m8kzv"]
Sep 16 18:46:15.884542 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:15.884525    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m8kzv"
Sep 16 18:46:15.890318 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:15.890292    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6l7cw\""
Sep 16 18:46:15.890466 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:15.890344    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 16 18:46:15.891236 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:15.891212    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 16 18:46:15.905044 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:15.905022    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m8kzv"]
Sep 16 18:46:15.985986 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:15.985952    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m8kzv\" (UID: \"b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53\") " pod="openshift-insights/insights-runtime-extractor-m8kzv"
Sep 16 18:46:15.986365 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:15.986001    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-data-volume\") pod \"insights-runtime-extractor-m8kzv\" (UID: \"b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53\") " pod="openshift-insights/insights-runtime-extractor-m8kzv"
Sep 16 18:46:15.986365 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:15.986026    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m8kzv\" (UID: \"b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53\") " pod="openshift-insights/insights-runtime-extractor-m8kzv"
Sep 16 18:46:15.986365 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:15.986098    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdhn4\" (UniqueName: \"kubernetes.io/projected/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-kube-api-access-sdhn4\") pod \"insights-runtime-extractor-m8kzv\" (UID: \"b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53\") " pod="openshift-insights/insights-runtime-extractor-m8kzv"
Sep 16 18:46:15.986365 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:15.986221    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-crio-socket\") pod \"insights-runtime-extractor-m8kzv\" (UID: \"b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53\") " pod="openshift-insights/insights-runtime-extractor-m8kzv"
Sep 16 18:46:16.087187 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:16.087152    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-data-volume\") pod \"insights-runtime-extractor-m8kzv\" (UID: \"b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53\") " pod="openshift-insights/insights-runtime-extractor-m8kzv"
Sep 16 18:46:16.087187 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:16.087187    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m8kzv\" (UID: \"b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53\") " pod="openshift-insights/insights-runtime-extractor-m8kzv"
Sep 16 18:46:16.087397 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:16.087217    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdhn4\" (UniqueName: \"kubernetes.io/projected/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-kube-api-access-sdhn4\") pod \"insights-runtime-extractor-m8kzv\" (UID: \"b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53\") " pod="openshift-insights/insights-runtime-extractor-m8kzv"
Sep 16 18:46:16.087397 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:16.087240    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-crio-socket\") pod \"insights-runtime-extractor-m8kzv\" (UID: \"b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53\") " pod="openshift-insights/insights-runtime-extractor-m8kzv"
Sep 16 18:46:16.087397 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:16.087295    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m8kzv\" (UID: \"b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53\") " pod="openshift-insights/insights-runtime-extractor-m8kzv"
Sep 16 18:46:16.087397 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:16.087322    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:46:16.087397 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:16.087384    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 18:46:16.087397 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:16.087386    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-crio-socket\") pod \"insights-runtime-extractor-m8kzv\" (UID: \"b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53\") " pod="openshift-insights/insights-runtime-extractor-m8kzv"
Sep 16 18:46:16.087633 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:16.087398    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-insights-runtime-extractor-tls podName:b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53 nodeName:}" failed. No retries permitted until 2026-09-16 18:46:16.58737609 +0000 UTC m=+90.664480944 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m8kzv" (UID: "b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:46:16.087633 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:16.087511    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-kube-rbac-proxy-cm podName:b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53 nodeName:}" failed. No retries permitted until 2026-09-16 18:46:16.587492227 +0000 UTC m=+90.664597065 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m8kzv" (UID: "b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53") : configmap "kube-rbac-proxy" not found
Sep 16 18:46:16.087633 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:16.087553    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-data-volume\") pod \"insights-runtime-extractor-m8kzv\" (UID: \"b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53\") " pod="openshift-insights/insights-runtime-extractor-m8kzv"
Sep 16 18:46:16.096078 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:16.096050    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdhn4\" (UniqueName: \"kubernetes.io/projected/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-kube-api-access-sdhn4\") pod \"insights-runtime-extractor-m8kzv\" (UID: \"b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53\") " pod="openshift-insights/insights-runtime-extractor-m8kzv"
Sep 16 18:46:16.591816 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:16.591765    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m8kzv\" (UID: \"b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53\") " pod="openshift-insights/insights-runtime-extractor-m8kzv"
Sep 16 18:46:16.591981 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:16.591827    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m8kzv\" (UID: \"b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53\") " pod="openshift-insights/insights-runtime-extractor-m8kzv"
Sep 16 18:46:16.591981 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:16.591887    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 18:46:16.591981 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:16.591922    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:46:16.591981 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:16.591947    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-kube-rbac-proxy-cm podName:b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53 nodeName:}" failed. No retries permitted until 2026-09-16 18:46:17.591933397 +0000 UTC m=+91.669038234 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m8kzv" (UID: "b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53") : configmap "kube-rbac-proxy" not found
Sep 16 18:46:16.591981 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:16.591967    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-insights-runtime-extractor-tls podName:b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53 nodeName:}" failed. No retries permitted until 2026-09-16 18:46:17.591956554 +0000 UTC m=+91.669061392 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m8kzv" (UID: "b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:46:17.599642 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:17.599600    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m8kzv\" (UID: \"b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53\") " pod="openshift-insights/insights-runtime-extractor-m8kzv"
Sep 16 18:46:17.600051 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:17.599668    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m8kzv\" (UID: \"b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53\") " pod="openshift-insights/insights-runtime-extractor-m8kzv"
Sep 16 18:46:17.600051 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:17.599722    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 18:46:17.600051 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:17.599808    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:46:17.600051 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:17.599819    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-kube-rbac-proxy-cm podName:b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53 nodeName:}" failed. No retries permitted until 2026-09-16 18:46:19.599802607 +0000 UTC m=+93.676907445 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m8kzv" (UID: "b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53") : configmap "kube-rbac-proxy" not found
Sep 16 18:46:17.600051 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:17.599868    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-insights-runtime-extractor-tls podName:b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53 nodeName:}" failed. No retries permitted until 2026-09-16 18:46:19.599853518 +0000 UTC m=+93.676958363 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m8kzv" (UID: "b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:46:17.828703 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:17.828677    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4rzjs_55642348-e5c2-42ed-af38-d0631bd164aa/dns-node-resolver/0.log"
Sep 16 18:46:18.067542 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.067508    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-c75d9488-qzft7"]
Sep 16 18:46:18.070652 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.070637    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:18.077092 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.076819    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 16 18:46:18.077092 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.076876    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 16 18:46:18.078017 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.077983    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 16 18:46:18.078117 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.077987    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7kp4l\""
Sep 16 18:46:18.083045 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.083028    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 16 18:46:18.103848 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.103825    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-registry-tls\") pod \"image-registry-c75d9488-qzft7\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") " pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:18.103946 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.103851    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-ca-trust-extracted\") pod \"image-registry-c75d9488-qzft7\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") " pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:18.103946 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.103882    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-trusted-ca\") pod \"image-registry-c75d9488-qzft7\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") " pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:18.104041 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.103967    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-registry-certificates\") pod \"image-registry-c75d9488-qzft7\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") " pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:18.104041 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.104021    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-image-registry-private-configuration\") pod \"image-registry-c75d9488-qzft7\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") " pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:18.104106 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.104060    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-installation-pull-secrets\") pod \"image-registry-c75d9488-qzft7\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") " pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:18.104155 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.104103    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-bound-sa-token\") pod \"image-registry-c75d9488-qzft7\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") " pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:18.104155 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.104124    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x574\" (UniqueName: \"kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-kube-api-access-7x574\") pod \"image-registry-c75d9488-qzft7\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") " pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:18.106956 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.106934    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c75d9488-qzft7"]
Sep 16 18:46:18.205403 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.205376    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-registry-tls\") pod \"image-registry-c75d9488-qzft7\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") " pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:18.205403 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.205404    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-ca-trust-extracted\") pod \"image-registry-c75d9488-qzft7\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") " pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:18.205582 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.205432    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-trusted-ca\") pod \"image-registry-c75d9488-qzft7\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") " pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:18.205582 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.205463    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-registry-certificates\") pod \"image-registry-c75d9488-qzft7\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") " pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:18.205582 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.205482    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-image-registry-private-configuration\") pod \"image-registry-c75d9488-qzft7\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") " pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:18.205582 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.205507    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-installation-pull-secrets\") pod \"image-registry-c75d9488-qzft7\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") " pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:18.205582 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:18.205509    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 18:46:18.205582 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:18.205526    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c75d9488-qzft7: secret "image-registry-tls" not found
Sep 16 18:46:18.205582 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:18.205579    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-registry-tls podName:54674a4a-c1b7-429a-a4a9-21f7f93d22ab nodeName:}" failed. No retries permitted until 2026-09-16 18:46:18.705560259 +0000 UTC m=+92.782665114 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-registry-tls") pod "image-registry-c75d9488-qzft7" (UID: "54674a4a-c1b7-429a-a4a9-21f7f93d22ab") : secret "image-registry-tls" not found
Sep 16 18:46:18.205920 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.205637    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-bound-sa-token\") pod \"image-registry-c75d9488-qzft7\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") " pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:18.205920 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.205674    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x574\" (UniqueName: \"kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-kube-api-access-7x574\") pod \"image-registry-c75d9488-qzft7\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") " pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:18.205920 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.205785    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-ca-trust-extracted\") pod \"image-registry-c75d9488-qzft7\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") " pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:18.206425 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.206401    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-registry-certificates\") pod \"image-registry-c75d9488-qzft7\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") " pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:18.206808 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.206791    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-trusted-ca\") pod \"image-registry-c75d9488-qzft7\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") " pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:18.207880 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.207862    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-installation-pull-secrets\") pod \"image-registry-c75d9488-qzft7\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") " pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:18.207961 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.207932    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-image-registry-private-configuration\") pod \"image-registry-c75d9488-qzft7\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") " pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:18.218289 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.218267    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x574\" (UniqueName: \"kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-kube-api-access-7x574\") pod \"image-registry-c75d9488-qzft7\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") " pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:18.218521 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.218504    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-bound-sa-token\") pod \"image-registry-c75d9488-qzft7\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") " pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:18.426269 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.426188    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-hs46q"]
Sep 16 18:46:18.430289 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.430272    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-hs46q"
Sep 16 18:46:18.433538 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.433515    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 16 18:46:18.433655 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.433624    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 16 18:46:18.433902 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.433879    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 16 18:46:18.434214 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.434195    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 16 18:46:18.434345 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.434326    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-4tzmq\""
Sep 16 18:46:18.434432 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.434417    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-q6g22_5c6f6f8b-e6fc-4a16-b084-7e851b3c3cc6/node-ca/0.log"
Sep 16 18:46:18.438177 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.438153    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-hs46q"]
Sep 16 18:46:18.507056 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.507026    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7c0f002e-79b0-4a26-ae09-bc9c727e066b-signing-cabundle\") pod \"service-ca-568869fdc5-hs46q\" (UID: \"7c0f002e-79b0-4a26-ae09-bc9c727e066b\") " pod="openshift-service-ca/service-ca-568869fdc5-hs46q"
Sep 16 18:46:18.507186 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.507065    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7c0f002e-79b0-4a26-ae09-bc9c727e066b-signing-key\") pod \"service-ca-568869fdc5-hs46q\" (UID: \"7c0f002e-79b0-4a26-ae09-bc9c727e066b\") " pod="openshift-service-ca/service-ca-568869fdc5-hs46q"
Sep 16 18:46:18.507186 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.507086    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9jwt\" (UniqueName: \"kubernetes.io/projected/7c0f002e-79b0-4a26-ae09-bc9c727e066b-kube-api-access-j9jwt\") pod \"service-ca-568869fdc5-hs46q\" (UID: \"7c0f002e-79b0-4a26-ae09-bc9c727e066b\") " pod="openshift-service-ca/service-ca-568869fdc5-hs46q"
Sep 16 18:46:18.608173 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.608141    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7c0f002e-79b0-4a26-ae09-bc9c727e066b-signing-cabundle\") pod \"service-ca-568869fdc5-hs46q\" (UID: \"7c0f002e-79b0-4a26-ae09-bc9c727e066b\") " pod="openshift-service-ca/service-ca-568869fdc5-hs46q"
Sep 16 18:46:18.608521 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.608183    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7c0f002e-79b0-4a26-ae09-bc9c727e066b-signing-key\") pod \"service-ca-568869fdc5-hs46q\" (UID: \"7c0f002e-79b0-4a26-ae09-bc9c727e066b\") " pod="openshift-service-ca/service-ca-568869fdc5-hs46q"
Sep 16 18:46:18.608521 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.608205    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9jwt\" (UniqueName: \"kubernetes.io/projected/7c0f002e-79b0-4a26-ae09-bc9c727e066b-kube-api-access-j9jwt\") pod \"service-ca-568869fdc5-hs46q\" (UID: \"7c0f002e-79b0-4a26-ae09-bc9c727e066b\") " pod="openshift-service-ca/service-ca-568869fdc5-hs46q"
Sep 16 18:46:18.608912 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.608886    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7c0f002e-79b0-4a26-ae09-bc9c727e066b-signing-cabundle\") pod \"service-ca-568869fdc5-hs46q\" (UID: \"7c0f002e-79b0-4a26-ae09-bc9c727e066b\") " pod="openshift-service-ca/service-ca-568869fdc5-hs46q"
Sep 16 18:46:18.610457 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.610439    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7c0f002e-79b0-4a26-ae09-bc9c727e066b-signing-key\") pod \"service-ca-568869fdc5-hs46q\" (UID: \"7c0f002e-79b0-4a26-ae09-bc9c727e066b\") " pod="openshift-service-ca/service-ca-568869fdc5-hs46q"
Sep 16 18:46:18.633255 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.633232    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9jwt\" (UniqueName: \"kubernetes.io/projected/7c0f002e-79b0-4a26-ae09-bc9c727e066b-kube-api-access-j9jwt\") pod \"service-ca-568869fdc5-hs46q\" (UID: \"7c0f002e-79b0-4a26-ae09-bc9c727e066b\") " pod="openshift-service-ca/service-ca-568869fdc5-hs46q"
Sep 16 18:46:18.709607 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.709552    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-registry-tls\") pod \"image-registry-c75d9488-qzft7\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") " pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:18.709718 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:18.709697    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 18:46:18.709758 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:18.709719    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c75d9488-qzft7: secret "image-registry-tls" not found
Sep 16 18:46:18.709816 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:18.709794    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-registry-tls podName:54674a4a-c1b7-429a-a4a9-21f7f93d22ab nodeName:}" failed. No retries permitted until 2026-09-16 18:46:19.709763126 +0000 UTC m=+93.786867964 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-registry-tls") pod "image-registry-c75d9488-qzft7" (UID: "54674a4a-c1b7-429a-a4a9-21f7f93d22ab") : secret "image-registry-tls" not found
Sep 16 18:46:18.739882 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:18.739863    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-hs46q"
Sep 16 18:46:19.019430 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:19.019359    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-hs46q"]
Sep 16 18:46:19.022169 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:46:19.022137    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c0f002e_79b0_4a26_ae09_bc9c727e066b.slice/crio-92a1657a66de9908be542057e53197b0f26f359ebe9ed23e8693cd4c2a66e220 WatchSource:0}: Error finding container 92a1657a66de9908be542057e53197b0f26f359ebe9ed23e8693cd4c2a66e220: Status 404 returned error can't find the container with id 92a1657a66de9908be542057e53197b0f26f359ebe9ed23e8693cd4c2a66e220
Sep 16 18:46:19.229108 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:19.229082    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-58tb5_2c43af6d-a516-4fc1-8f99-9a99fad657e8/serve-healthcheck-canary/0.log"
Sep 16 18:46:19.314170 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:19.314140    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c70456fc-7f1b-4b8c-bd77-fb3523016153-service-ca-bundle\") pod \"router-default-6cd8c6bfcd-mctrp\" (UID: \"c70456fc-7f1b-4b8c-bd77-fb3523016153\") " pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:19.314315 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:19.314181    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c70456fc-7f1b-4b8c-bd77-fb3523016153-metrics-certs\") pod \"router-default-6cd8c6bfcd-mctrp\" (UID: \"c70456fc-7f1b-4b8c-bd77-fb3523016153\") " pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:19.314357 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:19.314324    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c70456fc-7f1b-4b8c-bd77-fb3523016153-service-ca-bundle podName:c70456fc-7f1b-4b8c-bd77-fb3523016153 nodeName:}" failed. No retries permitted until 2026-09-16 18:46:27.314302532 +0000 UTC m=+101.391407390 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c70456fc-7f1b-4b8c-bd77-fb3523016153-service-ca-bundle") pod "router-default-6cd8c6bfcd-mctrp" (UID: "c70456fc-7f1b-4b8c-bd77-fb3523016153") : configmap references non-existent config key: service-ca.crt
Sep 16 18:46:19.314357 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:19.314338    2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 18:46:19.314434 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:19.314371    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c70456fc-7f1b-4b8c-bd77-fb3523016153-metrics-certs podName:c70456fc-7f1b-4b8c-bd77-fb3523016153 nodeName:}" failed. No retries permitted until 2026-09-16 18:46:27.314361357 +0000 UTC m=+101.391466209 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c70456fc-7f1b-4b8c-bd77-fb3523016153-metrics-certs") pod "router-default-6cd8c6bfcd-mctrp" (UID: "c70456fc-7f1b-4b8c-bd77-fb3523016153") : secret "router-metrics-certs-default" not found
Sep 16 18:46:19.414672 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:19.414638    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cb6831f-9106-490f-b51c-5d8bd08a347a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-rtk6r\" (UID: \"5cb6831f-9106-490f-b51c-5d8bd08a347a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rtk6r"
Sep 16 18:46:19.414851 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:19.414751    2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 18:46:19.414851 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:19.414832    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cb6831f-9106-490f-b51c-5d8bd08a347a-samples-operator-tls podName:5cb6831f-9106-490f-b51c-5d8bd08a347a nodeName:}" failed. No retries permitted until 2026-09-16 18:46:27.414810937 +0000 UTC m=+101.491915776 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5cb6831f-9106-490f-b51c-5d8bd08a347a-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-rtk6r" (UID: "5cb6831f-9106-490f-b51c-5d8bd08a347a") : secret "samples-operator-tls" not found
Sep 16 18:46:19.615469 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:19.615388    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m8kzv\" (UID: \"b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53\") " pod="openshift-insights/insights-runtime-extractor-m8kzv"
Sep 16 18:46:19.615469 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:19.615433    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m8kzv\" (UID: \"b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53\") " pod="openshift-insights/insights-runtime-extractor-m8kzv"
Sep 16 18:46:19.615895 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:19.615512    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 18:46:19.615895 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:19.615533    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:46:19.615895 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:19.615566    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-kube-rbac-proxy-cm podName:b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53 nodeName:}" failed. No retries permitted until 2026-09-16 18:46:23.615552471 +0000 UTC m=+97.692657325 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m8kzv" (UID: "b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53") : configmap "kube-rbac-proxy" not found
Sep 16 18:46:19.615895 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:19.615633    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-insights-runtime-extractor-tls podName:b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53 nodeName:}" failed. No retries permitted until 2026-09-16 18:46:23.615615223 +0000 UTC m=+97.692720065 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m8kzv" (UID: "b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:46:19.716136 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:19.716099    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-registry-tls\") pod \"image-registry-c75d9488-qzft7\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") " pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:19.716312 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:19.716242    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 18:46:19.716312 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:19.716263    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c75d9488-qzft7: secret "image-registry-tls" not found
Sep 16 18:46:19.716391 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:19.716316    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-registry-tls podName:54674a4a-c1b7-429a-a4a9-21f7f93d22ab nodeName:}" failed. No retries permitted until 2026-09-16 18:46:21.716302289 +0000 UTC m=+95.793407127 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-registry-tls") pod "image-registry-c75d9488-qzft7" (UID: "54674a4a-c1b7-429a-a4a9-21f7f93d22ab") : secret "image-registry-tls" not found
Sep 16 18:46:19.963524 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:19.963428    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-hs46q" event={"ID":"7c0f002e-79b0-4a26-ae09-bc9c727e066b","Type":"ContainerStarted","Data":"4a42945affb0c84182a3d0e528392af1208a4b38789bc5bfb8b956850750dd78"}
Sep 16 18:46:19.963524 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:19.963511    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-hs46q" event={"ID":"7c0f002e-79b0-4a26-ae09-bc9c727e066b","Type":"ContainerStarted","Data":"92a1657a66de9908be542057e53197b0f26f359ebe9ed23e8693cd4c2a66e220"}
Sep 16 18:46:19.989118 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:19.989073    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-hs46q" podStartSLOduration=1.989059872 podStartE2EDuration="1.989059872s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:46:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:46:19.988945343 +0000 UTC m=+94.066050202" watchObservedRunningTime="2026-09-16 18:46:19.989059872 +0000 UTC m=+94.066164732"
Sep 16 18:46:21.729788 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:21.729725    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-registry-tls\") pod \"image-registry-c75d9488-qzft7\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") " pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:21.730171 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:21.729883    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 18:46:21.730171 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:21.729904    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c75d9488-qzft7: secret "image-registry-tls" not found
Sep 16 18:46:21.730171 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:21.729958    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-registry-tls podName:54674a4a-c1b7-429a-a4a9-21f7f93d22ab nodeName:}" failed. No retries permitted until 2026-09-16 18:46:25.729943322 +0000 UTC m=+99.807048160 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-registry-tls") pod "image-registry-c75d9488-qzft7" (UID: "54674a4a-c1b7-429a-a4a9-21f7f93d22ab") : secret "image-registry-tls" not found
Sep 16 18:46:23.646386 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:23.646341    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m8kzv\" (UID: \"b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53\") " pod="openshift-insights/insights-runtime-extractor-m8kzv"
Sep 16 18:46:23.646800 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:23.646491    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m8kzv\" (UID: \"b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53\") " pod="openshift-insights/insights-runtime-extractor-m8kzv"
Sep 16 18:46:23.646800 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:23.646496    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:46:23.646800 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:23.646530    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 18:46:23.646800 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:23.646587    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-kube-rbac-proxy-cm podName:b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53 nodeName:}" failed. No retries permitted until 2026-09-16 18:46:31.646573839 +0000 UTC m=+105.723678678 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m8kzv" (UID: "b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53") : configmap "kube-rbac-proxy" not found
Sep 16 18:46:23.646800 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:23.646603    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-insights-runtime-extractor-tls podName:b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53 nodeName:}" failed. No retries permitted until 2026-09-16 18:46:31.646595755 +0000 UTC m=+105.723700593 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m8kzv" (UID: "b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:46:25.763903 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:25.763864    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-registry-tls\") pod \"image-registry-c75d9488-qzft7\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") " pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:25.766465 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:25.766443    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-registry-tls\") pod \"image-registry-c75d9488-qzft7\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") " pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:25.880388 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:25.880340    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:26.159851 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:26.159826    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c75d9488-qzft7"]
Sep 16 18:46:26.162464 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:46:26.162443    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54674a4a_c1b7_429a_a4a9_21f7f93d22ab.slice/crio-380f7162f71dee0b2a6fce1416a1cdbbb5c16527b36ed5a528ccbb01d3132944 WatchSource:0}: Error finding container 380f7162f71dee0b2a6fce1416a1cdbbb5c16527b36ed5a528ccbb01d3132944: Status 404 returned error can't find the container with id 380f7162f71dee0b2a6fce1416a1cdbbb5c16527b36ed5a528ccbb01d3132944
Sep 16 18:46:26.270055 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:26.269515    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-metrics-tls\") pod \"dns-default-sdw9s\" (UID: \"8ac962d8-4a04-40f2-a02c-ad61a26d1fbd\") " pod="openshift-dns/dns-default-sdw9s"
Sep 16 18:46:26.277252 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:26.277212    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ac962d8-4a04-40f2-a02c-ad61a26d1fbd-metrics-tls\") pod \"dns-default-sdw9s\" (UID: \"8ac962d8-4a04-40f2-a02c-ad61a26d1fbd\") " pod="openshift-dns/dns-default-sdw9s"
Sep 16 18:46:26.287688 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:26.287667    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-g84ch\""
Sep 16 18:46:26.296040 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:26.296023    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sdw9s"
Sep 16 18:46:26.589538 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:26.589502    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sdw9s"]
Sep 16 18:46:26.594374 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:46:26.594337    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ac962d8_4a04_40f2_a02c_ad61a26d1fbd.slice/crio-57423bb7bfa6ced061c0a0174922eeeec69247d1d450bad27e97f27b044d0999 WatchSource:0}: Error finding container 57423bb7bfa6ced061c0a0174922eeeec69247d1d450bad27e97f27b044d0999: Status 404 returned error can't find the container with id 57423bb7bfa6ced061c0a0174922eeeec69247d1d450bad27e97f27b044d0999
Sep 16 18:46:26.984659 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:26.984570    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sdw9s" event={"ID":"8ac962d8-4a04-40f2-a02c-ad61a26d1fbd","Type":"ContainerStarted","Data":"57423bb7bfa6ced061c0a0174922eeeec69247d1d450bad27e97f27b044d0999"}
Sep 16 18:46:26.986058 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:26.986032    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c75d9488-qzft7" event={"ID":"54674a4a-c1b7-429a-a4a9-21f7f93d22ab","Type":"ContainerStarted","Data":"b442d4edcc61fbf39d851e3717cc37899950bb023de6a43c35193478923c02b0"}
Sep 16 18:46:26.986187 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:26.986069    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c75d9488-qzft7" event={"ID":"54674a4a-c1b7-429a-a4a9-21f7f93d22ab","Type":"ContainerStarted","Data":"380f7162f71dee0b2a6fce1416a1cdbbb5c16527b36ed5a528ccbb01d3132944"}
Sep 16 18:46:26.986187 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:26.986107    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:27.009648 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:27.009605    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-c75d9488-qzft7" podStartSLOduration=9.009591905 podStartE2EDuration="9.009591905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:46:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:46:27.00898594 +0000 UTC m=+101.086090801" watchObservedRunningTime="2026-09-16 18:46:27.009591905 +0000 UTC m=+101.086696765"
Sep 16 18:46:27.380605 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:27.380570    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c70456fc-7f1b-4b8c-bd77-fb3523016153-service-ca-bundle\") pod \"router-default-6cd8c6bfcd-mctrp\" (UID: \"c70456fc-7f1b-4b8c-bd77-fb3523016153\") " pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:27.380784 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:27.380620    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c70456fc-7f1b-4b8c-bd77-fb3523016153-metrics-certs\") pod \"router-default-6cd8c6bfcd-mctrp\" (UID: \"c70456fc-7f1b-4b8c-bd77-fb3523016153\") " pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:27.381313 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:27.381285    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c70456fc-7f1b-4b8c-bd77-fb3523016153-service-ca-bundle\") pod \"router-default-6cd8c6bfcd-mctrp\" (UID: \"c70456fc-7f1b-4b8c-bd77-fb3523016153\") " pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:27.383115 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:27.383091    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c70456fc-7f1b-4b8c-bd77-fb3523016153-metrics-certs\") pod \"router-default-6cd8c6bfcd-mctrp\" (UID: \"c70456fc-7f1b-4b8c-bd77-fb3523016153\") " pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:27.481342 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:27.481310    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cb6831f-9106-490f-b51c-5d8bd08a347a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-rtk6r\" (UID: \"5cb6831f-9106-490f-b51c-5d8bd08a347a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rtk6r"
Sep 16 18:46:27.482062 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:27.482036    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:27.484013 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:27.483988    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cb6831f-9106-490f-b51c-5d8bd08a347a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-rtk6r\" (UID: \"5cb6831f-9106-490f-b51c-5d8bd08a347a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rtk6r"
Sep 16 18:46:27.605543 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:27.605507    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rtk6r"
Sep 16 18:46:27.760763 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:27.760741    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6cd8c6bfcd-mctrp"]
Sep 16 18:46:27.879352 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:27.879319    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rtk6r"]
Sep 16 18:46:28.213150 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:46:28.213113    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc70456fc_7f1b_4b8c_bd77_fb3523016153.slice/crio-24c83f8e132817a6c0bbc525c8030bfdac73f23018f50092b426d674d6f5267c WatchSource:0}: Error finding container 24c83f8e132817a6c0bbc525c8030bfdac73f23018f50092b426d674d6f5267c: Status 404 returned error can't find the container with id 24c83f8e132817a6c0bbc525c8030bfdac73f23018f50092b426d674d6f5267c
Sep 16 18:46:28.905666 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:28.905632    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gtj9s"
Sep 16 18:46:28.994827 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:28.994761    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rtk6r" event={"ID":"5cb6831f-9106-490f-b51c-5d8bd08a347a","Type":"ContainerStarted","Data":"80f9bf5753db4a1295d73c8aa62fdb7f4832a3735e64abd96a2eb24137e1a54a"}
Sep 16 18:46:28.996662 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:28.996634    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sdw9s" event={"ID":"8ac962d8-4a04-40f2-a02c-ad61a26d1fbd","Type":"ContainerStarted","Data":"960df027157cdc5de25a0d0d2aac0adc55beaa8ee827136de016cdc3c52c387a"}
Sep 16 18:46:28.996817 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:28.996675    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sdw9s" event={"ID":"8ac962d8-4a04-40f2-a02c-ad61a26d1fbd","Type":"ContainerStarted","Data":"a9988fac651bed3ccd232a1e68c23eaaa2a18c6fac981e01d4661c8bb5afdaca"}
Sep 16 18:46:28.996817 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:28.996765    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-sdw9s"
Sep 16 18:46:28.998114 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:28.998091    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp" event={"ID":"c70456fc-7f1b-4b8c-bd77-fb3523016153","Type":"ContainerStarted","Data":"b9c5da604bd8eaf1363ab7daf38e5ee9cee3678243ffc215759c8eb032187877"}
Sep 16 18:46:28.998190 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:28.998129    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp" event={"ID":"c70456fc-7f1b-4b8c-bd77-fb3523016153","Type":"ContainerStarted","Data":"24c83f8e132817a6c0bbc525c8030bfdac73f23018f50092b426d674d6f5267c"}
Sep 16 18:46:29.044729 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:29.044678    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp" podStartSLOduration=18.044661365 podStartE2EDuration="18.044661365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:46:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:46:29.042935897 +0000 UTC m=+103.120040756" watchObservedRunningTime="2026-09-16 18:46:29.044661365 +0000 UTC m=+103.121766225"
Sep 16 18:46:29.045022 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:29.044998    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-sdw9s" podStartSLOduration=65.376748367 podStartE2EDuration="1m7.044991066s" totalImagesPullingTime="1.668242699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:45:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:46:29.019846115 +0000 UTC m=+103.096950977" watchObservedRunningTime="2026-09-16 18:46:29.044991066 +0000 UTC m=+103.122095925"
Sep 16 18:46:29.482952 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:29.482917    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:29.485545 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:29.485523    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:30.002294 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:30.002237    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:30.003486 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:30.003459    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6cd8c6bfcd-mctrp"
Sep 16 18:46:31.007459 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:31.007353    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rtk6r" event={"ID":"5cb6831f-9106-490f-b51c-5d8bd08a347a","Type":"ContainerStarted","Data":"a09a4216e7aa75e262d89a6abeaf953ddb89b550982db4c6bd58d1029ee16c87"}
Sep 16 18:46:31.007459 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:31.007415    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rtk6r" event={"ID":"5cb6831f-9106-490f-b51c-5d8bd08a347a","Type":"ContainerStarted","Data":"0449db38b621d11463389129db233e462e82da11b7c59625ed0cd2adfa50ea08"}
Sep 16 18:46:31.027623 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:31.027578    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-rtk6r" podStartSLOduration=17.643268254 podStartE2EDuration="20.027561934s" totalImagesPullingTime="2.38429368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:46:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:46:31.027025917 +0000 UTC m=+105.104130769" watchObservedRunningTime="2026-09-16 18:46:31.027561934 +0000 UTC m=+105.104666794"
Sep 16 18:46:31.718395 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:31.718360    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m8kzv\" (UID: \"b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53\") " pod="openshift-insights/insights-runtime-extractor-m8kzv"
Sep 16 18:46:31.718395 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:31.718402    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m8kzv\" (UID: \"b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53\") " pod="openshift-insights/insights-runtime-extractor-m8kzv"
Sep 16 18:46:31.718966 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:31.718936    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m8kzv\" (UID: \"b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53\") " pod="openshift-insights/insights-runtime-extractor-m8kzv"
Sep 16 18:46:31.720641 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:31.720624    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m8kzv\" (UID: \"b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53\") " pod="openshift-insights/insights-runtime-extractor-m8kzv"
Sep 16 18:46:31.793014 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:31.792977    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m8kzv"
Sep 16 18:46:32.076865 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:32.076838    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m8kzv"]
Sep 16 18:46:32.079219 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:46:32.079189    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb237c1e5_ba74_4f7c_b6d9_9c4f85d7bb53.slice/crio-2423a1bd8f80384a7bef74b1e1be527a8a7f9501500c982e1bf19c4971ee97a0 WatchSource:0}: Error finding container 2423a1bd8f80384a7bef74b1e1be527a8a7f9501500c982e1bf19c4971ee97a0: Status 404 returned error can't find the container with id 2423a1bd8f80384a7bef74b1e1be527a8a7f9501500c982e1bf19c4971ee97a0
Sep 16 18:46:33.014527 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:33.014486    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m8kzv" event={"ID":"b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53","Type":"ContainerStarted","Data":"6d68ce1190af10646ba3239addf97148ac49b1c17ea86f6d080f4ba1faae721a"}
Sep 16 18:46:33.014676 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:33.014549    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m8kzv" event={"ID":"b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53","Type":"ContainerStarted","Data":"2423a1bd8f80384a7bef74b1e1be527a8a7f9501500c982e1bf19c4971ee97a0"}
Sep 16 18:46:34.020757 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:34.020712    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m8kzv" event={"ID":"b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53","Type":"ContainerStarted","Data":"09f8289166a8100bb1ee2de5dcd9f9e55c5512548e2a0547b340ef5f6c7b6ece"}
Sep 16 18:46:35.025640 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:35.025603    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m8kzv" event={"ID":"b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53","Type":"ContainerStarted","Data":"dd1c07880af673a8371dcbb39a6f5d9068423ea9351b3f73a6b08e898d05ef7b"}
Sep 16 18:46:35.055896 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:35.055844    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-m8kzv" podStartSLOduration=17.971486709 podStartE2EDuration="20.055826708s" totalImagesPullingTime="2.084339999s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:46:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:46:35.055560479 +0000 UTC m=+109.132665338" watchObservedRunningTime="2026-09-16 18:46:35.055826708 +0000 UTC m=+109.132931568"
Sep 16 18:46:39.004458 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:39.004420    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-sdw9s"
Sep 16 18:46:43.371555 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:43.371526    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-c75d9488-qzft7"]
Sep 16 18:46:43.376293 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:43.376263    2557 patch_prober.go:28] interesting pod/image-registry-c75d9488-qzft7 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 16 18:46:43.376439 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:43.376312    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-c75d9488-qzft7" podUID="54674a4a-c1b7-429a-a4a9-21f7f93d22ab" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 18:46:50.634818 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.634783    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-72c5r"]
Sep 16 18:46:50.639136 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.639113    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-72c5r"
Sep 16 18:46:50.643977 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.643949    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 16 18:46:50.644240 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.644223    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 16 18:46:50.644709 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.644688    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 16 18:46:50.644847 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.644688    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 16 18:46:50.646047 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.646029    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 16 18:46:50.646528 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.646510    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-4xf6w\""
Sep 16 18:46:50.646989 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.646968    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-r9sr5"]
Sep 16 18:46:50.649978 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.649962    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-r9sr5"
Sep 16 18:46:50.653103 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.653080    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 16 18:46:50.654022 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.654004    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 16 18:46:50.654620 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.654604    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 16 18:46:50.655407 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.655083    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-hwrw8\""
Sep 16 18:46:50.659705 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.659683    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xfmzw"]
Sep 16 18:46:50.663796 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.662936    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xfmzw"
Sep 16 18:46:50.664748 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.664720    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-72c5r"]
Sep 16 18:46:50.665563 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.665544    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-22rpb\""
Sep 16 18:46:50.665811 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.665794    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 16 18:46:50.666047 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.666032    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 16 18:46:50.666546 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.666527    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-r9sr5"]
Sep 16 18:46:50.668055 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.668037    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 16 18:46:50.749531 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.749492    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c63581c7-f628-4dcd-9f39-a88418aff6e0-node-exporter-wtmp\") pod \"node-exporter-xfmzw\" (UID: \"c63581c7-f628-4dcd-9f39-a88418aff6e0\") " pod="openshift-monitoring/node-exporter-xfmzw"
Sep 16 18:46:50.749531 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.749528    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c63581c7-f628-4dcd-9f39-a88418aff6e0-node-exporter-tls\") pod \"node-exporter-xfmzw\" (UID: \"c63581c7-f628-4dcd-9f39-a88418aff6e0\") " pod="openshift-monitoring/node-exporter-xfmzw"
Sep 16 18:46:50.749732 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.749567    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c63581c7-f628-4dcd-9f39-a88418aff6e0-sys\") pod \"node-exporter-xfmzw\" (UID: \"c63581c7-f628-4dcd-9f39-a88418aff6e0\") " pod="openshift-monitoring/node-exporter-xfmzw"
Sep 16 18:46:50.749732 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.749593    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c63581c7-f628-4dcd-9f39-a88418aff6e0-root\") pod \"node-exporter-xfmzw\" (UID: \"c63581c7-f628-4dcd-9f39-a88418aff6e0\") " pod="openshift-monitoring/node-exporter-xfmzw"
Sep 16 18:46:50.749732 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.749623    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c63581c7-f628-4dcd-9f39-a88418aff6e0-metrics-client-ca\") pod \"node-exporter-xfmzw\" (UID: \"c63581c7-f628-4dcd-9f39-a88418aff6e0\") " pod="openshift-monitoring/node-exporter-xfmzw"
Sep 16 18:46:50.749732 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.749643    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8c5c7bd7-cce3-4e15-8d25-07cea35438fc-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-r9sr5\" (UID: \"8c5c7bd7-cce3-4e15-8d25-07cea35438fc\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-r9sr5"
Sep 16 18:46:50.749732 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.749670    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8c5c7bd7-cce3-4e15-8d25-07cea35438fc-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-r9sr5\" (UID: \"8c5c7bd7-cce3-4e15-8d25-07cea35438fc\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-r9sr5"
Sep 16 18:46:50.749940 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.749753    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn6dj\" (UniqueName: \"kubernetes.io/projected/8c5c7bd7-cce3-4e15-8d25-07cea35438fc-kube-api-access-vn6dj\") pod \"kube-state-metrics-59fbdd77d-r9sr5\" (UID: \"8c5c7bd7-cce3-4e15-8d25-07cea35438fc\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-r9sr5"
Sep 16 18:46:50.749940 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.749795    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c63581c7-f628-4dcd-9f39-a88418aff6e0-node-exporter-accelerators-collector-config\") pod \"node-exporter-xfmzw\" (UID: \"c63581c7-f628-4dcd-9f39-a88418aff6e0\") " pod="openshift-monitoring/node-exporter-xfmzw"
Sep 16 18:46:50.749940 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.749823    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2696852b-3bfd-4f06-8378-f88b8485e27a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-72c5r\" (UID: \"2696852b-3bfd-4f06-8378-f88b8485e27a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-72c5r"
Sep 16 18:46:50.749940 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.749840    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c63581c7-f628-4dcd-9f39-a88418aff6e0-node-exporter-textfile\") pod \"node-exporter-xfmzw\" (UID: \"c63581c7-f628-4dcd-9f39-a88418aff6e0\") " pod="openshift-monitoring/node-exporter-xfmzw"
Sep 16 18:46:50.749940 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.749885    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjbqz\" (UniqueName: \"kubernetes.io/projected/2696852b-3bfd-4f06-8378-f88b8485e27a-kube-api-access-mjbqz\") pod \"openshift-state-metrics-67dcf8c979-72c5r\" (UID: \"2696852b-3bfd-4f06-8378-f88b8485e27a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-72c5r"
Sep 16 18:46:50.749940 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.749925    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c5c7bd7-cce3-4e15-8d25-07cea35438fc-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-r9sr5\" (UID: \"8c5c7bd7-cce3-4e15-8d25-07cea35438fc\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-r9sr5"
Sep 16 18:46:50.750157 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.749952    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c5c7bd7-cce3-4e15-8d25-07cea35438fc-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-r9sr5\" (UID: \"8c5c7bd7-cce3-4e15-8d25-07cea35438fc\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-r9sr5"
Sep 16 18:46:50.750157 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.749976    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62lxn\" (UniqueName: \"kubernetes.io/projected/c63581c7-f628-4dcd-9f39-a88418aff6e0-kube-api-access-62lxn\") pod \"node-exporter-xfmzw\" (UID: \"c63581c7-f628-4dcd-9f39-a88418aff6e0\") " pod="openshift-monitoring/node-exporter-xfmzw"
Sep 16 18:46:50.750157 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.750001    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2696852b-3bfd-4f06-8378-f88b8485e27a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-72c5r\" (UID: \"2696852b-3bfd-4f06-8378-f88b8485e27a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-72c5r"
Sep 16 18:46:50.750157 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.750028    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c63581c7-f628-4dcd-9f39-a88418aff6e0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xfmzw\" (UID: \"c63581c7-f628-4dcd-9f39-a88418aff6e0\") " pod="openshift-monitoring/node-exporter-xfmzw"
Sep 16 18:46:50.750157 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.750054    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2696852b-3bfd-4f06-8378-f88b8485e27a-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-72c5r\" (UID: \"2696852b-3bfd-4f06-8378-f88b8485e27a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-72c5r"
Sep 16 18:46:50.750157 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.750078    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c5c7bd7-cce3-4e15-8d25-07cea35438fc-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-r9sr5\" (UID: \"8c5c7bd7-cce3-4e15-8d25-07cea35438fc\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-r9sr5"
Sep 16 18:46:50.850999 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.850965    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn6dj\" (UniqueName: \"kubernetes.io/projected/8c5c7bd7-cce3-4e15-8d25-07cea35438fc-kube-api-access-vn6dj\") pod \"kube-state-metrics-59fbdd77d-r9sr5\" (UID: \"8c5c7bd7-cce3-4e15-8d25-07cea35438fc\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-r9sr5"
Sep 16 18:46:50.850999 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.851003    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c63581c7-f628-4dcd-9f39-a88418aff6e0-node-exporter-accelerators-collector-config\") pod \"node-exporter-xfmzw\" (UID: \"c63581c7-f628-4dcd-9f39-a88418aff6e0\") " pod="openshift-monitoring/node-exporter-xfmzw"
Sep 16 18:46:50.851262 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.851026    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2696852b-3bfd-4f06-8378-f88b8485e27a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-72c5r\" (UID: \"2696852b-3bfd-4f06-8378-f88b8485e27a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-72c5r"
Sep 16 18:46:50.851262 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.851052    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c63581c7-f628-4dcd-9f39-a88418aff6e0-node-exporter-textfile\") pod \"node-exporter-xfmzw\" (UID: \"c63581c7-f628-4dcd-9f39-a88418aff6e0\") " pod="openshift-monitoring/node-exporter-xfmzw"
Sep 16 18:46:50.851262 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.851081    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjbqz\" (UniqueName: \"kubernetes.io/projected/2696852b-3bfd-4f06-8378-f88b8485e27a-kube-api-access-mjbqz\") pod \"openshift-state-metrics-67dcf8c979-72c5r\" (UID: \"2696852b-3bfd-4f06-8378-f88b8485e27a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-72c5r"
Sep 16 18:46:50.851262 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.851123    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c5c7bd7-cce3-4e15-8d25-07cea35438fc-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-r9sr5\" (UID: \"8c5c7bd7-cce3-4e15-8d25-07cea35438fc\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-r9sr5"
Sep 16 18:46:50.851463 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.851427    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c5c7bd7-cce3-4e15-8d25-07cea35438fc-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-r9sr5\" (UID: \"8c5c7bd7-cce3-4e15-8d25-07cea35438fc\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-r9sr5"
Sep 16 18:46:50.851519 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.851467    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62lxn\" (UniqueName: \"kubernetes.io/projected/c63581c7-f628-4dcd-9f39-a88418aff6e0-kube-api-access-62lxn\") pod \"node-exporter-xfmzw\" (UID: \"c63581c7-f628-4dcd-9f39-a88418aff6e0\") " pod="openshift-monitoring/node-exporter-xfmzw"
Sep 16 18:46:50.851519 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.851494    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2696852b-3bfd-4f06-8378-f88b8485e27a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-72c5r\" (UID: \"2696852b-3bfd-4f06-8378-f88b8485e27a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-72c5r"
Sep 16 18:46:50.851616 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.851523    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c63581c7-f628-4dcd-9f39-a88418aff6e0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xfmzw\" (UID: \"c63581c7-f628-4dcd-9f39-a88418aff6e0\") " pod="openshift-monitoring/node-exporter-xfmzw"
Sep 16 18:46:50.851616 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.851529    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c63581c7-f628-4dcd-9f39-a88418aff6e0-node-exporter-textfile\") pod \"node-exporter-xfmzw\" (UID: \"c63581c7-f628-4dcd-9f39-a88418aff6e0\") " pod="openshift-monitoring/node-exporter-xfmzw"
Sep 16 18:46:50.851721 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:50.851624    2557 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 16 18:46:50.851721 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:50.851678    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2696852b-3bfd-4f06-8378-f88b8485e27a-openshift-state-metrics-tls podName:2696852b-3bfd-4f06-8378-f88b8485e27a nodeName:}" failed. No retries permitted until 2026-09-16 18:46:51.351660593 +0000 UTC m=+125.428765431 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/2696852b-3bfd-4f06-8378-f88b8485e27a-openshift-state-metrics-tls") pod "openshift-state-metrics-67dcf8c979-72c5r" (UID: "2696852b-3bfd-4f06-8378-f88b8485e27a") : secret "openshift-state-metrics-tls" not found
Sep 16 18:46:50.851856 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.851732    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c63581c7-f628-4dcd-9f39-a88418aff6e0-node-exporter-accelerators-collector-config\") pod \"node-exporter-xfmzw\" (UID: \"c63581c7-f628-4dcd-9f39-a88418aff6e0\") " pod="openshift-monitoring/node-exporter-xfmzw"
Sep 16 18:46:50.851856 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.851793    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2696852b-3bfd-4f06-8378-f88b8485e27a-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-72c5r\" (UID: \"2696852b-3bfd-4f06-8378-f88b8485e27a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-72c5r"
Sep 16 18:46:50.851856 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.851834    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c5c7bd7-cce3-4e15-8d25-07cea35438fc-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-r9sr5\" (UID: \"8c5c7bd7-cce3-4e15-8d25-07cea35438fc\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-r9sr5"
Sep 16 18:46:50.852016 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.851880    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c63581c7-f628-4dcd-9f39-a88418aff6e0-node-exporter-wtmp\") pod \"node-exporter-xfmzw\" (UID: \"c63581c7-f628-4dcd-9f39-a88418aff6e0\") " pod="openshift-monitoring/node-exporter-xfmzw"
Sep 16 18:46:50.852016 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.851911    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c63581c7-f628-4dcd-9f39-a88418aff6e0-node-exporter-tls\") pod \"node-exporter-xfmzw\" (UID: \"c63581c7-f628-4dcd-9f39-a88418aff6e0\") " pod="openshift-monitoring/node-exporter-xfmzw"
Sep 16 18:46:50.852016 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.851956    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c63581c7-f628-4dcd-9f39-a88418aff6e0-sys\") pod \"node-exporter-xfmzw\" (UID: \"c63581c7-f628-4dcd-9f39-a88418aff6e0\") " pod="openshift-monitoring/node-exporter-xfmzw"
Sep 16 18:46:50.852016 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.851997    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c63581c7-f628-4dcd-9f39-a88418aff6e0-root\") pod \"node-exporter-xfmzw\" (UID: \"c63581c7-f628-4dcd-9f39-a88418aff6e0\") " pod="openshift-monitoring/node-exporter-xfmzw"
Sep 16 18:46:50.852209 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.852044    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c63581c7-f628-4dcd-9f39-a88418aff6e0-metrics-client-ca\") pod \"node-exporter-xfmzw\" (UID: \"c63581c7-f628-4dcd-9f39-a88418aff6e0\") " pod="openshift-monitoring/node-exporter-xfmzw"
Sep 16 18:46:50.852209 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.852073    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8c5c7bd7-cce3-4e15-8d25-07cea35438fc-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-r9sr5\" (UID: \"8c5c7bd7-cce3-4e15-8d25-07cea35438fc\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-r9sr5"
Sep 16 18:46:50.852209 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.852116    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8c5c7bd7-cce3-4e15-8d25-07cea35438fc-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-r9sr5\" (UID: \"8c5c7bd7-cce3-4e15-8d25-07cea35438fc\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-r9sr5"
Sep 16 18:46:50.852361 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.852247    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c5c7bd7-cce3-4e15-8d25-07cea35438fc-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-r9sr5\" (UID: \"8c5c7bd7-cce3-4e15-8d25-07cea35438fc\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-r9sr5"
Sep 16 18:46:50.852361 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.852320    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c63581c7-f628-4dcd-9f39-a88418aff6e0-sys\") pod \"node-exporter-xfmzw\" (UID: \"c63581c7-f628-4dcd-9f39-a88418aff6e0\") " pod="openshift-monitoring/node-exporter-xfmzw"
Sep 16 18:46:50.852526 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.852435    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c63581c7-f628-4dcd-9f39-a88418aff6e0-node-exporter-wtmp\") pod \"node-exporter-xfmzw\" (UID: \"c63581c7-f628-4dcd-9f39-a88418aff6e0\") " pod="openshift-monitoring/node-exporter-xfmzw"
Sep 16 18:46:50.852526 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.852455    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8c5c7bd7-cce3-4e15-8d25-07cea35438fc-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-r9sr5\" (UID: \"8c5c7bd7-cce3-4e15-8d25-07cea35438fc\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-r9sr5"
Sep 16 18:46:50.852526 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.852515    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2696852b-3bfd-4f06-8378-f88b8485e27a-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-72c5r\" (UID: \"2696852b-3bfd-4f06-8378-f88b8485e27a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-72c5r"
Sep 16 18:46:50.852724 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.852522    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c63581c7-f628-4dcd-9f39-a88418aff6e0-root\") pod \"node-exporter-xfmzw\" (UID: \"c63581c7-f628-4dcd-9f39-a88418aff6e0\") " pod="openshift-monitoring/node-exporter-xfmzw"
Sep 16 18:46:50.853137 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.853113    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c63581c7-f628-4dcd-9f39-a88418aff6e0-metrics-client-ca\") pod \"node-exporter-xfmzw\" (UID: \"c63581c7-f628-4dcd-9f39-a88418aff6e0\") " pod="openshift-monitoring/node-exporter-xfmzw"
Sep 16 18:46:50.853283 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.853161    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8c5c7bd7-cce3-4e15-8d25-07cea35438fc-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-r9sr5\" (UID: \"8c5c7bd7-cce3-4e15-8d25-07cea35438fc\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-r9sr5"
Sep 16 18:46:50.854238 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.854215    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c63581c7-f628-4dcd-9f39-a88418aff6e0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xfmzw\" (UID: \"c63581c7-f628-4dcd-9f39-a88418aff6e0\") " pod="openshift-monitoring/node-exporter-xfmzw"
Sep 16 18:46:50.854418 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.854400    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c5c7bd7-cce3-4e15-8d25-07cea35438fc-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-r9sr5\" (UID: \"8c5c7bd7-cce3-4e15-8d25-07cea35438fc\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-r9sr5"
Sep 16 18:46:50.855083 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.855063    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2696852b-3bfd-4f06-8378-f88b8485e27a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-72c5r\" (UID: \"2696852b-3bfd-4f06-8378-f88b8485e27a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-72c5r"
Sep 16 18:46:50.855284 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.855265    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8c5c7bd7-cce3-4e15-8d25-07cea35438fc-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-r9sr5\" (UID: \"8c5c7bd7-cce3-4e15-8d25-07cea35438fc\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-r9sr5"
Sep 16 18:46:50.856000 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.855980    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c63581c7-f628-4dcd-9f39-a88418aff6e0-node-exporter-tls\") pod \"node-exporter-xfmzw\" (UID: \"c63581c7-f628-4dcd-9f39-a88418aff6e0\") " pod="openshift-monitoring/node-exporter-xfmzw"
Sep 16 18:46:50.862553 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.862528    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjbqz\" (UniqueName: \"kubernetes.io/projected/2696852b-3bfd-4f06-8378-f88b8485e27a-kube-api-access-mjbqz\") pod \"openshift-state-metrics-67dcf8c979-72c5r\" (UID: \"2696852b-3bfd-4f06-8378-f88b8485e27a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-72c5r"
Sep 16 18:46:50.863144 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.863121    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62lxn\" (UniqueName: \"kubernetes.io/projected/c63581c7-f628-4dcd-9f39-a88418aff6e0-kube-api-access-62lxn\") pod \"node-exporter-xfmzw\" (UID: \"c63581c7-f628-4dcd-9f39-a88418aff6e0\") " pod="openshift-monitoring/node-exporter-xfmzw"
Sep 16 18:46:50.864464 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.864445    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn6dj\" (UniqueName: \"kubernetes.io/projected/8c5c7bd7-cce3-4e15-8d25-07cea35438fc-kube-api-access-vn6dj\") pod \"kube-state-metrics-59fbdd77d-r9sr5\" (UID: \"8c5c7bd7-cce3-4e15-8d25-07cea35438fc\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-r9sr5"
Sep 16 18:46:50.961107 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.961021    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-r9sr5"
Sep 16 18:46:50.976199 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:50.976173    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xfmzw"
Sep 16 18:46:50.988820 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:46:50.988795    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc63581c7_f628_4dcd_9f39_a88418aff6e0.slice/crio-0825b4c59ba11e577dde74361611278e1f2049f5eccc1c9d18f0f68d87f26fb1 WatchSource:0}: Error finding container 0825b4c59ba11e577dde74361611278e1f2049f5eccc1c9d18f0f68d87f26fb1: Status 404 returned error can't find the container with id 0825b4c59ba11e577dde74361611278e1f2049f5eccc1c9d18f0f68d87f26fb1
Sep 16 18:46:51.083557 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.083520    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xfmzw" event={"ID":"c63581c7-f628-4dcd-9f39-a88418aff6e0","Type":"ContainerStarted","Data":"0825b4c59ba11e577dde74361611278e1f2049f5eccc1c9d18f0f68d87f26fb1"}
Sep 16 18:46:51.243055 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.242984    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-r9sr5"]
Sep 16 18:46:51.245700 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:46:51.245667    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c5c7bd7_cce3_4e15_8d25_07cea35438fc.slice/crio-2422856f4d699ff481dea392924433fc40a4b1880e69659f591afd8c65e2130a WatchSource:0}: Error finding container 2422856f4d699ff481dea392924433fc40a4b1880e69659f591afd8c65e2130a: Status 404 returned error can't find the container with id 2422856f4d699ff481dea392924433fc40a4b1880e69659f591afd8c65e2130a
Sep 16 18:46:51.356538 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.356512    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2696852b-3bfd-4f06-8378-f88b8485e27a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-72c5r\" (UID: \"2696852b-3bfd-4f06-8378-f88b8485e27a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-72c5r"
Sep 16 18:46:51.358681 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.358659    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2696852b-3bfd-4f06-8378-f88b8485e27a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-72c5r\" (UID: \"2696852b-3bfd-4f06-8378-f88b8485e27a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-72c5r"
Sep 16 18:46:51.551054 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.551019    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-72c5r"
Sep 16 18:46:51.699800 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.699666    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 18:46:51.703905 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.703880    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.708133 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.707862    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-zss6w\""
Sep 16 18:46:51.708133 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.707878    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 16 18:46:51.708133 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.707890    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 16 18:46:51.708413 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.708235    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 16 18:46:51.708413 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.708317    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 16 18:46:51.708413 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.708235    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 16 18:46:51.708625 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.708497    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 16 18:46:51.708625 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.708619    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 16 18:46:51.709062 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.709044    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 16 18:46:51.709414 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.709316    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 16 18:46:51.725054 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.725033    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 18:46:51.831868 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.831763    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-72c5r"]
Sep 16 18:46:51.862704 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.862062    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5e719e21-4e76-4c1c-89aa-032e80db0638-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.862704 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.862107    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.862704 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.862148    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.862704 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.862180    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.862704 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.862221    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.862704 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.862248    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5e719e21-4e76-4c1c-89aa-032e80db0638-config-out\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.862704 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.862285    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5e719e21-4e76-4c1c-89aa-032e80db0638-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.862704 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.862356    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e719e21-4e76-4c1c-89aa-032e80db0638-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.862704 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.862420    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-web-config\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.862704 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.862452    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-config-volume\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.862704 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.862487    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e719e21-4e76-4c1c-89aa-032e80db0638-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.862704 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.862517    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.862704 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.862543    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dhgb\" (UniqueName: \"kubernetes.io/projected/5e719e21-4e76-4c1c-89aa-032e80db0638-kube-api-access-4dhgb\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.959299 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:46:51.959263    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2696852b_3bfd_4f06_8378_f88b8485e27a.slice/crio-05b54dc1fa6763dce5e7cf2bdd34cae267b69d36c2d2462024e9dc9654f733b2 WatchSource:0}: Error finding container 05b54dc1fa6763dce5e7cf2bdd34cae267b69d36c2d2462024e9dc9654f733b2: Status 404 returned error can't find the container with id 05b54dc1fa6763dce5e7cf2bdd34cae267b69d36c2d2462024e9dc9654f733b2
Sep 16 18:46:51.964090 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.964039    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5e719e21-4e76-4c1c-89aa-032e80db0638-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.964606 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.964367    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e719e21-4e76-4c1c-89aa-032e80db0638-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.964606 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.964506    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-web-config\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.964606 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.964540    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-config-volume\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.965083 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.964898    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e719e21-4e76-4c1c-89aa-032e80db0638-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.965083 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.964963    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.965083 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.965003    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dhgb\" (UniqueName: \"kubernetes.io/projected/5e719e21-4e76-4c1c-89aa-032e80db0638-kube-api-access-4dhgb\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.965083 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.965039    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5e719e21-4e76-4c1c-89aa-032e80db0638-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.965325 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.965064    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.965462 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.965381    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.965462 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.965422    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.965555 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.965464    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.966570 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.965512    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5e719e21-4e76-4c1c-89aa-032e80db0638-config-out\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.966570 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:51.965679    2557 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 16 18:46:51.966570 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:51.965790    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-secret-alertmanager-main-tls podName:5e719e21-4e76-4c1c-89aa-032e80db0638 nodeName:}" failed. No retries permitted until 2026-09-16 18:46:52.465725537 +0000 UTC m=+126.542830391 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "5e719e21-4e76-4c1c-89aa-032e80db0638") : secret "alertmanager-main-tls" not found
Sep 16 18:46:51.966755 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.966619    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e719e21-4e76-4c1c-89aa-032e80db0638-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.967285 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.967257    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5e719e21-4e76-4c1c-89aa-032e80db0638-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.969976 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.969573    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e719e21-4e76-4c1c-89aa-032e80db0638-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.971672 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.970186    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.973243 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.973185    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-web-config\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.974841 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.974780    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5e719e21-4e76-4c1c-89aa-032e80db0638-config-out\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.975434 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.975409    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.976260 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.976236    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.976391 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.976368    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.976454 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.976417    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5e719e21-4e76-4c1c-89aa-032e80db0638-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.977481 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.977452    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-config-volume\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:51.982142 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:51.982118    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dhgb\" (UniqueName: \"kubernetes.io/projected/5e719e21-4e76-4c1c-89aa-032e80db0638-kube-api-access-4dhgb\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:52.091584 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:52.091549    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-72c5r" event={"ID":"2696852b-3bfd-4f06-8378-f88b8485e27a","Type":"ContainerStarted","Data":"1622fccb371a55040582db9ea71579e50c36e6b3be110a8d6b4e01ad718c3747"}
Sep 16 18:46:52.091731 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:52.091659    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-72c5r" event={"ID":"2696852b-3bfd-4f06-8378-f88b8485e27a","Type":"ContainerStarted","Data":"05b54dc1fa6763dce5e7cf2bdd34cae267b69d36c2d2462024e9dc9654f733b2"}
Sep 16 18:46:52.093571 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:52.093539    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-r9sr5" event={"ID":"8c5c7bd7-cce3-4e15-8d25-07cea35438fc","Type":"ContainerStarted","Data":"2422856f4d699ff481dea392924433fc40a4b1880e69659f591afd8c65e2130a"}
Sep 16 18:46:52.471180 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:52.471114    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:52.473296 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:52.473271    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:52.616254 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:52.616067    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:46:52.920417 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:52.920389    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 18:46:53.101145 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:53.101097    2557 generic.go:356] "Generic (PLEG): container finished" podID="c63581c7-f628-4dcd-9f39-a88418aff6e0" containerID="ed01c3ce37d1ff23e0cb83de0a543747ed74a1785519c111148496e911cd471b" exitCode=0
Sep 16 18:46:53.101327 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:53.101166    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xfmzw" event={"ID":"c63581c7-f628-4dcd-9f39-a88418aff6e0","Type":"ContainerDied","Data":"ed01c3ce37d1ff23e0cb83de0a543747ed74a1785519c111148496e911cd471b"}
Sep 16 18:46:53.103422 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:53.103398    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-72c5r" event={"ID":"2696852b-3bfd-4f06-8378-f88b8485e27a","Type":"ContainerStarted","Data":"3b7158e9e836b19f582ed6f783da8ec29c11b4d629e6ce270bbad6c459b9caf7"}
Sep 16 18:46:53.105682 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:53.105656    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-r9sr5" event={"ID":"8c5c7bd7-cce3-4e15-8d25-07cea35438fc","Type":"ContainerStarted","Data":"104d2aaf8733c8a0995aa7caa0e7e3a1db8e1939405ffa3efd63ab9c982195a1"}
Sep 16 18:46:53.105805 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:53.105707    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-r9sr5" event={"ID":"8c5c7bd7-cce3-4e15-8d25-07cea35438fc","Type":"ContainerStarted","Data":"db045d7b507406889a07b17cc82e568744b34bd4b62d30d693a7d578e1201397"}
Sep 16 18:46:53.105805 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:53.105737    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-r9sr5" event={"ID":"8c5c7bd7-cce3-4e15-8d25-07cea35438fc","Type":"ContainerStarted","Data":"b4876ebdb0190eb29811ff0bdf177f9e4edb9ccb7335f5e2499849591f4f1b63"}
Sep 16 18:46:53.115740 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:46:53.115718    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e719e21_4e76_4c1c_89aa_032e80db0638.slice/crio-c8153dbb073e715186d4814c0f9a71ba6398943c5ae7424df85cc8696b06f735 WatchSource:0}: Error finding container c8153dbb073e715186d4814c0f9a71ba6398943c5ae7424df85cc8696b06f735: Status 404 returned error can't find the container with id c8153dbb073e715186d4814c0f9a71ba6398943c5ae7424df85cc8696b06f735
Sep 16 18:46:53.147845 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:53.147784    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-r9sr5" podStartSLOduration=1.9202713340000002 podStartE2EDuration="3.147753958s" totalImagesPullingTime="1.227482624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:46:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:46:53.146360913 +0000 UTC m=+127.223465770" watchObservedRunningTime="2026-09-16 18:46:53.147753958 +0000 UTC m=+127.224858818"
Sep 16 18:46:53.376438 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:53.376411    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:46:54.111179 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:54.111146    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xfmzw" event={"ID":"c63581c7-f628-4dcd-9f39-a88418aff6e0","Type":"ContainerStarted","Data":"5ab193493525821c322345ae8a2e5d3dda020085da1575a0be94132ba839ffa6"}
Sep 16 18:46:54.111489 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:54.111223    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xfmzw" event={"ID":"c63581c7-f628-4dcd-9f39-a88418aff6e0","Type":"ContainerStarted","Data":"ac4135339c5fdde24bf3f9959802e5df555d6e7a66d5ad76af6cfcedb63eb732"}
Sep 16 18:46:54.112350 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:54.112326    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5e719e21-4e76-4c1c-89aa-032e80db0638","Type":"ContainerStarted","Data":"c8153dbb073e715186d4814c0f9a71ba6398943c5ae7424df85cc8696b06f735"}
Sep 16 18:46:54.114606 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:54.114588    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-72c5r" event={"ID":"2696852b-3bfd-4f06-8378-f88b8485e27a","Type":"ContainerStarted","Data":"06316663c7b9e14d495ec23cf9208afbc01a30779d567f041b82cd378c78f014"}
Sep 16 18:46:54.132686 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:54.132641    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xfmzw" podStartSLOduration=3.03000288 podStartE2EDuration="4.132621695s" totalImagesPullingTime="1.030729231s" totalInitContainerRuntime="71.889584ms" isStatefulPod=true podCreationTimestamp="2026-09-16 18:46:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:46:54.131493127 +0000 UTC m=+128.208597986" watchObservedRunningTime="2026-09-16 18:46:54.132621695 +0000 UTC m=+128.209726558"
Sep 16 18:46:54.153588 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:54.153541    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-72c5r" podStartSLOduration=3.102870384 podStartE2EDuration="4.153525385s" totalImagesPullingTime="1.050655001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:46:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:46:54.152079233 +0000 UTC m=+128.229184095" watchObservedRunningTime="2026-09-16 18:46:54.153525385 +0000 UTC m=+128.230630245"
Sep 16 18:46:54.994375 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:54.994342    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"]
Sep 16 18:46:54.997548 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:54.997519    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"
Sep 16 18:46:55.001444 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.001418    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 16 18:46:55.001549 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.001466    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 16 18:46:55.002388 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.002372    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-pfmzw\""
Sep 16 18:46:55.003011 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.002992    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 16 18:46:55.003103 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.002997    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-eiqdq4mujfnci\""
Sep 16 18:46:55.003985 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.003949    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 16 18:46:55.015677 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.015651    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"]
Sep 16 18:46:55.092059 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.092015    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a388087b-93a5-410d-8fd8-d98a8ff1eb77-metrics-server-audit-profiles\") pod \"metrics-server-68b5bf68d8-vwbqr\" (UID: \"a388087b-93a5-410d-8fd8-d98a8ff1eb77\") " pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"
Sep 16 18:46:55.092059 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.092061    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a388087b-93a5-410d-8fd8-d98a8ff1eb77-audit-log\") pod \"metrics-server-68b5bf68d8-vwbqr\" (UID: \"a388087b-93a5-410d-8fd8-d98a8ff1eb77\") " pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"
Sep 16 18:46:55.092240 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.092081    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a388087b-93a5-410d-8fd8-d98a8ff1eb77-client-ca-bundle\") pod \"metrics-server-68b5bf68d8-vwbqr\" (UID: \"a388087b-93a5-410d-8fd8-d98a8ff1eb77\") " pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"
Sep 16 18:46:55.092240 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.092128    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rthjp\" (UniqueName: \"kubernetes.io/projected/a388087b-93a5-410d-8fd8-d98a8ff1eb77-kube-api-access-rthjp\") pod \"metrics-server-68b5bf68d8-vwbqr\" (UID: \"a388087b-93a5-410d-8fd8-d98a8ff1eb77\") " pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"
Sep 16 18:46:55.092240 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.092162    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a388087b-93a5-410d-8fd8-d98a8ff1eb77-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-68b5bf68d8-vwbqr\" (UID: \"a388087b-93a5-410d-8fd8-d98a8ff1eb77\") " pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"
Sep 16 18:46:55.092240 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.092226    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a388087b-93a5-410d-8fd8-d98a8ff1eb77-secret-metrics-server-client-certs\") pod \"metrics-server-68b5bf68d8-vwbqr\" (UID: \"a388087b-93a5-410d-8fd8-d98a8ff1eb77\") " pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"
Sep 16 18:46:55.092387 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.092247    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a388087b-93a5-410d-8fd8-d98a8ff1eb77-secret-metrics-server-tls\") pod \"metrics-server-68b5bf68d8-vwbqr\" (UID: \"a388087b-93a5-410d-8fd8-d98a8ff1eb77\") " pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"
Sep 16 18:46:55.118998 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.118974    2557 generic.go:356] "Generic (PLEG): container finished" podID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerID="775d108fc9f1143269474e5905706317b1d2be6c46273f5efff9324d7761c099" exitCode=0
Sep 16 18:46:55.119415 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.119063    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5e719e21-4e76-4c1c-89aa-032e80db0638","Type":"ContainerDied","Data":"775d108fc9f1143269474e5905706317b1d2be6c46273f5efff9324d7761c099"}
Sep 16 18:46:55.193376 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.193344    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a388087b-93a5-410d-8fd8-d98a8ff1eb77-secret-metrics-server-tls\") pod \"metrics-server-68b5bf68d8-vwbqr\" (UID: \"a388087b-93a5-410d-8fd8-d98a8ff1eb77\") " pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"
Sep 16 18:46:55.193540 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.193423    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a388087b-93a5-410d-8fd8-d98a8ff1eb77-metrics-server-audit-profiles\") pod \"metrics-server-68b5bf68d8-vwbqr\" (UID: \"a388087b-93a5-410d-8fd8-d98a8ff1eb77\") " pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"
Sep 16 18:46:55.193540 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.193445    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a388087b-93a5-410d-8fd8-d98a8ff1eb77-audit-log\") pod \"metrics-server-68b5bf68d8-vwbqr\" (UID: \"a388087b-93a5-410d-8fd8-d98a8ff1eb77\") " pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"
Sep 16 18:46:55.193540 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.193462    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a388087b-93a5-410d-8fd8-d98a8ff1eb77-client-ca-bundle\") pod \"metrics-server-68b5bf68d8-vwbqr\" (UID: \"a388087b-93a5-410d-8fd8-d98a8ff1eb77\") " pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"
Sep 16 18:46:55.193540 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.193487    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rthjp\" (UniqueName: \"kubernetes.io/projected/a388087b-93a5-410d-8fd8-d98a8ff1eb77-kube-api-access-rthjp\") pod \"metrics-server-68b5bf68d8-vwbqr\" (UID: \"a388087b-93a5-410d-8fd8-d98a8ff1eb77\") " pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"
Sep 16 18:46:55.193540 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.193515    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a388087b-93a5-410d-8fd8-d98a8ff1eb77-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-68b5bf68d8-vwbqr\" (UID: \"a388087b-93a5-410d-8fd8-d98a8ff1eb77\") " pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"
Sep 16 18:46:55.193874 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.193710    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a388087b-93a5-410d-8fd8-d98a8ff1eb77-secret-metrics-server-client-certs\") pod \"metrics-server-68b5bf68d8-vwbqr\" (UID: \"a388087b-93a5-410d-8fd8-d98a8ff1eb77\") " pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"
Sep 16 18:46:55.194025 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.194002    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a388087b-93a5-410d-8fd8-d98a8ff1eb77-audit-log\") pod \"metrics-server-68b5bf68d8-vwbqr\" (UID: \"a388087b-93a5-410d-8fd8-d98a8ff1eb77\") " pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"
Sep 16 18:46:55.194378 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.194348    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a388087b-93a5-410d-8fd8-d98a8ff1eb77-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-68b5bf68d8-vwbqr\" (UID: \"a388087b-93a5-410d-8fd8-d98a8ff1eb77\") " pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"
Sep 16 18:46:55.194567 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.194548    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a388087b-93a5-410d-8fd8-d98a8ff1eb77-metrics-server-audit-profiles\") pod \"metrics-server-68b5bf68d8-vwbqr\" (UID: \"a388087b-93a5-410d-8fd8-d98a8ff1eb77\") " pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"
Sep 16 18:46:55.195922 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.195903    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a388087b-93a5-410d-8fd8-d98a8ff1eb77-client-ca-bundle\") pod \"metrics-server-68b5bf68d8-vwbqr\" (UID: \"a388087b-93a5-410d-8fd8-d98a8ff1eb77\") " pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"
Sep 16 18:46:55.196216 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.196195    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a388087b-93a5-410d-8fd8-d98a8ff1eb77-secret-metrics-server-tls\") pod \"metrics-server-68b5bf68d8-vwbqr\" (UID: \"a388087b-93a5-410d-8fd8-d98a8ff1eb77\") " pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"
Sep 16 18:46:55.196262 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.196195    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a388087b-93a5-410d-8fd8-d98a8ff1eb77-secret-metrics-server-client-certs\") pod \"metrics-server-68b5bf68d8-vwbqr\" (UID: \"a388087b-93a5-410d-8fd8-d98a8ff1eb77\") " pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"
Sep 16 18:46:55.203470 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.203445    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rthjp\" (UniqueName: \"kubernetes.io/projected/a388087b-93a5-410d-8fd8-d98a8ff1eb77-kube-api-access-rthjp\") pod \"metrics-server-68b5bf68d8-vwbqr\" (UID: \"a388087b-93a5-410d-8fd8-d98a8ff1eb77\") " pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"
Sep 16 18:46:55.306704 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.306675    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"
Sep 16 18:46:55.360198 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.360122    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-nm7rr"]
Sep 16 18:46:55.366151 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.366131    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-nm7rr"
Sep 16 18:46:55.369098 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.368855    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-gmcjw\""
Sep 16 18:46:55.369098 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.369073    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 16 18:46:55.376351 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.376328    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-nm7rr"]
Sep 16 18:46:55.497128 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.497092    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a27c3da5-d71f-47bd-b483-f5f486da3a63-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-nm7rr\" (UID: \"a27c3da5-d71f-47bd-b483-f5f486da3a63\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-nm7rr"
Sep 16 18:46:55.497288 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.497158    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56hpb\" (UniqueName: \"kubernetes.io/projected/a27c3da5-d71f-47bd-b483-f5f486da3a63-kube-api-access-56hpb\") pod \"monitoring-plugin-5d85778b9d-nm7rr\" (UID: \"a27c3da5-d71f-47bd-b483-f5f486da3a63\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-nm7rr"
Sep 16 18:46:55.577645 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.577620    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"]
Sep 16 18:46:55.580708 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:46:55.580667    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda388087b_93a5_410d_8fd8_d98a8ff1eb77.slice/crio-186ffc670aca2f171aa87f9e7b20d6f864a1e2afe9c87a8cdc93cf808d10ead7 WatchSource:0}: Error finding container 186ffc670aca2f171aa87f9e7b20d6f864a1e2afe9c87a8cdc93cf808d10ead7: Status 404 returned error can't find the container with id 186ffc670aca2f171aa87f9e7b20d6f864a1e2afe9c87a8cdc93cf808d10ead7
Sep 16 18:46:55.597626 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.597601    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a27c3da5-d71f-47bd-b483-f5f486da3a63-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-nm7rr\" (UID: \"a27c3da5-d71f-47bd-b483-f5f486da3a63\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-nm7rr"
Sep 16 18:46:55.597732 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.597676    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56hpb\" (UniqueName: \"kubernetes.io/projected/a27c3da5-d71f-47bd-b483-f5f486da3a63-kube-api-access-56hpb\") pod \"monitoring-plugin-5d85778b9d-nm7rr\" (UID: \"a27c3da5-d71f-47bd-b483-f5f486da3a63\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-nm7rr"
Sep 16 18:46:55.597803 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:55.597753    2557 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 16 18:46:55.597866 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:46:55.597848    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a27c3da5-d71f-47bd-b483-f5f486da3a63-monitoring-plugin-cert podName:a27c3da5-d71f-47bd-b483-f5f486da3a63 nodeName:}" failed. No retries permitted until 2026-09-16 18:46:56.097826712 +0000 UTC m=+130.174931567 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/a27c3da5-d71f-47bd-b483-f5f486da3a63-monitoring-plugin-cert") pod "monitoring-plugin-5d85778b9d-nm7rr" (UID: "a27c3da5-d71f-47bd-b483-f5f486da3a63") : secret "monitoring-plugin-cert" not found
Sep 16 18:46:55.607983 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:55.607953    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56hpb\" (UniqueName: \"kubernetes.io/projected/a27c3da5-d71f-47bd-b483-f5f486da3a63-kube-api-access-56hpb\") pod \"monitoring-plugin-5d85778b9d-nm7rr\" (UID: \"a27c3da5-d71f-47bd-b483-f5f486da3a63\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-nm7rr"
Sep 16 18:46:56.103892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.103852    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a27c3da5-d71f-47bd-b483-f5f486da3a63-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-nm7rr\" (UID: \"a27c3da5-d71f-47bd-b483-f5f486da3a63\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-nm7rr"
Sep 16 18:46:56.106793 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.106751    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a27c3da5-d71f-47bd-b483-f5f486da3a63-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-nm7rr\" (UID: \"a27c3da5-d71f-47bd-b483-f5f486da3a63\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-nm7rr"
Sep 16 18:46:56.125819 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.125759    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr" event={"ID":"a388087b-93a5-410d-8fd8-d98a8ff1eb77","Type":"ContainerStarted","Data":"186ffc670aca2f171aa87f9e7b20d6f864a1e2afe9c87a8cdc93cf808d10ead7"}
Sep 16 18:46:56.205407 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.205373    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c19f514e-c3a8-41d2-9de1-ad2d86fce986-metrics-certs\") pod \"network-metrics-daemon-4wnsv\" (UID: \"c19f514e-c3a8-41d2-9de1-ad2d86fce986\") " pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:46:56.208215 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.208178    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c19f514e-c3a8-41d2-9de1-ad2d86fce986-metrics-certs\") pod \"network-metrics-daemon-4wnsv\" (UID: \"c19f514e-c3a8-41d2-9de1-ad2d86fce986\") " pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:46:56.277518 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.277482    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-nm7rr"
Sep 16 18:46:56.372170 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.372085    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dgmmm\""
Sep 16 18:46:56.379404 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.379368    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wnsv"
Sep 16 18:46:56.807068 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.807024    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 18:46:56.811565 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.811541    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:56.817940 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.817744    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 16 18:46:56.817940 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.817765    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 16 18:46:56.817940 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.817813    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 16 18:46:56.817940 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.817919    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-qtbxc\""
Sep 16 18:46:56.818284 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.818259    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 16 18:46:56.818496 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.818476    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 16 18:46:56.819541 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.819248    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 16 18:46:56.819541 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.819344    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 16 18:46:56.820443 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.820425    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-f4m3f7i7atasa\""
Sep 16 18:46:56.821077 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.821058    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 16 18:46:56.821156 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.821068    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 16 18:46:56.824537 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.824515    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 16 18:46:56.824702 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.824683    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 16 18:46:56.825037 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.825020    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 16 18:46:56.826570 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.826550    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 16 18:46:56.828290 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.828266    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 16 18:46:56.842282 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.842248    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 18:46:56.912132 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.912100    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8dmc\" (UniqueName: \"kubernetes.io/projected/4639fcfa-4d2b-4f72-be92-1c9a3954c101-kube-api-access-r8dmc\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:56.912278 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.912145    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:56.912278 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.912222    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4639fcfa-4d2b-4f72-be92-1c9a3954c101-config-out\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:56.912278 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.912249    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:56.912278 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.912270    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:56.912411 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.912290    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-web-config\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:56.912411 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.912315    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:56.912411 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.912334    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:56.912411 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.912361    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-config\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:56.912577 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.912419    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:56.912577 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.912453    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:56.912577 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.912472    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:56.912577 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.912538    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:56.912577 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.912570    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:56.912837 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.912593    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:56.912837 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.912622    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:56.912837 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.912663    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4639fcfa-4d2b-4f72-be92-1c9a3954c101-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:56.912837 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.912684    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:56.912837 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.912721    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:56.912837 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:56.912815    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.014222 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.013890    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.014222 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.013941    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.014222 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.013968    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.014222 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.014002    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.014222 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.014046    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4639fcfa-4d2b-4f72-be92-1c9a3954c101-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.014222 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.014072    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.014222 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.014097    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.014222 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.014125    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.014222 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.014150    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8dmc\" (UniqueName: \"kubernetes.io/projected/4639fcfa-4d2b-4f72-be92-1c9a3954c101-kube-api-access-r8dmc\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.014222 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.014187    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.014764 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.014251    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4639fcfa-4d2b-4f72-be92-1c9a3954c101-config-out\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.014764 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.014279    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.014764 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.014308    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.014764 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.014337    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-web-config\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.014764 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.014361    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.014764 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.014389    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.014764 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.014422    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-config\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.014764 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.014449    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.014764 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.014474    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.014764 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.014502    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.015995 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.015698    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.020851 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.017839    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.020851 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.018355    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.020851 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.020434    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.021841 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.021708    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.022822 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.022334    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.022822 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.022576    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.023584 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.023148    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.023584 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.023337    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4639fcfa-4d2b-4f72-be92-1c9a3954c101-config-out\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.023584 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.023505    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.024091 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.024067    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-web-config\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.025742 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.025558    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-config\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.025742 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.025698    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.026301 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.026173    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.026301 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.026239    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.026301 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.026251    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4639fcfa-4d2b-4f72-be92-1c9a3954c101-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.026608 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.026588    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.026914 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.026876    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.027361 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.027340    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.032570 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.032547    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8dmc\" (UniqueName: \"kubernetes.io/projected/4639fcfa-4d2b-4f72-be92-1c9a3954c101-kube-api-access-r8dmc\") pod \"prometheus-k8s-0\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.124001 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.123978    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:46:57.132785 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.132740    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr" event={"ID":"a388087b-93a5-410d-8fd8-d98a8ff1eb77","Type":"ContainerStarted","Data":"da4caa92f9d4df738cf8725f3625a039b226a1f6eab2a72f30a8a534ec97ed99"}
Sep 16 18:46:57.135315 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.135275    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5e719e21-4e76-4c1c-89aa-032e80db0638","Type":"ContainerStarted","Data":"948f22f715318b554c954317592f1376e63b55fdb29f1afb46b422264a390b48"}
Sep 16 18:46:57.135381 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.135347    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5e719e21-4e76-4c1c-89aa-032e80db0638","Type":"ContainerStarted","Data":"fb09cad336313c3cf657cc1a5d1c6bfdfd9efec3b5933442611924287e2151be"}
Sep 16 18:46:57.155186 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.155115    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr" podStartSLOduration=1.784027171 podStartE2EDuration="3.155099043s" totalImagesPullingTime="1.371071872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:46:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:46:57.15468827 +0000 UTC m=+131.231793131" watchObservedRunningTime="2026-09-16 18:46:57.155099043 +0000 UTC m=+131.232203903"
Sep 16 18:46:57.227270 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.226756    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-nm7rr"]
Sep 16 18:46:57.229560 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.229540    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4wnsv"]
Sep 16 18:46:57.236403 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:46:57.236357    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda27c3da5_d71f_47bd_b483_f5f486da3a63.slice/crio-93eba4d8a608b9d55b453da041ce1e93613f6d891a544b6c8b55ec64b68a6804 WatchSource:0}: Error finding container 93eba4d8a608b9d55b453da041ce1e93613f6d891a544b6c8b55ec64b68a6804: Status 404 returned error can't find the container with id 93eba4d8a608b9d55b453da041ce1e93613f6d891a544b6c8b55ec64b68a6804
Sep 16 18:46:57.417481 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:57.417450    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 18:46:57.420642 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:46:57.420594    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4639fcfa_4d2b_4f72_be92_1c9a3954c101.slice/crio-44fe27ee41353f575a29fdd135ae8346bce3587d8446173328a7fc7e92ce8e40 WatchSource:0}: Error finding container 44fe27ee41353f575a29fdd135ae8346bce3587d8446173328a7fc7e92ce8e40: Status 404 returned error can't find the container with id 44fe27ee41353f575a29fdd135ae8346bce3587d8446173328a7fc7e92ce8e40
Sep 16 18:46:58.147300 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:58.147266    2557 generic.go:356] "Generic (PLEG): container finished" podID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerID="b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104" exitCode=0
Sep 16 18:46:58.147785 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:58.147354    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4639fcfa-4d2b-4f72-be92-1c9a3954c101","Type":"ContainerDied","Data":"b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104"}
Sep 16 18:46:58.147785 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:58.147479    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4639fcfa-4d2b-4f72-be92-1c9a3954c101","Type":"ContainerStarted","Data":"44fe27ee41353f575a29fdd135ae8346bce3587d8446173328a7fc7e92ce8e40"}
Sep 16 18:46:58.151136 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:58.151114    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5e719e21-4e76-4c1c-89aa-032e80db0638","Type":"ContainerStarted","Data":"f3b641a5b055a849a2438acca64835b4b6948479b9edab8222e03396ad5d52ca"}
Sep 16 18:46:58.151229 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:58.151177    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5e719e21-4e76-4c1c-89aa-032e80db0638","Type":"ContainerStarted","Data":"233d9244589fc27dd0637211d8175f0a13f29346a2cd5780dda50df5226ced98"}
Sep 16 18:46:58.151229 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:58.151219    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5e719e21-4e76-4c1c-89aa-032e80db0638","Type":"ContainerStarted","Data":"9c00c0326f3f73bf4e7c606fd5f1580c7d9a0411b06ac439943c53363b6629cd"}
Sep 16 18:46:58.152583 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:58.152557    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-nm7rr" event={"ID":"a27c3da5-d71f-47bd-b483-f5f486da3a63","Type":"ContainerStarted","Data":"93eba4d8a608b9d55b453da041ce1e93613f6d891a544b6c8b55ec64b68a6804"}
Sep 16 18:46:58.154130 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:58.154093    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4wnsv" event={"ID":"c19f514e-c3a8-41d2-9de1-ad2d86fce986","Type":"ContainerStarted","Data":"4dab1fb0303f4905dfe683b603343ef11de4cea70b8fe58d6d62f75860ebb7a8"}
Sep 16 18:46:59.162548 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:59.162503    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5e719e21-4e76-4c1c-89aa-032e80db0638","Type":"ContainerStarted","Data":"f137b0632120d3efbadc4a6ef3583e4549257129b8b7294f066e9a5d01b43584"}
Sep 16 18:46:59.164537 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:59.164508    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-nm7rr" event={"ID":"a27c3da5-d71f-47bd-b483-f5f486da3a63","Type":"ContainerStarted","Data":"1766e73b8ce079f9cd6cb02f932f95d12980603a05118bf38cb20760a9c92d97"}
Sep 16 18:46:59.164929 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:59.164742    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-nm7rr"
Sep 16 18:46:59.166740 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:59.166704    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4wnsv" event={"ID":"c19f514e-c3a8-41d2-9de1-ad2d86fce986","Type":"ContainerStarted","Data":"dfff56a4235093a8d8c2b291c82526d04879b8e630db195bf370aa3775194426"}
Sep 16 18:46:59.166863 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:59.166754    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4wnsv" event={"ID":"c19f514e-c3a8-41d2-9de1-ad2d86fce986","Type":"ContainerStarted","Data":"abc488c03d06b65fc53d5203a8addf9a44caf2fb860f98de53b6465790a9db51"}
Sep 16 18:46:59.171278 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:59.171253    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-nm7rr"
Sep 16 18:46:59.201729 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:59.201681    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.961254604 podStartE2EDuration="8.201652131s" totalImagesPullingTime="4.224369329s" totalInitContainerRuntime="16.028198ms" isStatefulPod=false podCreationTimestamp="2026-09-16 18:46:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:46:59.19880763 +0000 UTC m=+133.275912494" watchObservedRunningTime="2026-09-16 18:46:59.201652131 +0000 UTC m=+133.278756991"
Sep 16 18:46:59.230313 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:59.230006    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4wnsv" podStartSLOduration=131.310288828 podStartE2EDuration="2m12.229979663s" totalImagesPullingTime="919.690835ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:44:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:46:59.229478788 +0000 UTC m=+133.306583649" watchObservedRunningTime="2026-09-16 18:46:59.229979663 +0000 UTC m=+133.307084524"
Sep 16 18:46:59.256904 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:46:59.256845    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-nm7rr" podStartSLOduration=2.764987645 podStartE2EDuration="4.256825908s" totalImagesPullingTime="1.491838263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:46:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:46:59.255422852 +0000 UTC m=+133.332527714" watchObservedRunningTime="2026-09-16 18:46:59.256825908 +0000 UTC m=+133.333930808"
Sep 16 18:47:02.183566 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:02.183525    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4639fcfa-4d2b-4f72-be92-1c9a3954c101","Type":"ContainerStarted","Data":"050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87"}
Sep 16 18:47:02.183953 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:02.183607    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4639fcfa-4d2b-4f72-be92-1c9a3954c101","Type":"ContainerStarted","Data":"c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526"}
Sep 16 18:47:03.190727 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:03.190693    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4639fcfa-4d2b-4f72-be92-1c9a3954c101","Type":"ContainerStarted","Data":"e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729"}
Sep 16 18:47:04.198735 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:04.198701    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4639fcfa-4d2b-4f72-be92-1c9a3954c101","Type":"ContainerStarted","Data":"d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb"}
Sep 16 18:47:04.199126 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:04.198762    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4639fcfa-4d2b-4f72-be92-1c9a3954c101","Type":"ContainerStarted","Data":"3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185"}
Sep 16 18:47:04.199126 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:04.198804    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4639fcfa-4d2b-4f72-be92-1c9a3954c101","Type":"ContainerStarted","Data":"d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe"}
Sep 16 18:47:04.233371 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:04.233308    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.45108977 podStartE2EDuration="8.233291115s" totalImagesPullingTime="4.758694221s" totalInitContainerRuntime="23.507124ms" isStatefulPod=false podCreationTimestamp="2026-09-16 18:46:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:47:04.230282797 +0000 UTC m=+138.307387657" watchObservedRunningTime="2026-09-16 18:47:04.233291115 +0000 UTC m=+138.310395976"
Sep 16 18:47:07.124395 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:07.124359    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:47:08.390413 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:08.390359    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-c75d9488-qzft7" podUID="54674a4a-c1b7-429a-a4a9-21f7f93d22ab" containerName="registry" containerID="cri-o://b442d4edcc61fbf39d851e3717cc37899950bb023de6a43c35193478923c02b0" gracePeriod=30
Sep 16 18:47:08.634342 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:08.634319    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:47:08.724762 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:08.724684    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-image-registry-private-configuration\") pod \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") "
Sep 16 18:47:08.724762 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:08.724716    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7x574\" (UniqueName: \"kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-kube-api-access-7x574\") pod \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") "
Sep 16 18:47:08.724762 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:08.724753    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-installation-pull-secrets\") pod \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") "
Sep 16 18:47:08.725046 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:08.724908    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-bound-sa-token\") pod \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") "
Sep 16 18:47:08.725046 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:08.725007    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-ca-trust-extracted\") pod \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") "
Sep 16 18:47:08.725046 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:08.725040    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-registry-certificates\") pod \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") "
Sep 16 18:47:08.725201 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:08.725081    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-trusted-ca\") pod \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") "
Sep 16 18:47:08.725201 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:08.725153    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-registry-tls\") pod \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\" (UID: \"54674a4a-c1b7-429a-a4a9-21f7f93d22ab\") "
Sep 16 18:47:08.725659 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:08.725627    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "54674a4a-c1b7-429a-a4a9-21f7f93d22ab" (UID: "54674a4a-c1b7-429a-a4a9-21f7f93d22ab"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:47:08.726050 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:08.726023    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "54674a4a-c1b7-429a-a4a9-21f7f93d22ab" (UID: "54674a4a-c1b7-429a-a4a9-21f7f93d22ab"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:47:08.727194 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:08.727169    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "54674a4a-c1b7-429a-a4a9-21f7f93d22ab" (UID: "54674a4a-c1b7-429a-a4a9-21f7f93d22ab"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:47:08.727353 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:08.727332    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "54674a4a-c1b7-429a-a4a9-21f7f93d22ab" (UID: "54674a4a-c1b7-429a-a4a9-21f7f93d22ab"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:47:08.727428 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:08.727348    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "54674a4a-c1b7-429a-a4a9-21f7f93d22ab" (UID: "54674a4a-c1b7-429a-a4a9-21f7f93d22ab"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:47:08.727428 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:08.727356    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-kube-api-access-7x574" (OuterVolumeSpecName: "kube-api-access-7x574") pod "54674a4a-c1b7-429a-a4a9-21f7f93d22ab" (UID: "54674a4a-c1b7-429a-a4a9-21f7f93d22ab"). InnerVolumeSpecName "kube-api-access-7x574". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:47:08.727428 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:08.727372    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "54674a4a-c1b7-429a-a4a9-21f7f93d22ab" (UID: "54674a4a-c1b7-429a-a4a9-21f7f93d22ab"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:47:08.733233 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:08.733208    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "54674a4a-c1b7-429a-a4a9-21f7f93d22ab" (UID: "54674a4a-c1b7-429a-a4a9-21f7f93d22ab"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:47:08.825964 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:08.825910    2557 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-bound-sa-token\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:47:08.825964 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:08.825957    2557 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-ca-trust-extracted\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:47:08.825964 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:08.825975    2557 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-registry-certificates\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:47:08.826258 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:08.825988    2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-trusted-ca\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:47:08.826258 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:08.826000    2557 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-registry-tls\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:47:08.826258 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:08.826013    2557 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-image-registry-private-configuration\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:47:08.826258 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:08.826025    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7x574\" (UniqueName: \"kubernetes.io/projected/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-kube-api-access-7x574\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:47:08.826258 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:08.826036    2557 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/54674a4a-c1b7-429a-a4a9-21f7f93d22ab-installation-pull-secrets\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:47:09.219978 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:09.219937    2557 generic.go:356] "Generic (PLEG): container finished" podID="54674a4a-c1b7-429a-a4a9-21f7f93d22ab" containerID="b442d4edcc61fbf39d851e3717cc37899950bb023de6a43c35193478923c02b0" exitCode=0
Sep 16 18:47:09.220178 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:09.219987    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c75d9488-qzft7"
Sep 16 18:47:09.220178 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:09.219997    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c75d9488-qzft7" event={"ID":"54674a4a-c1b7-429a-a4a9-21f7f93d22ab","Type":"ContainerDied","Data":"b442d4edcc61fbf39d851e3717cc37899950bb023de6a43c35193478923c02b0"}
Sep 16 18:47:09.220178 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:09.220037    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c75d9488-qzft7" event={"ID":"54674a4a-c1b7-429a-a4a9-21f7f93d22ab","Type":"ContainerDied","Data":"380f7162f71dee0b2a6fce1416a1cdbbb5c16527b36ed5a528ccbb01d3132944"}
Sep 16 18:47:09.220178 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:09.220057    2557 scope.go:117] "RemoveContainer" containerID="b442d4edcc61fbf39d851e3717cc37899950bb023de6a43c35193478923c02b0"
Sep 16 18:47:09.229302 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:09.229277    2557 scope.go:117] "RemoveContainer" containerID="b442d4edcc61fbf39d851e3717cc37899950bb023de6a43c35193478923c02b0"
Sep 16 18:47:09.229596 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:47:09.229574    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b442d4edcc61fbf39d851e3717cc37899950bb023de6a43c35193478923c02b0\": container with ID starting with b442d4edcc61fbf39d851e3717cc37899950bb023de6a43c35193478923c02b0 not found: ID does not exist" containerID="b442d4edcc61fbf39d851e3717cc37899950bb023de6a43c35193478923c02b0"
Sep 16 18:47:09.229645 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:09.229605    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b442d4edcc61fbf39d851e3717cc37899950bb023de6a43c35193478923c02b0"} err="failed to get container status \"b442d4edcc61fbf39d851e3717cc37899950bb023de6a43c35193478923c02b0\": rpc error: code = NotFound desc = could not find container \"b442d4edcc61fbf39d851e3717cc37899950bb023de6a43c35193478923c02b0\": container with ID starting with b442d4edcc61fbf39d851e3717cc37899950bb023de6a43c35193478923c02b0 not found: ID does not exist"
Sep 16 18:47:09.249896 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:09.249865    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-c75d9488-qzft7"]
Sep 16 18:47:09.251995 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:09.251971    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-c75d9488-qzft7"]
Sep 16 18:47:10.465892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:10.465857    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54674a4a-c1b7-429a-a4a9-21f7f93d22ab" path="/var/lib/kubelet/pods/54674a4a-c1b7-429a-a4a9-21f7f93d22ab/volumes"
Sep 16 18:47:15.306876 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:15.306842    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"
Sep 16 18:47:15.307340 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:15.306912    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"
Sep 16 18:47:30.304265 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:30.304173    2557 generic.go:356] "Generic (PLEG): container finished" podID="c3c4753f-d91a-4409-b89e-be97ec7b523e" containerID="4c7aeac3bebf3b4f20af1f6a84afde2930236ce102c7618e82a475c231042585" exitCode=0
Sep 16 18:47:30.304265 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:30.304214    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7jxmm" event={"ID":"c3c4753f-d91a-4409-b89e-be97ec7b523e","Type":"ContainerDied","Data":"4c7aeac3bebf3b4f20af1f6a84afde2930236ce102c7618e82a475c231042585"}
Sep 16 18:47:30.304748 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:30.304555    2557 scope.go:117] "RemoveContainer" containerID="4c7aeac3bebf3b4f20af1f6a84afde2930236ce102c7618e82a475c231042585"
Sep 16 18:47:31.309782 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:31.309739    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-7jxmm" event={"ID":"c3c4753f-d91a-4409-b89e-be97ec7b523e","Type":"ContainerStarted","Data":"3b0c6c3126034de142413fa48c841126a516fff47a72abbab7f5fa9a44e23178"}
Sep 16 18:47:35.312517 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:35.312483    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"
Sep 16 18:47:35.316317 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:35.316288    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-68b5bf68d8-vwbqr"
Sep 16 18:47:36.330334 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:36.330301    2557 generic.go:356] "Generic (PLEG): container finished" podID="fb6525d5-7015-41e7-a307-c282e1f6bd79" containerID="e470340fbf6d75192ca346a84df7d604d6d2cf98666b55a3423aaebb0cdf12f0" exitCode=0
Sep 16 18:47:36.330720 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:36.330340    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-ftvj4" event={"ID":"fb6525d5-7015-41e7-a307-c282e1f6bd79","Type":"ContainerDied","Data":"e470340fbf6d75192ca346a84df7d604d6d2cf98666b55a3423aaebb0cdf12f0"}
Sep 16 18:47:36.330789 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:36.330719    2557 scope.go:117] "RemoveContainer" containerID="e470340fbf6d75192ca346a84df7d604d6d2cf98666b55a3423aaebb0cdf12f0"
Sep 16 18:47:37.337019 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:37.336981    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-ftvj4" event={"ID":"fb6525d5-7015-41e7-a307-c282e1f6bd79","Type":"ContainerStarted","Data":"eb7126086637bc55453f456e773c0c4d1a92848b848c78b5193d844162b70efa"}
Sep 16 18:47:57.124567 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:57.124527    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:47:57.140150 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:57.140125    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:47:57.428861 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:47:57.428781    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:11.173859 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:11.173824    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 18:48:11.174347 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:11.174105    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerName="alertmanager" containerID="cri-o://fb09cad336313c3cf657cc1a5d1c6bfdfd9efec3b5933442611924287e2151be" gracePeriod=120
Sep 16 18:48:11.174347 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:11.174172    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerName="kube-rbac-proxy-metric" containerID="cri-o://f3b641a5b055a849a2438acca64835b4b6948479b9edab8222e03396ad5d52ca" gracePeriod=120
Sep 16 18:48:11.174347 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:11.174216    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerName="kube-rbac-proxy-web" containerID="cri-o://9c00c0326f3f73bf4e7c606fd5f1580c7d9a0411b06ac439943c53363b6629cd" gracePeriod=120
Sep 16 18:48:11.174347 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:11.174275    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerName="kube-rbac-proxy" containerID="cri-o://233d9244589fc27dd0637211d8175f0a13f29346a2cd5780dda50df5226ced98" gracePeriod=120
Sep 16 18:48:11.174347 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:11.174281    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerName="prom-label-proxy" containerID="cri-o://f137b0632120d3efbadc4a6ef3583e4549257129b8b7294f066e9a5d01b43584" gracePeriod=120
Sep 16 18:48:11.174347 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:11.174229    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerName="config-reloader" containerID="cri-o://948f22f715318b554c954317592f1376e63b55fdb29f1afb46b422264a390b48" gracePeriod=120
Sep 16 18:48:11.471288 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:11.471203    2557 generic.go:356] "Generic (PLEG): container finished" podID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerID="f137b0632120d3efbadc4a6ef3583e4549257129b8b7294f066e9a5d01b43584" exitCode=0
Sep 16 18:48:11.471288 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:11.471227    2557 generic.go:356] "Generic (PLEG): container finished" podID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerID="233d9244589fc27dd0637211d8175f0a13f29346a2cd5780dda50df5226ced98" exitCode=0
Sep 16 18:48:11.471288 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:11.471234    2557 generic.go:356] "Generic (PLEG): container finished" podID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerID="9c00c0326f3f73bf4e7c606fd5f1580c7d9a0411b06ac439943c53363b6629cd" exitCode=0
Sep 16 18:48:11.471288 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:11.471240    2557 generic.go:356] "Generic (PLEG): container finished" podID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerID="948f22f715318b554c954317592f1376e63b55fdb29f1afb46b422264a390b48" exitCode=0
Sep 16 18:48:11.471288 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:11.471245    2557 generic.go:356] "Generic (PLEG): container finished" podID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerID="fb09cad336313c3cf657cc1a5d1c6bfdfd9efec3b5933442611924287e2151be" exitCode=0
Sep 16 18:48:11.471536 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:11.471283    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5e719e21-4e76-4c1c-89aa-032e80db0638","Type":"ContainerDied","Data":"f137b0632120d3efbadc4a6ef3583e4549257129b8b7294f066e9a5d01b43584"}
Sep 16 18:48:11.471536 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:11.471353    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5e719e21-4e76-4c1c-89aa-032e80db0638","Type":"ContainerDied","Data":"233d9244589fc27dd0637211d8175f0a13f29346a2cd5780dda50df5226ced98"}
Sep 16 18:48:11.471536 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:11.471381    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5e719e21-4e76-4c1c-89aa-032e80db0638","Type":"ContainerDied","Data":"9c00c0326f3f73bf4e7c606fd5f1580c7d9a0411b06ac439943c53363b6629cd"}
Sep 16 18:48:11.471536 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:11.471414    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5e719e21-4e76-4c1c-89aa-032e80db0638","Type":"ContainerDied","Data":"948f22f715318b554c954317592f1376e63b55fdb29f1afb46b422264a390b48"}
Sep 16 18:48:11.471536 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:11.471440    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5e719e21-4e76-4c1c-89aa-032e80db0638","Type":"ContainerDied","Data":"fb09cad336313c3cf657cc1a5d1c6bfdfd9efec3b5933442611924287e2151be"}
Sep 16 18:48:12.425756 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.425734    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:48:12.479976 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.479943    2557 generic.go:356] "Generic (PLEG): container finished" podID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerID="f3b641a5b055a849a2438acca64835b4b6948479b9edab8222e03396ad5d52ca" exitCode=0
Sep 16 18:48:12.480129 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.480015    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:48:12.480129 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.480026    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5e719e21-4e76-4c1c-89aa-032e80db0638","Type":"ContainerDied","Data":"f3b641a5b055a849a2438acca64835b4b6948479b9edab8222e03396ad5d52ca"}
Sep 16 18:48:12.480129 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.480095    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5e719e21-4e76-4c1c-89aa-032e80db0638","Type":"ContainerDied","Data":"c8153dbb073e715186d4814c0f9a71ba6398943c5ae7424df85cc8696b06f735"}
Sep 16 18:48:12.480251 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.480135    2557 scope.go:117] "RemoveContainer" containerID="f137b0632120d3efbadc4a6ef3583e4549257129b8b7294f066e9a5d01b43584"
Sep 16 18:48:12.489332 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.489313    2557 scope.go:117] "RemoveContainer" containerID="f3b641a5b055a849a2438acca64835b4b6948479b9edab8222e03396ad5d52ca"
Sep 16 18:48:12.498234 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.498216    2557 scope.go:117] "RemoveContainer" containerID="233d9244589fc27dd0637211d8175f0a13f29346a2cd5780dda50df5226ced98"
Sep 16 18:48:12.507869 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.507846    2557 scope.go:117] "RemoveContainer" containerID="9c00c0326f3f73bf4e7c606fd5f1580c7d9a0411b06ac439943c53363b6629cd"
Sep 16 18:48:12.514818 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.514798    2557 scope.go:117] "RemoveContainer" containerID="948f22f715318b554c954317592f1376e63b55fdb29f1afb46b422264a390b48"
Sep 16 18:48:12.522451 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.522435    2557 scope.go:117] "RemoveContainer" containerID="fb09cad336313c3cf657cc1a5d1c6bfdfd9efec3b5933442611924287e2151be"
Sep 16 18:48:12.529420 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.529405    2557 scope.go:117] "RemoveContainer" containerID="775d108fc9f1143269474e5905706317b1d2be6c46273f5efff9324d7761c099"
Sep 16 18:48:12.535854 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.535838    2557 scope.go:117] "RemoveContainer" containerID="f137b0632120d3efbadc4a6ef3583e4549257129b8b7294f066e9a5d01b43584"
Sep 16 18:48:12.536129 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:48:12.536108    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f137b0632120d3efbadc4a6ef3583e4549257129b8b7294f066e9a5d01b43584\": container with ID starting with f137b0632120d3efbadc4a6ef3583e4549257129b8b7294f066e9a5d01b43584 not found: ID does not exist" containerID="f137b0632120d3efbadc4a6ef3583e4549257129b8b7294f066e9a5d01b43584"
Sep 16 18:48:12.536177 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.536137    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f137b0632120d3efbadc4a6ef3583e4549257129b8b7294f066e9a5d01b43584"} err="failed to get container status \"f137b0632120d3efbadc4a6ef3583e4549257129b8b7294f066e9a5d01b43584\": rpc error: code = NotFound desc = could not find container \"f137b0632120d3efbadc4a6ef3583e4549257129b8b7294f066e9a5d01b43584\": container with ID starting with f137b0632120d3efbadc4a6ef3583e4549257129b8b7294f066e9a5d01b43584 not found: ID does not exist"
Sep 16 18:48:12.536177 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.536155    2557 scope.go:117] "RemoveContainer" containerID="f3b641a5b055a849a2438acca64835b4b6948479b9edab8222e03396ad5d52ca"
Sep 16 18:48:12.536392 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:48:12.536373    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3b641a5b055a849a2438acca64835b4b6948479b9edab8222e03396ad5d52ca\": container with ID starting with f3b641a5b055a849a2438acca64835b4b6948479b9edab8222e03396ad5d52ca not found: ID does not exist" containerID="f3b641a5b055a849a2438acca64835b4b6948479b9edab8222e03396ad5d52ca"
Sep 16 18:48:12.536434 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.536399    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3b641a5b055a849a2438acca64835b4b6948479b9edab8222e03396ad5d52ca"} err="failed to get container status \"f3b641a5b055a849a2438acca64835b4b6948479b9edab8222e03396ad5d52ca\": rpc error: code = NotFound desc = could not find container \"f3b641a5b055a849a2438acca64835b4b6948479b9edab8222e03396ad5d52ca\": container with ID starting with f3b641a5b055a849a2438acca64835b4b6948479b9edab8222e03396ad5d52ca not found: ID does not exist"
Sep 16 18:48:12.536434 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.536414    2557 scope.go:117] "RemoveContainer" containerID="233d9244589fc27dd0637211d8175f0a13f29346a2cd5780dda50df5226ced98"
Sep 16 18:48:12.536659 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:48:12.536642    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"233d9244589fc27dd0637211d8175f0a13f29346a2cd5780dda50df5226ced98\": container with ID starting with 233d9244589fc27dd0637211d8175f0a13f29346a2cd5780dda50df5226ced98 not found: ID does not exist" containerID="233d9244589fc27dd0637211d8175f0a13f29346a2cd5780dda50df5226ced98"
Sep 16 18:48:12.536725 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.536665    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"233d9244589fc27dd0637211d8175f0a13f29346a2cd5780dda50df5226ced98"} err="failed to get container status \"233d9244589fc27dd0637211d8175f0a13f29346a2cd5780dda50df5226ced98\": rpc error: code = NotFound desc = could not find container \"233d9244589fc27dd0637211d8175f0a13f29346a2cd5780dda50df5226ced98\": container with ID starting with 233d9244589fc27dd0637211d8175f0a13f29346a2cd5780dda50df5226ced98 not found: ID does not exist"
Sep 16 18:48:12.536725 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.536685    2557 scope.go:117] "RemoveContainer" containerID="9c00c0326f3f73bf4e7c606fd5f1580c7d9a0411b06ac439943c53363b6629cd"
Sep 16 18:48:12.536926 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:48:12.536911    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c00c0326f3f73bf4e7c606fd5f1580c7d9a0411b06ac439943c53363b6629cd\": container with ID starting with 9c00c0326f3f73bf4e7c606fd5f1580c7d9a0411b06ac439943c53363b6629cd not found: ID does not exist" containerID="9c00c0326f3f73bf4e7c606fd5f1580c7d9a0411b06ac439943c53363b6629cd"
Sep 16 18:48:12.536978 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.536929    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c00c0326f3f73bf4e7c606fd5f1580c7d9a0411b06ac439943c53363b6629cd"} err="failed to get container status \"9c00c0326f3f73bf4e7c606fd5f1580c7d9a0411b06ac439943c53363b6629cd\": rpc error: code = NotFound desc = could not find container \"9c00c0326f3f73bf4e7c606fd5f1580c7d9a0411b06ac439943c53363b6629cd\": container with ID starting with 9c00c0326f3f73bf4e7c606fd5f1580c7d9a0411b06ac439943c53363b6629cd not found: ID does not exist"
Sep 16 18:48:12.536978 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.536942    2557 scope.go:117] "RemoveContainer" containerID="948f22f715318b554c954317592f1376e63b55fdb29f1afb46b422264a390b48"
Sep 16 18:48:12.537159 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:48:12.537141    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"948f22f715318b554c954317592f1376e63b55fdb29f1afb46b422264a390b48\": container with ID starting with 948f22f715318b554c954317592f1376e63b55fdb29f1afb46b422264a390b48 not found: ID does not exist" containerID="948f22f715318b554c954317592f1376e63b55fdb29f1afb46b422264a390b48"
Sep 16 18:48:12.537265 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.537162    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"948f22f715318b554c954317592f1376e63b55fdb29f1afb46b422264a390b48"} err="failed to get container status \"948f22f715318b554c954317592f1376e63b55fdb29f1afb46b422264a390b48\": rpc error: code = NotFound desc = could not find container \"948f22f715318b554c954317592f1376e63b55fdb29f1afb46b422264a390b48\": container with ID starting with 948f22f715318b554c954317592f1376e63b55fdb29f1afb46b422264a390b48 not found: ID does not exist"
Sep 16 18:48:12.537265 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.537181    2557 scope.go:117] "RemoveContainer" containerID="fb09cad336313c3cf657cc1a5d1c6bfdfd9efec3b5933442611924287e2151be"
Sep 16 18:48:12.537387 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:48:12.537357    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb09cad336313c3cf657cc1a5d1c6bfdfd9efec3b5933442611924287e2151be\": container with ID starting with fb09cad336313c3cf657cc1a5d1c6bfdfd9efec3b5933442611924287e2151be not found: ID does not exist" containerID="fb09cad336313c3cf657cc1a5d1c6bfdfd9efec3b5933442611924287e2151be"
Sep 16 18:48:12.537442 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.537380    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb09cad336313c3cf657cc1a5d1c6bfdfd9efec3b5933442611924287e2151be"} err="failed to get container status \"fb09cad336313c3cf657cc1a5d1c6bfdfd9efec3b5933442611924287e2151be\": rpc error: code = NotFound desc = could not find container \"fb09cad336313c3cf657cc1a5d1c6bfdfd9efec3b5933442611924287e2151be\": container with ID starting with fb09cad336313c3cf657cc1a5d1c6bfdfd9efec3b5933442611924287e2151be not found: ID does not exist"
Sep 16 18:48:12.537442 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.537397    2557 scope.go:117] "RemoveContainer" containerID="775d108fc9f1143269474e5905706317b1d2be6c46273f5efff9324d7761c099"
Sep 16 18:48:12.537614 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:48:12.537600    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"775d108fc9f1143269474e5905706317b1d2be6c46273f5efff9324d7761c099\": container with ID starting with 775d108fc9f1143269474e5905706317b1d2be6c46273f5efff9324d7761c099 not found: ID does not exist" containerID="775d108fc9f1143269474e5905706317b1d2be6c46273f5efff9324d7761c099"
Sep 16 18:48:12.537657 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.537618    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"775d108fc9f1143269474e5905706317b1d2be6c46273f5efff9324d7761c099"} err="failed to get container status \"775d108fc9f1143269474e5905706317b1d2be6c46273f5efff9324d7761c099\": rpc error: code = NotFound desc = could not find container \"775d108fc9f1143269474e5905706317b1d2be6c46273f5efff9324d7761c099\": container with ID starting with 775d108fc9f1143269474e5905706317b1d2be6c46273f5efff9324d7761c099 not found: ID does not exist"
Sep 16 18:48:12.569935 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.569911    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-web-config\") pod \"5e719e21-4e76-4c1c-89aa-032e80db0638\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") "
Sep 16 18:48:12.570041 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.569944    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5e719e21-4e76-4c1c-89aa-032e80db0638-tls-assets\") pod \"5e719e21-4e76-4c1c-89aa-032e80db0638\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") "
Sep 16 18:48:12.570041 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.569972    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-secret-alertmanager-main-tls\") pod \"5e719e21-4e76-4c1c-89aa-032e80db0638\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") "
Sep 16 18:48:12.570041 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.570004    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-secret-alertmanager-kube-rbac-proxy\") pod \"5e719e21-4e76-4c1c-89aa-032e80db0638\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") "
Sep 16 18:48:12.570041 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.570030    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5e719e21-4e76-4c1c-89aa-032e80db0638-alertmanager-main-db\") pod \"5e719e21-4e76-4c1c-89aa-032e80db0638\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") "
Sep 16 18:48:12.570355 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.570064    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e719e21-4e76-4c1c-89aa-032e80db0638-metrics-client-ca\") pod \"5e719e21-4e76-4c1c-89aa-032e80db0638\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") "
Sep 16 18:48:12.570355 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.570090    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-config-volume\") pod \"5e719e21-4e76-4c1c-89aa-032e80db0638\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") "
Sep 16 18:48:12.570355 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.570143    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-cluster-tls-config\") pod \"5e719e21-4e76-4c1c-89aa-032e80db0638\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") "
Sep 16 18:48:12.570508 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.570423    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e719e21-4e76-4c1c-89aa-032e80db0638-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "5e719e21-4e76-4c1c-89aa-032e80db0638" (UID: "5e719e21-4e76-4c1c-89aa-032e80db0638"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:48:12.570564 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.570517    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-secret-alertmanager-kube-rbac-proxy-metric\") pod \"5e719e21-4e76-4c1c-89aa-032e80db0638\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") "
Sep 16 18:48:12.570564 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.570532    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e719e21-4e76-4c1c-89aa-032e80db0638-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "5e719e21-4e76-4c1c-89aa-032e80db0638" (UID: "5e719e21-4e76-4c1c-89aa-032e80db0638"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:48:12.570668 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.570579    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-secret-alertmanager-kube-rbac-proxy-web\") pod \"5e719e21-4e76-4c1c-89aa-032e80db0638\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") "
Sep 16 18:48:12.570668 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.570616    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5e719e21-4e76-4c1c-89aa-032e80db0638-config-out\") pod \"5e719e21-4e76-4c1c-89aa-032e80db0638\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") "
Sep 16 18:48:12.570668 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.570658    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4dhgb\" (UniqueName: \"kubernetes.io/projected/5e719e21-4e76-4c1c-89aa-032e80db0638-kube-api-access-4dhgb\") pod \"5e719e21-4e76-4c1c-89aa-032e80db0638\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") "
Sep 16 18:48:12.570849 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.570686    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e719e21-4e76-4c1c-89aa-032e80db0638-alertmanager-trusted-ca-bundle\") pod \"5e719e21-4e76-4c1c-89aa-032e80db0638\" (UID: \"5e719e21-4e76-4c1c-89aa-032e80db0638\") "
Sep 16 18:48:12.570970 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.570952    2557 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5e719e21-4e76-4c1c-89aa-032e80db0638-alertmanager-main-db\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:12.571043 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.570975    2557 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e719e21-4e76-4c1c-89aa-032e80db0638-metrics-client-ca\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:12.571389 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.571364    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e719e21-4e76-4c1c-89aa-032e80db0638-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "5e719e21-4e76-4c1c-89aa-032e80db0638" (UID: "5e719e21-4e76-4c1c-89aa-032e80db0638"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:48:12.573266 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.573200    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e719e21-4e76-4c1c-89aa-032e80db0638-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "5e719e21-4e76-4c1c-89aa-032e80db0638" (UID: "5e719e21-4e76-4c1c-89aa-032e80db0638"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:48:12.573266 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.573247    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "5e719e21-4e76-4c1c-89aa-032e80db0638" (UID: "5e719e21-4e76-4c1c-89aa-032e80db0638"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:48:12.573422 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.573350    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "5e719e21-4e76-4c1c-89aa-032e80db0638" (UID: "5e719e21-4e76-4c1c-89aa-032e80db0638"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:48:12.574093 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.574060    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-config-volume" (OuterVolumeSpecName: "config-volume") pod "5e719e21-4e76-4c1c-89aa-032e80db0638" (UID: "5e719e21-4e76-4c1c-89aa-032e80db0638"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:48:12.574318 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.574236    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e719e21-4e76-4c1c-89aa-032e80db0638-kube-api-access-4dhgb" (OuterVolumeSpecName: "kube-api-access-4dhgb") pod "5e719e21-4e76-4c1c-89aa-032e80db0638" (UID: "5e719e21-4e76-4c1c-89aa-032e80db0638"). InnerVolumeSpecName "kube-api-access-4dhgb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:48:12.574388 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.574360    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "5e719e21-4e76-4c1c-89aa-032e80db0638" (UID: "5e719e21-4e76-4c1c-89aa-032e80db0638"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:48:12.574517 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.574495    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "5e719e21-4e76-4c1c-89aa-032e80db0638" (UID: "5e719e21-4e76-4c1c-89aa-032e80db0638"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:48:12.574605 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.574582    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e719e21-4e76-4c1c-89aa-032e80db0638-config-out" (OuterVolumeSpecName: "config-out") pod "5e719e21-4e76-4c1c-89aa-032e80db0638" (UID: "5e719e21-4e76-4c1c-89aa-032e80db0638"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:48:12.578273 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.578231    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "5e719e21-4e76-4c1c-89aa-032e80db0638" (UID: "5e719e21-4e76-4c1c-89aa-032e80db0638"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:48:12.583817 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.583798    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-web-config" (OuterVolumeSpecName: "web-config") pod "5e719e21-4e76-4c1c-89aa-032e80db0638" (UID: "5e719e21-4e76-4c1c-89aa-032e80db0638"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:48:12.671908 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.671869    2557 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:12.671908 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.671901    2557 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5e719e21-4e76-4c1c-89aa-032e80db0638-config-out\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:12.672111 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.671916    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dhgb\" (UniqueName: \"kubernetes.io/projected/5e719e21-4e76-4c1c-89aa-032e80db0638-kube-api-access-4dhgb\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:12.672111 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.671929    2557 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e719e21-4e76-4c1c-89aa-032e80db0638-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:12.672111 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.671941    2557 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-web-config\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:12.672111 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.671952    2557 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5e719e21-4e76-4c1c-89aa-032e80db0638-tls-assets\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:12.672111 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.671963    2557 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-secret-alertmanager-main-tls\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:12.672111 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.671974    2557 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:12.672111 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.671985    2557 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-config-volume\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:12.672111 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.671997    2557 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-cluster-tls-config\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:12.672111 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.672008    2557 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5e719e21-4e76-4c1c-89aa-032e80db0638-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:12.822515 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.822479    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 18:48:12.828399 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:12.828363    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 18:48:14.465695 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:14.465654    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e719e21-4e76-4c1c-89aa-032e80db0638" path="/var/lib/kubelet/pods/5e719e21-4e76-4c1c-89aa-032e80db0638/volumes"
Sep 16 18:48:15.495992 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.495960    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 18:48:15.496406 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.496233    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerName="prometheus" containerID="cri-o://c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526" gracePeriod=600
Sep 16 18:48:15.496406 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.496261    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerName="kube-rbac-proxy" containerID="cri-o://3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185" gracePeriod=600
Sep 16 18:48:15.496406 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.496305    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerName="config-reloader" containerID="cri-o://050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87" gracePeriod=600
Sep 16 18:48:15.496406 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.496308    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerName="kube-rbac-proxy-thanos" containerID="cri-o://d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb" gracePeriod=600
Sep 16 18:48:15.496406 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.496332    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerName="thanos-sidecar" containerID="cri-o://e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729" gracePeriod=600
Sep 16 18:48:15.496662 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.496332    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerName="kube-rbac-proxy-web" containerID="cri-o://d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe" gracePeriod=600
Sep 16 18:48:15.743977 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.743947    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:15.897102 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.897060    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8dmc\" (UniqueName: \"kubernetes.io/projected/4639fcfa-4d2b-4f72-be92-1c9a3954c101-kube-api-access-r8dmc\") pod \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") "
Sep 16 18:48:15.897102 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.897101    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") "
Sep 16 18:48:15.897349 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.897119    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-web-config\") pod \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") "
Sep 16 18:48:15.897349 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.897143    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-config\") pod \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") "
Sep 16 18:48:15.897349 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.897173    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-configmap-metrics-client-ca\") pod \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") "
Sep 16 18:48:15.897349 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.897207    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-k8s-rulefiles-0\") pod \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") "
Sep 16 18:48:15.897349 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.897235    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-kube-rbac-proxy\") pod \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") "
Sep 16 18:48:15.897349 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.897265    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4639fcfa-4d2b-4f72-be92-1c9a3954c101-config-out\") pod \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") "
Sep 16 18:48:15.897349 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.897292    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-trusted-ca-bundle\") pod \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") "
Sep 16 18:48:15.897349 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.897323    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-k8s-rulefiles-1\") pod \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") "
Sep 16 18:48:15.897742 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.897361    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4639fcfa-4d2b-4f72-be92-1c9a3954c101-tls-assets\") pod \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") "
Sep 16 18:48:15.897742 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.897397    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-k8s-db\") pod \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") "
Sep 16 18:48:15.897742 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.897425    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") "
Sep 16 18:48:15.897742 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.897452    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-configmap-kubelet-serving-ca-bundle\") pod \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") "
Sep 16 18:48:15.897742 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.897488    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-metrics-client-certs\") pod \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") "
Sep 16 18:48:15.897742 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.897517    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-k8s-rulefiles-2\") pod \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") "
Sep 16 18:48:15.897742 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.897553    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-grpc-tls\") pod \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") "
Sep 16 18:48:15.897742 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.897609    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-prometheus-k8s-tls\") pod \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") "
Sep 16 18:48:15.897742 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.897632    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-configmap-serving-certs-ca-bundle\") pod \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") "
Sep 16 18:48:15.897742 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.897671    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-thanos-prometheus-http-client-file\") pod \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\" (UID: \"4639fcfa-4d2b-4f72-be92-1c9a3954c101\") "
Sep 16 18:48:15.898264 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.897660    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "4639fcfa-4d2b-4f72-be92-1c9a3954c101" (UID: "4639fcfa-4d2b-4f72-be92-1c9a3954c101"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:48:15.898264 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.897751    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "4639fcfa-4d2b-4f72-be92-1c9a3954c101" (UID: "4639fcfa-4d2b-4f72-be92-1c9a3954c101"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:48:15.900187 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.898799    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "4639fcfa-4d2b-4f72-be92-1c9a3954c101" (UID: "4639fcfa-4d2b-4f72-be92-1c9a3954c101"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:48:15.900187 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.898974    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "4639fcfa-4d2b-4f72-be92-1c9a3954c101" (UID: "4639fcfa-4d2b-4f72-be92-1c9a3954c101"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:48:15.900187 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.899357    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "4639fcfa-4d2b-4f72-be92-1c9a3954c101" (UID: "4639fcfa-4d2b-4f72-be92-1c9a3954c101"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:48:15.900187 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.899465    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "4639fcfa-4d2b-4f72-be92-1c9a3954c101" (UID: "4639fcfa-4d2b-4f72-be92-1c9a3954c101"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:48:15.900187 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.899933    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4639fcfa-4d2b-4f72-be92-1c9a3954c101-kube-api-access-r8dmc" (OuterVolumeSpecName: "kube-api-access-r8dmc") pod "4639fcfa-4d2b-4f72-be92-1c9a3954c101" (UID: "4639fcfa-4d2b-4f72-be92-1c9a3954c101"). InnerVolumeSpecName "kube-api-access-r8dmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:48:15.900187 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.899974    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "4639fcfa-4d2b-4f72-be92-1c9a3954c101" (UID: "4639fcfa-4d2b-4f72-be92-1c9a3954c101"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:48:15.900187 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.900154    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "4639fcfa-4d2b-4f72-be92-1c9a3954c101" (UID: "4639fcfa-4d2b-4f72-be92-1c9a3954c101"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:48:15.901487 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.901457    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "4639fcfa-4d2b-4f72-be92-1c9a3954c101" (UID: "4639fcfa-4d2b-4f72-be92-1c9a3954c101"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:48:15.901597 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.901523    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4639fcfa-4d2b-4f72-be92-1c9a3954c101-config-out" (OuterVolumeSpecName: "config-out") pod "4639fcfa-4d2b-4f72-be92-1c9a3954c101" (UID: "4639fcfa-4d2b-4f72-be92-1c9a3954c101"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:48:15.901597 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.901549    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-config" (OuterVolumeSpecName: "config") pod "4639fcfa-4d2b-4f72-be92-1c9a3954c101" (UID: "4639fcfa-4d2b-4f72-be92-1c9a3954c101"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:48:15.901832 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.901802    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "4639fcfa-4d2b-4f72-be92-1c9a3954c101" (UID: "4639fcfa-4d2b-4f72-be92-1c9a3954c101"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:48:15.902107 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.902085    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "4639fcfa-4d2b-4f72-be92-1c9a3954c101" (UID: "4639fcfa-4d2b-4f72-be92-1c9a3954c101"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:48:15.902223 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.902198    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4639fcfa-4d2b-4f72-be92-1c9a3954c101-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "4639fcfa-4d2b-4f72-be92-1c9a3954c101" (UID: "4639fcfa-4d2b-4f72-be92-1c9a3954c101"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:48:15.902303 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.902279    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "4639fcfa-4d2b-4f72-be92-1c9a3954c101" (UID: "4639fcfa-4d2b-4f72-be92-1c9a3954c101"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:48:15.902346 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.902311    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "4639fcfa-4d2b-4f72-be92-1c9a3954c101" (UID: "4639fcfa-4d2b-4f72-be92-1c9a3954c101"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:48:15.902787 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.902750    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "4639fcfa-4d2b-4f72-be92-1c9a3954c101" (UID: "4639fcfa-4d2b-4f72-be92-1c9a3954c101"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:48:15.902957 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.902930    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "4639fcfa-4d2b-4f72-be92-1c9a3954c101" (UID: "4639fcfa-4d2b-4f72-be92-1c9a3954c101"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:48:15.910716 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.910694    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-web-config" (OuterVolumeSpecName: "web-config") pod "4639fcfa-4d2b-4f72-be92-1c9a3954c101" (UID: "4639fcfa-4d2b-4f72-be92-1c9a3954c101"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:48:15.998803 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.998748    2557 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4639fcfa-4d2b-4f72-be92-1c9a3954c101-config-out\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:15.998803 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.998795    2557 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-trusted-ca-bundle\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:15.998803 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.998808    2557 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:15.999021 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.998818    2557 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4639fcfa-4d2b-4f72-be92-1c9a3954c101-tls-assets\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:15.999021 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.998827    2557 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-k8s-db\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:15.999021 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.998836    2557 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:15.999021 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.998846    2557 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:15.999021 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.998855    2557 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-metrics-client-certs\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:15.999021 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.998863    2557 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:15.999021 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.998872    2557 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-grpc-tls\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:15.999021 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.998880    2557 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-prometheus-k8s-tls\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:15.999021 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.998887    2557 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:15.999021 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.998895    2557 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-thanos-prometheus-http-client-file\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:15.999021 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.998904    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8dmc\" (UniqueName: \"kubernetes.io/projected/4639fcfa-4d2b-4f72-be92-1c9a3954c101-kube-api-access-r8dmc\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:15.999021 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.998912    2557 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:15.999021 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.998921    2557 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-web-config\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:15.999021 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.998929    2557 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-config\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:15.999021 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.998936    2557 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-configmap-metrics-client-ca\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:15.999021 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.998943    2557 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4639fcfa-4d2b-4f72-be92-1c9a3954c101-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:15.999021 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:15.998951    2557 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4639fcfa-4d2b-4f72-be92-1c9a3954c101-secret-kube-rbac-proxy\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:48:16.500463 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.500435    2557 generic.go:356] "Generic (PLEG): container finished" podID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerID="d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb" exitCode=0
Sep 16 18:48:16.500463 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.500460    2557 generic.go:356] "Generic (PLEG): container finished" podID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerID="3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185" exitCode=0
Sep 16 18:48:16.500463 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.500467    2557 generic.go:356] "Generic (PLEG): container finished" podID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerID="d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe" exitCode=0
Sep 16 18:48:16.500890 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.500473    2557 generic.go:356] "Generic (PLEG): container finished" podID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerID="e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729" exitCode=0
Sep 16 18:48:16.500890 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.500478    2557 generic.go:356] "Generic (PLEG): container finished" podID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerID="050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87" exitCode=0
Sep 16 18:48:16.500890 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.500483    2557 generic.go:356] "Generic (PLEG): container finished" podID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerID="c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526" exitCode=0
Sep 16 18:48:16.500890 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.500510    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.500890 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.500523    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4639fcfa-4d2b-4f72-be92-1c9a3954c101","Type":"ContainerDied","Data":"d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb"}
Sep 16 18:48:16.500890 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.500589    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4639fcfa-4d2b-4f72-be92-1c9a3954c101","Type":"ContainerDied","Data":"3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185"}
Sep 16 18:48:16.500890 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.500616    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4639fcfa-4d2b-4f72-be92-1c9a3954c101","Type":"ContainerDied","Data":"d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe"}
Sep 16 18:48:16.500890 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.500642    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4639fcfa-4d2b-4f72-be92-1c9a3954c101","Type":"ContainerDied","Data":"e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729"}
Sep 16 18:48:16.500890 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.500663    2557 scope.go:117] "RemoveContainer" containerID="d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb"
Sep 16 18:48:16.500890 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.500667    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4639fcfa-4d2b-4f72-be92-1c9a3954c101","Type":"ContainerDied","Data":"050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87"}
Sep 16 18:48:16.500890 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.500817    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4639fcfa-4d2b-4f72-be92-1c9a3954c101","Type":"ContainerDied","Data":"c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526"}
Sep 16 18:48:16.500890 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.500861    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4639fcfa-4d2b-4f72-be92-1c9a3954c101","Type":"ContainerDied","Data":"44fe27ee41353f575a29fdd135ae8346bce3587d8446173328a7fc7e92ce8e40"}
Sep 16 18:48:16.513380 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.513279    2557 scope.go:117] "RemoveContainer" containerID="3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185"
Sep 16 18:48:16.525708 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.525691    2557 scope.go:117] "RemoveContainer" containerID="d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe"
Sep 16 18:48:16.532407 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.532391    2557 scope.go:117] "RemoveContainer" containerID="e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729"
Sep 16 18:48:16.538665 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.538650    2557 scope.go:117] "RemoveContainer" containerID="050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87"
Sep 16 18:48:16.544390 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.544365    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 18:48:16.546721 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.546704    2557 scope.go:117] "RemoveContainer" containerID="c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526"
Sep 16 18:48:16.547176 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.547155    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 18:48:16.553581 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.553566    2557 scope.go:117] "RemoveContainer" containerID="b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104"
Sep 16 18:48:16.560151 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.560135    2557 scope.go:117] "RemoveContainer" containerID="d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb"
Sep 16 18:48:16.560419 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:48:16.560403    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb\": container with ID starting with d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb not found: ID does not exist" containerID="d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb"
Sep 16 18:48:16.560465 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.560426    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb"} err="failed to get container status \"d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb\": rpc error: code = NotFound desc = could not find container \"d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb\": container with ID starting with d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb not found: ID does not exist"
Sep 16 18:48:16.560465 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.560443    2557 scope.go:117] "RemoveContainer" containerID="3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185"
Sep 16 18:48:16.560684 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:48:16.560665    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185\": container with ID starting with 3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185 not found: ID does not exist" containerID="3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185"
Sep 16 18:48:16.560729 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.560689    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185"} err="failed to get container status \"3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185\": rpc error: code = NotFound desc = could not find container \"3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185\": container with ID starting with 3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185 not found: ID does not exist"
Sep 16 18:48:16.560729 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.560704    2557 scope.go:117] "RemoveContainer" containerID="d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe"
Sep 16 18:48:16.560942 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:48:16.560925    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe\": container with ID starting with d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe not found: ID does not exist" containerID="d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe"
Sep 16 18:48:16.560982 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.560946    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe"} err="failed to get container status \"d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe\": rpc error: code = NotFound desc = could not find container \"d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe\": container with ID starting with d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe not found: ID does not exist"
Sep 16 18:48:16.560982 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.560958    2557 scope.go:117] "RemoveContainer" containerID="e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729"
Sep 16 18:48:16.561171 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:48:16.561157    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729\": container with ID starting with e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729 not found: ID does not exist" containerID="e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729"
Sep 16 18:48:16.561213 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.561173    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729"} err="failed to get container status \"e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729\": rpc error: code = NotFound desc = could not find container \"e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729\": container with ID starting with e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729 not found: ID does not exist"
Sep 16 18:48:16.561213 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.561185    2557 scope.go:117] "RemoveContainer" containerID="050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87"
Sep 16 18:48:16.561381 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:48:16.561367    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87\": container with ID starting with 050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87 not found: ID does not exist" containerID="050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87"
Sep 16 18:48:16.561418 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.561384    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87"} err="failed to get container status \"050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87\": rpc error: code = NotFound desc = could not find container \"050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87\": container with ID starting with 050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87 not found: ID does not exist"
Sep 16 18:48:16.561418 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.561395    2557 scope.go:117] "RemoveContainer" containerID="c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526"
Sep 16 18:48:16.561616 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:48:16.561600    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526\": container with ID starting with c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526 not found: ID does not exist" containerID="c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526"
Sep 16 18:48:16.561664 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.561618    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526"} err="failed to get container status \"c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526\": rpc error: code = NotFound desc = could not find container \"c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526\": container with ID starting with c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526 not found: ID does not exist"
Sep 16 18:48:16.561664 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.561630    2557 scope.go:117] "RemoveContainer" containerID="b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104"
Sep 16 18:48:16.561877 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:48:16.561860    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104\": container with ID starting with b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104 not found: ID does not exist" containerID="b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104"
Sep 16 18:48:16.561941 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.561881    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104"} err="failed to get container status \"b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104\": rpc error: code = NotFound desc = could not find container \"b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104\": container with ID starting with b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104 not found: ID does not exist"
Sep 16 18:48:16.561941 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.561894    2557 scope.go:117] "RemoveContainer" containerID="d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb"
Sep 16 18:48:16.562116 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.562098    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb"} err="failed to get container status \"d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb\": rpc error: code = NotFound desc = could not find container \"d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb\": container with ID starting with d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb not found: ID does not exist"
Sep 16 18:48:16.562156 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.562116    2557 scope.go:117] "RemoveContainer" containerID="3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185"
Sep 16 18:48:16.562308 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.562294    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185"} err="failed to get container status \"3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185\": rpc error: code = NotFound desc = could not find container \"3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185\": container with ID starting with 3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185 not found: ID does not exist"
Sep 16 18:48:16.562345 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.562308    2557 scope.go:117] "RemoveContainer" containerID="d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe"
Sep 16 18:48:16.562569 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.562544    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe"} err="failed to get container status \"d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe\": rpc error: code = NotFound desc = could not find container \"d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe\": container with ID starting with d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe not found: ID does not exist"
Sep 16 18:48:16.562569 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.562568    2557 scope.go:117] "RemoveContainer" containerID="e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729"
Sep 16 18:48:16.562800 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.562784    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729"} err="failed to get container status \"e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729\": rpc error: code = NotFound desc = could not find container \"e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729\": container with ID starting with e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729 not found: ID does not exist"
Sep 16 18:48:16.562800 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.562800    2557 scope.go:117] "RemoveContainer" containerID="050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87"
Sep 16 18:48:16.563024 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.563006    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87"} err="failed to get container status \"050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87\": rpc error: code = NotFound desc = could not find container \"050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87\": container with ID starting with 050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87 not found: ID does not exist"
Sep 16 18:48:16.563024 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.563023    2557 scope.go:117] "RemoveContainer" containerID="c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526"
Sep 16 18:48:16.563211 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.563193    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526"} err="failed to get container status \"c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526\": rpc error: code = NotFound desc = could not find container \"c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526\": container with ID starting with c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526 not found: ID does not exist"
Sep 16 18:48:16.563211 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.563207    2557 scope.go:117] "RemoveContainer" containerID="b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104"
Sep 16 18:48:16.563415 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.563400    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104"} err="failed to get container status \"b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104\": rpc error: code = NotFound desc = could not find container \"b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104\": container with ID starting with b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104 not found: ID does not exist"
Sep 16 18:48:16.563415 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.563413    2557 scope.go:117] "RemoveContainer" containerID="d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb"
Sep 16 18:48:16.563593 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.563578    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb"} err="failed to get container status \"d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb\": rpc error: code = NotFound desc = could not find container \"d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb\": container with ID starting with d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb not found: ID does not exist"
Sep 16 18:48:16.563646 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.563593    2557 scope.go:117] "RemoveContainer" containerID="3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185"
Sep 16 18:48:16.563810 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.563792    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185"} err="failed to get container status \"3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185\": rpc error: code = NotFound desc = could not find container \"3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185\": container with ID starting with 3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185 not found: ID does not exist"
Sep 16 18:48:16.563810 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.563809    2557 scope.go:117] "RemoveContainer" containerID="d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe"
Sep 16 18:48:16.564022 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.564004    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe"} err="failed to get container status \"d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe\": rpc error: code = NotFound desc = could not find container \"d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe\": container with ID starting with d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe not found: ID does not exist"
Sep 16 18:48:16.564098 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.564024    2557 scope.go:117] "RemoveContainer" containerID="e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729"
Sep 16 18:48:16.564251 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.564236    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729"} err="failed to get container status \"e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729\": rpc error: code = NotFound desc = could not find container \"e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729\": container with ID starting with e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729 not found: ID does not exist"
Sep 16 18:48:16.564251 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.564250    2557 scope.go:117] "RemoveContainer" containerID="050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87"
Sep 16 18:48:16.564421 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.564407    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87"} err="failed to get container status \"050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87\": rpc error: code = NotFound desc = could not find container \"050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87\": container with ID starting with 050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87 not found: ID does not exist"
Sep 16 18:48:16.564421 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.564420    2557 scope.go:117] "RemoveContainer" containerID="c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526"
Sep 16 18:48:16.564616 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.564599    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526"} err="failed to get container status \"c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526\": rpc error: code = NotFound desc = could not find container \"c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526\": container with ID starting with c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526 not found: ID does not exist"
Sep 16 18:48:16.564665 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.564618    2557 scope.go:117] "RemoveContainer" containerID="b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104"
Sep 16 18:48:16.564866 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.564852    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104"} err="failed to get container status \"b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104\": rpc error: code = NotFound desc = could not find container \"b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104\": container with ID starting with b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104 not found: ID does not exist"
Sep 16 18:48:16.564866 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.564865    2557 scope.go:117] "RemoveContainer" containerID="d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb"
Sep 16 18:48:16.565089 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.565073    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb"} err="failed to get container status \"d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb\": rpc error: code = NotFound desc = could not find container \"d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb\": container with ID starting with d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb not found: ID does not exist"
Sep 16 18:48:16.565131 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.565088    2557 scope.go:117] "RemoveContainer" containerID="3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185"
Sep 16 18:48:16.565309 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.565294    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185"} err="failed to get container status \"3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185\": rpc error: code = NotFound desc = could not find container \"3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185\": container with ID starting with 3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185 not found: ID does not exist"
Sep 16 18:48:16.565309 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.565308    2557 scope.go:117] "RemoveContainer" containerID="d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe"
Sep 16 18:48:16.565510 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.565496    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe"} err="failed to get container status \"d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe\": rpc error: code = NotFound desc = could not find container \"d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe\": container with ID starting with d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe not found: ID does not exist"
Sep 16 18:48:16.565555 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.565510    2557 scope.go:117] "RemoveContainer" containerID="e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729"
Sep 16 18:48:16.565710 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.565692    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729"} err="failed to get container status \"e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729\": rpc error: code = NotFound desc = could not find container \"e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729\": container with ID starting with e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729 not found: ID does not exist"
Sep 16 18:48:16.565747 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.565711    2557 scope.go:117] "RemoveContainer" containerID="050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87"
Sep 16 18:48:16.565953 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.565935    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87"} err="failed to get container status \"050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87\": rpc error: code = NotFound desc = could not find container \"050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87\": container with ID starting with 050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87 not found: ID does not exist"
Sep 16 18:48:16.565953 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.565952    2557 scope.go:117] "RemoveContainer" containerID="c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526"
Sep 16 18:48:16.566139 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.566124    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526"} err="failed to get container status \"c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526\": rpc error: code = NotFound desc = could not find container \"c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526\": container with ID starting with c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526 not found: ID does not exist"
Sep 16 18:48:16.566177 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.566139    2557 scope.go:117] "RemoveContainer" containerID="b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104"
Sep 16 18:48:16.566356 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.566340    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104"} err="failed to get container status \"b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104\": rpc error: code = NotFound desc = could not find container \"b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104\": container with ID starting with b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104 not found: ID does not exist"
Sep 16 18:48:16.566356 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.566355    2557 scope.go:117] "RemoveContainer" containerID="d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb"
Sep 16 18:48:16.566596 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.566575    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb"} err="failed to get container status \"d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb\": rpc error: code = NotFound desc = could not find container \"d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb\": container with ID starting with d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb not found: ID does not exist"
Sep 16 18:48:16.566596 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.566595    2557 scope.go:117] "RemoveContainer" containerID="3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185"
Sep 16 18:48:16.566820 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.566799    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185"} err="failed to get container status \"3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185\": rpc error: code = NotFound desc = could not find container \"3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185\": container with ID starting with 3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185 not found: ID does not exist"
Sep 16 18:48:16.566875 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.566820    2557 scope.go:117] "RemoveContainer" containerID="d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe"
Sep 16 18:48:16.567018 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.567000    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe"} err="failed to get container status \"d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe\": rpc error: code = NotFound desc = could not find container \"d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe\": container with ID starting with d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe not found: ID does not exist"
Sep 16 18:48:16.567084 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.567020    2557 scope.go:117] "RemoveContainer" containerID="e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729"
Sep 16 18:48:16.567215 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.567198    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729"} err="failed to get container status \"e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729\": rpc error: code = NotFound desc = could not find container \"e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729\": container with ID starting with e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729 not found: ID does not exist"
Sep 16 18:48:16.567281 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.567216    2557 scope.go:117] "RemoveContainer" containerID="050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87"
Sep 16 18:48:16.567465 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.567443    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87"} err="failed to get container status \"050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87\": rpc error: code = NotFound desc = could not find container \"050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87\": container with ID starting with 050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87 not found: ID does not exist"
Sep 16 18:48:16.567513 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.567466    2557 scope.go:117] "RemoveContainer" containerID="c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526"
Sep 16 18:48:16.567710 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.567694    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526"} err="failed to get container status \"c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526\": rpc error: code = NotFound desc = could not find container \"c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526\": container with ID starting with c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526 not found: ID does not exist"
Sep 16 18:48:16.567751 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.567711    2557 scope.go:117] "RemoveContainer" containerID="b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104"
Sep 16 18:48:16.567941 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.567924    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104"} err="failed to get container status \"b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104\": rpc error: code = NotFound desc = could not find container \"b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104\": container with ID starting with b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104 not found: ID does not exist"
Sep 16 18:48:16.567989 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.567942    2557 scope.go:117] "RemoveContainer" containerID="d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb"
Sep 16 18:48:16.568157 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.568140    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb"} err="failed to get container status \"d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb\": rpc error: code = NotFound desc = could not find container \"d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb\": container with ID starting with d9e6d8ccd1feaa10b37e547cfde840d84b6970b4e220767655bd0bc3c096dbcb not found: ID does not exist"
Sep 16 18:48:16.568205 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.568157    2557 scope.go:117] "RemoveContainer" containerID="3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185"
Sep 16 18:48:16.568372 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.568356    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185"} err="failed to get container status \"3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185\": rpc error: code = NotFound desc = could not find container \"3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185\": container with ID starting with 3486daca2366e2eff70ceb2a5f71e68149e30ffc25dff9f229301f16a6b31185 not found: ID does not exist"
Sep 16 18:48:16.568412 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.568372    2557 scope.go:117] "RemoveContainer" containerID="d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe"
Sep 16 18:48:16.568579 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.568562    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe"} err="failed to get container status \"d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe\": rpc error: code = NotFound desc = could not find container \"d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe\": container with ID starting with d175347b4c675a6d3360f16218f014285f4c9845742940055d51b3c8c46f3dbe not found: ID does not exist"
Sep 16 18:48:16.568623 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.568579    2557 scope.go:117] "RemoveContainer" containerID="e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729"
Sep 16 18:48:16.568835 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.568816    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729"} err="failed to get container status \"e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729\": rpc error: code = NotFound desc = could not find container \"e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729\": container with ID starting with e37b4da1fccf8f0ec58377f9255b00d4bfcd02de7ef1bdebcdedfd849caa8729 not found: ID does not exist"
Sep 16 18:48:16.568835 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.568835    2557 scope.go:117] "RemoveContainer" containerID="050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87"
Sep 16 18:48:16.569052 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.569035    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87"} err="failed to get container status \"050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87\": rpc error: code = NotFound desc = could not find container \"050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87\": container with ID starting with 050b850f2e59924b92f139b82b8b34d5abf8625b0c8ee4af873de566b945cb87 not found: ID does not exist"
Sep 16 18:48:16.569097 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.569052    2557 scope.go:117] "RemoveContainer" containerID="c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526"
Sep 16 18:48:16.569273 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.569256    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526"} err="failed to get container status \"c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526\": rpc error: code = NotFound desc = could not find container \"c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526\": container with ID starting with c65d7a3a4da72e84245d72f343bd4d3d69ba64017a16835e001e31bbd63eb526 not found: ID does not exist"
Sep 16 18:48:16.569337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.569274    2557 scope.go:117] "RemoveContainer" containerID="b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104"
Sep 16 18:48:16.569535 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.569517    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104"} err="failed to get container status \"b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104\": rpc error: code = NotFound desc = could not find container \"b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104\": container with ID starting with b2e47254d805bb985eccbdad15bb46f7150fda3aa2edea7c6cee23718e3a3104 not found: ID does not exist"
Sep 16 18:48:16.602807 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.602784    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 18:48:16.603073 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603062    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54674a4a-c1b7-429a-a4a9-21f7f93d22ab" containerName="registry"
Sep 16 18:48:16.603114 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603075    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="54674a4a-c1b7-429a-a4a9-21f7f93d22ab" containerName="registry"
Sep 16 18:48:16.603114 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603083    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerName="alertmanager"
Sep 16 18:48:16.603114 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603089    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerName="alertmanager"
Sep 16 18:48:16.603114 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603097    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerName="kube-rbac-proxy"
Sep 16 18:48:16.603114 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603102    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerName="kube-rbac-proxy"
Sep 16 18:48:16.603114 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603108    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerName="kube-rbac-proxy-metric"
Sep 16 18:48:16.603114 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603113    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerName="kube-rbac-proxy-metric"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603121    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerName="thanos-sidecar"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603126    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerName="thanos-sidecar"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603134    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerName="kube-rbac-proxy"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603139    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerName="kube-rbac-proxy"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603145    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerName="prom-label-proxy"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603149    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerName="prom-label-proxy"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603156    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerName="kube-rbac-proxy-thanos"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603161    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerName="kube-rbac-proxy-thanos"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603168    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerName="init-config-reloader"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603173    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerName="init-config-reloader"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603179    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerName="init-config-reloader"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603184    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerName="init-config-reloader"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603191    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerName="config-reloader"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603196    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerName="config-reloader"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603202    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerName="prometheus"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603207    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerName="prometheus"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603213    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerName="kube-rbac-proxy-web"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603218    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerName="kube-rbac-proxy-web"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603224    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerName="kube-rbac-proxy-web"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603230    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerName="kube-rbac-proxy-web"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603236    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerName="config-reloader"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603241    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerName="config-reloader"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603285    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerName="prom-label-proxy"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603294    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerName="kube-rbac-proxy"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603300    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerName="thanos-sidecar"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603305    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerName="kube-rbac-proxy"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603311    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerName="kube-rbac-proxy-metric"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603317    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerName="alertmanager"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603321    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerName="config-reloader"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603326    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerName="kube-rbac-proxy-web"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603332    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="54674a4a-c1b7-429a-a4a9-21f7f93d22ab" containerName="registry"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603337    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerName="config-reloader"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603342    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerName="kube-rbac-proxy-thanos"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603348    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e719e21-4e76-4c1c-89aa-032e80db0638" containerName="kube-rbac-proxy-web"
Sep 16 18:48:16.603337 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.603352    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" containerName="prometheus"
Sep 16 18:48:16.608509 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.608492    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.619452 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.619435    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 16 18:48:16.619604 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.619588    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 16 18:48:16.620446 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.620431    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-qtbxc\""
Sep 16 18:48:16.621314 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.621299    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 16 18:48:16.621356 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.621325    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 16 18:48:16.621387 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.621301    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 16 18:48:16.621964 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.621951    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 16 18:48:16.622069 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.622056    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-f4m3f7i7atasa\""
Sep 16 18:48:16.623778 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.623756    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 16 18:48:16.624053 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.624039    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 16 18:48:16.624126 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.624111    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 16 18:48:16.624171 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.624121    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 16 18:48:16.624352 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.624337    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 16 18:48:16.624412 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.624346    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 16 18:48:16.626995 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.626978    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 16 18:48:16.630493 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.630475    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 16 18:48:16.644939 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.644910    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 18:48:16.705526 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.705494    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/497bc5a6-4deb-4ca4-a954-469f358a1a01-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.705690 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.705529    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/497bc5a6-4deb-4ca4-a954-469f358a1a01-web-config\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.705690 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.705552    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqnjs\" (UniqueName: \"kubernetes.io/projected/497bc5a6-4deb-4ca4-a954-469f358a1a01-kube-api-access-tqnjs\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.705690 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.705569    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/497bc5a6-4deb-4ca4-a954-469f358a1a01-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.705690 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.705587    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/497bc5a6-4deb-4ca4-a954-469f358a1a01-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.705690 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.705625    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/497bc5a6-4deb-4ca4-a954-469f358a1a01-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.705690 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.705642    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/497bc5a6-4deb-4ca4-a954-469f358a1a01-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.705892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.705706    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/497bc5a6-4deb-4ca4-a954-469f358a1a01-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.705892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.705764    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/497bc5a6-4deb-4ca4-a954-469f358a1a01-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.705892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.705808    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/497bc5a6-4deb-4ca4-a954-469f358a1a01-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.705892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.705831    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/497bc5a6-4deb-4ca4-a954-469f358a1a01-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.705892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.705847    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/497bc5a6-4deb-4ca4-a954-469f358a1a01-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.705892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.705868    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/497bc5a6-4deb-4ca4-a954-469f358a1a01-config\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.705892 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.705885    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/497bc5a6-4deb-4ca4-a954-469f358a1a01-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.706093 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.705903    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/497bc5a6-4deb-4ca4-a954-469f358a1a01-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.706093 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.705942    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/497bc5a6-4deb-4ca4-a954-469f358a1a01-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.706093 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.705964    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/497bc5a6-4deb-4ca4-a954-469f358a1a01-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.706093 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.705977    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/497bc5a6-4deb-4ca4-a954-469f358a1a01-config-out\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.706093 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.705992    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/497bc5a6-4deb-4ca4-a954-469f358a1a01-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.706093 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.706005    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/497bc5a6-4deb-4ca4-a954-469f358a1a01-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.806737 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.806711    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/497bc5a6-4deb-4ca4-a954-469f358a1a01-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.806924 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.806740    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/497bc5a6-4deb-4ca4-a954-469f358a1a01-config-out\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.806924 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.806757    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/497bc5a6-4deb-4ca4-a954-469f358a1a01-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.806924 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.806783    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/497bc5a6-4deb-4ca4-a954-469f358a1a01-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.806924 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.806807    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/497bc5a6-4deb-4ca4-a954-469f358a1a01-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.806924 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.806832    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/497bc5a6-4deb-4ca4-a954-469f358a1a01-web-config\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.806924 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.806853    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqnjs\" (UniqueName: \"kubernetes.io/projected/497bc5a6-4deb-4ca4-a954-469f358a1a01-kube-api-access-tqnjs\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.806924 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.806878    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/497bc5a6-4deb-4ca4-a954-469f358a1a01-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.807264 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.807189    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/497bc5a6-4deb-4ca4-a954-469f358a1a01-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.807264 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.807238    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/497bc5a6-4deb-4ca4-a954-469f358a1a01-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.807366 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.807278    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/497bc5a6-4deb-4ca4-a954-469f358a1a01-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.807366 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.807322    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/497bc5a6-4deb-4ca4-a954-469f358a1a01-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.807366 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.807354    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/497bc5a6-4deb-4ca4-a954-469f358a1a01-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.807687 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.807383    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/497bc5a6-4deb-4ca4-a954-469f358a1a01-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.807687 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.807508    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/497bc5a6-4deb-4ca4-a954-469f358a1a01-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.809021 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.807708    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/497bc5a6-4deb-4ca4-a954-469f358a1a01-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.809021 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.807865    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/497bc5a6-4deb-4ca4-a954-469f358a1a01-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.809021 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.807909    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/497bc5a6-4deb-4ca4-a954-469f358a1a01-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.809021 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.807944    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/497bc5a6-4deb-4ca4-a954-469f358a1a01-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.809021 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.807995    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/497bc5a6-4deb-4ca4-a954-469f358a1a01-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.809021 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.807908    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/497bc5a6-4deb-4ca4-a954-469f358a1a01-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.809021 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.808095    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/497bc5a6-4deb-4ca4-a954-469f358a1a01-config\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.809021 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.808148    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/497bc5a6-4deb-4ca4-a954-469f358a1a01-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.809021 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.808183    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/497bc5a6-4deb-4ca4-a954-469f358a1a01-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.809021 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.808311    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/497bc5a6-4deb-4ca4-a954-469f358a1a01-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.809021 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.808576    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/497bc5a6-4deb-4ca4-a954-469f358a1a01-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.809021 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.808618    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/497bc5a6-4deb-4ca4-a954-469f358a1a01-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.810536 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.810510    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/497bc5a6-4deb-4ca4-a954-469f358a1a01-web-config\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.810635 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.810544    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/497bc5a6-4deb-4ca4-a954-469f358a1a01-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.810980 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.810952    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/497bc5a6-4deb-4ca4-a954-469f358a1a01-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.811161 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.811113    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/497bc5a6-4deb-4ca4-a954-469f358a1a01-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.811601 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.811573    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/497bc5a6-4deb-4ca4-a954-469f358a1a01-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.811947 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.811923    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/497bc5a6-4deb-4ca4-a954-469f358a1a01-config\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.812461 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.812407    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/497bc5a6-4deb-4ca4-a954-469f358a1a01-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.812461 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.812433    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/497bc5a6-4deb-4ca4-a954-469f358a1a01-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.812614 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.812468    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/497bc5a6-4deb-4ca4-a954-469f358a1a01-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.812712 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.812695    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/497bc5a6-4deb-4ca4-a954-469f358a1a01-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.813089 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.813072    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/497bc5a6-4deb-4ca4-a954-469f358a1a01-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.813224 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.813208    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/497bc5a6-4deb-4ca4-a954-469f358a1a01-config-out\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.817141 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.817118    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqnjs\" (UniqueName: \"kubernetes.io/projected/497bc5a6-4deb-4ca4-a954-469f358a1a01-kube-api-access-tqnjs\") pod \"prometheus-k8s-0\" (UID: \"497bc5a6-4deb-4ca4-a954-469f358a1a01\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:16.916634 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:16.916602    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:48:17.201572 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:17.201549    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 18:48:17.507311 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:17.507230    2557 generic.go:356] "Generic (PLEG): container finished" podID="497bc5a6-4deb-4ca4-a954-469f358a1a01" containerID="3c97114857b80ebf398edf027b61d2f45180c9b9326a4f3df9c4c3f851faa962" exitCode=0
Sep 16 18:48:17.507311 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:17.507267    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"497bc5a6-4deb-4ca4-a954-469f358a1a01","Type":"ContainerDied","Data":"3c97114857b80ebf398edf027b61d2f45180c9b9326a4f3df9c4c3f851faa962"}
Sep 16 18:48:17.507666 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:17.507320    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"497bc5a6-4deb-4ca4-a954-469f358a1a01","Type":"ContainerStarted","Data":"0e4fb162a56382620f17ba8e902f748a41992bc412d6442c3a4444c0784ea1f9"}
Sep 16 18:48:18.466085 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:18.466050    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4639fcfa-4d2b-4f72-be92-1c9a3954c101" path="/var/lib/kubelet/pods/4639fcfa-4d2b-4f72-be92-1c9a3954c101/volumes"
Sep 16 18:48:18.516269 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:18.516241    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"497bc5a6-4deb-4ca4-a954-469f358a1a01","Type":"ContainerStarted","Data":"69cff3cdcddf3f63bfd3bb9d59f9371cae9353cf35b5eceabeb4f06886b4469b"}
Sep 16 18:48:18.516580 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:18.516299    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"497bc5a6-4deb-4ca4-a954-469f358a1a01","Type":"ContainerStarted","Data":"34c436629927b54e58ebf475219af867608d8e34427f71208312cb786920a218"}
Sep 16 18:48:18.516580 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:18.516324    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"497bc5a6-4deb-4ca4-a954-469f358a1a01","Type":"ContainerStarted","Data":"bf897863bfaa9661ccf0ff258a704b950450610c967f11721c239681cb3a92f4"}
Sep 16 18:48:18.516580 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:18.516352    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"497bc5a6-4deb-4ca4-a954-469f358a1a01","Type":"ContainerStarted","Data":"2031064ca1c991ebfe53a2ff6cd9467b1d7bdf9f4e5a50166274eb4a1dbed87d"}
Sep 16 18:48:18.516580 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:18.516376    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"497bc5a6-4deb-4ca4-a954-469f358a1a01","Type":"ContainerStarted","Data":"905134e3f04611b04192f37b54eadfb2667f8e70bbc1bc55e3e7a8a7f75ca26c"}
Sep 16 18:48:18.516580 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:18.516397    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"497bc5a6-4deb-4ca4-a954-469f358a1a01","Type":"ContainerStarted","Data":"fdf5faf0eff0ed9a955dc7b24374033de8b2c92a54ddaae4031931c459f3ae52"}
Sep 16 18:48:18.550000 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:18.549957    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.527970624 podStartE2EDuration="2.549942851s" totalImagesPullingTime="0s" totalInitContainerRuntime="21.972227ms" isStatefulPod=false podCreationTimestamp="2026-09-16 18:48:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:48:18.547365516 +0000 UTC m=+212.624470390" watchObservedRunningTime="2026-09-16 18:48:18.549942851 +0000 UTC m=+212.627047711"
Sep 16 18:48:21.917198 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:48:21.917158    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:49:16.917544 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:49:16.917461    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:49:16.932521 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:49:16.932492    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:49:17.756171 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:49:17.756140    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:49:46.350834 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:49:46.350808    2557 kubelet.go:1648] "Image garbage collection succeeded"
Sep 16 18:52:39.767239 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:39.767208    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-7b966d7984-pks9w"]
Sep 16 18:52:39.770522 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:39.770501    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7b966d7984-pks9w"
Sep 16 18:52:39.773511 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:39.773490    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-metrics-tls\""
Sep 16 18:52:39.773692 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:39.773673    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Sep 16 18:52:39.773796 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:39.773735    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Sep 16 18:52:39.774891 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:39.774875    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Sep 16 18:52:39.774984 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:39.774967    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-4r8zr\""
Sep 16 18:52:39.787492 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:39.787469    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7b966d7984-pks9w"]
Sep 16 18:52:39.939524 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:39.939488    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ec3e5f7-fc05-463f-8623-1f294e553cbd-metrics-certs\") pod \"odh-model-controller-7b966d7984-pks9w\" (UID: \"4ec3e5f7-fc05-463f-8623-1f294e553cbd\") " pod="kserve/odh-model-controller-7b966d7984-pks9w"
Sep 16 18:52:39.939694 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:39.939546    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ec3e5f7-fc05-463f-8623-1f294e553cbd-cert\") pod \"odh-model-controller-7b966d7984-pks9w\" (UID: \"4ec3e5f7-fc05-463f-8623-1f294e553cbd\") " pod="kserve/odh-model-controller-7b966d7984-pks9w"
Sep 16 18:52:39.939694 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:39.939632    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lktx2\" (UniqueName: \"kubernetes.io/projected/4ec3e5f7-fc05-463f-8623-1f294e553cbd-kube-api-access-lktx2\") pod \"odh-model-controller-7b966d7984-pks9w\" (UID: \"4ec3e5f7-fc05-463f-8623-1f294e553cbd\") " pod="kserve/odh-model-controller-7b966d7984-pks9w"
Sep 16 18:52:40.040905 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:40.040879    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ec3e5f7-fc05-463f-8623-1f294e553cbd-cert\") pod \"odh-model-controller-7b966d7984-pks9w\" (UID: \"4ec3e5f7-fc05-463f-8623-1f294e553cbd\") " pod="kserve/odh-model-controller-7b966d7984-pks9w"
Sep 16 18:52:40.041092 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:40.040957    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lktx2\" (UniqueName: \"kubernetes.io/projected/4ec3e5f7-fc05-463f-8623-1f294e553cbd-kube-api-access-lktx2\") pod \"odh-model-controller-7b966d7984-pks9w\" (UID: \"4ec3e5f7-fc05-463f-8623-1f294e553cbd\") " pod="kserve/odh-model-controller-7b966d7984-pks9w"
Sep 16 18:52:40.041092 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:40.041003    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ec3e5f7-fc05-463f-8623-1f294e553cbd-metrics-certs\") pod \"odh-model-controller-7b966d7984-pks9w\" (UID: \"4ec3e5f7-fc05-463f-8623-1f294e553cbd\") " pod="kserve/odh-model-controller-7b966d7984-pks9w"
Sep 16 18:52:40.043496 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:40.043466    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ec3e5f7-fc05-463f-8623-1f294e553cbd-cert\") pod \"odh-model-controller-7b966d7984-pks9w\" (UID: \"4ec3e5f7-fc05-463f-8623-1f294e553cbd\") " pod="kserve/odh-model-controller-7b966d7984-pks9w"
Sep 16 18:52:40.043601 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:40.043466    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ec3e5f7-fc05-463f-8623-1f294e553cbd-metrics-certs\") pod \"odh-model-controller-7b966d7984-pks9w\" (UID: \"4ec3e5f7-fc05-463f-8623-1f294e553cbd\") " pod="kserve/odh-model-controller-7b966d7984-pks9w"
Sep 16 18:52:40.051000 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:40.050978    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lktx2\" (UniqueName: \"kubernetes.io/projected/4ec3e5f7-fc05-463f-8623-1f294e553cbd-kube-api-access-lktx2\") pod \"odh-model-controller-7b966d7984-pks9w\" (UID: \"4ec3e5f7-fc05-463f-8623-1f294e553cbd\") " pod="kserve/odh-model-controller-7b966d7984-pks9w"
Sep 16 18:52:40.080826 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:40.080806    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7b966d7984-pks9w"
Sep 16 18:52:40.352086 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:40.352062    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7b966d7984-pks9w"]
Sep 16 18:52:40.354338 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:52:40.354313    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ec3e5f7_fc05_463f_8623_1f294e553cbd.slice/crio-1b9b4ff5fc297ecd4cc4d10327c9be8a4afccec91f005e0eeabd686e13bc82a5 WatchSource:0}: Error finding container 1b9b4ff5fc297ecd4cc4d10327c9be8a4afccec91f005e0eeabd686e13bc82a5: Status 404 returned error can't find the container with id 1b9b4ff5fc297ecd4cc4d10327c9be8a4afccec91f005e0eeabd686e13bc82a5
Sep 16 18:52:40.355575 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:40.355558    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:52:40.507900 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:40.507866    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7b966d7984-pks9w" event={"ID":"4ec3e5f7-fc05-463f-8623-1f294e553cbd","Type":"ContainerStarted","Data":"1b9b4ff5fc297ecd4cc4d10327c9be8a4afccec91f005e0eeabd686e13bc82a5"}
Sep 16 18:52:43.524059 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:43.524017    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7b966d7984-pks9w" event={"ID":"4ec3e5f7-fc05-463f-8623-1f294e553cbd","Type":"ContainerStarted","Data":"36ef421c89eee50e3fc89b6f0ee0db48715fc31818c2ff617cb2a6345d9146da"}
Sep 16 18:52:43.524520 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:43.524148    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-7b966d7984-pks9w"
Sep 16 18:52:54.529281 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:54.529247    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-7b966d7984-pks9w"
Sep 16 18:52:54.547176 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:54.547127    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-7b966d7984-pks9w" podStartSLOduration=12.901354086 podStartE2EDuration="15.54711329s" totalImagesPullingTime="2.645759204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:52:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:52:43.551939434 +0000 UTC m=+477.629044295" watchObservedRunningTime="2026-09-16 18:52:54.54711329 +0000 UTC m=+488.624218154"
Sep 16 18:52:55.339673 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:55.339644    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-87rph"]
Sep 16 18:52:55.343389 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:55.343371    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-87rph"
Sep 16 18:52:55.347873 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:55.347849    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Sep 16 18:52:55.347997 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:55.347946    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"hf-token\""
Sep 16 18:52:55.347997 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:55.347955    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-6x7qm\""
Sep 16 18:52:55.353817 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:55.353795    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-87rph"]
Sep 16 18:52:55.375073 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:55.375044    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t6n4\" (UniqueName: \"kubernetes.io/projected/f37977cb-a549-4543-9061-f7e25bef7bbb-kube-api-access-9t6n4\") pod \"s3-init-87rph\" (UID: \"f37977cb-a549-4543-9061-f7e25bef7bbb\") " pod="kserve/s3-init-87rph"
Sep 16 18:52:55.375193 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:55.375084    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f37977cb-a549-4543-9061-f7e25bef7bbb-model-cache\") pod \"s3-init-87rph\" (UID: \"f37977cb-a549-4543-9061-f7e25bef7bbb\") " pod="kserve/s3-init-87rph"
Sep 16 18:52:55.475682 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:55.475651    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t6n4\" (UniqueName: \"kubernetes.io/projected/f37977cb-a549-4543-9061-f7e25bef7bbb-kube-api-access-9t6n4\") pod \"s3-init-87rph\" (UID: \"f37977cb-a549-4543-9061-f7e25bef7bbb\") " pod="kserve/s3-init-87rph"
Sep 16 18:52:55.475682 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:55.475687    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f37977cb-a549-4543-9061-f7e25bef7bbb-model-cache\") pod \"s3-init-87rph\" (UID: \"f37977cb-a549-4543-9061-f7e25bef7bbb\") " pod="kserve/s3-init-87rph"
Sep 16 18:52:55.476017 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:55.476002    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f37977cb-a549-4543-9061-f7e25bef7bbb-model-cache\") pod \"s3-init-87rph\" (UID: \"f37977cb-a549-4543-9061-f7e25bef7bbb\") " pod="kserve/s3-init-87rph"
Sep 16 18:52:55.484501 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:55.484478    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t6n4\" (UniqueName: \"kubernetes.io/projected/f37977cb-a549-4543-9061-f7e25bef7bbb-kube-api-access-9t6n4\") pod \"s3-init-87rph\" (UID: \"f37977cb-a549-4543-9061-f7e25bef7bbb\") " pod="kserve/s3-init-87rph"
Sep 16 18:52:55.659359 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:55.659278    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-87rph"
Sep 16 18:52:55.931923 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:55.931897    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-87rph"]
Sep 16 18:52:55.934211 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:52:55.934184    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf37977cb_a549_4543_9061_f7e25bef7bbb.slice/crio-7fdc28ee0bdf5345ca8efc10859df6bd113c71302884ca4e11761570695a9143 WatchSource:0}: Error finding container 7fdc28ee0bdf5345ca8efc10859df6bd113c71302884ca4e11761570695a9143: Status 404 returned error can't find the container with id 7fdc28ee0bdf5345ca8efc10859df6bd113c71302884ca4e11761570695a9143
Sep 16 18:52:56.573594 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:56.573558    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-87rph" event={"ID":"f37977cb-a549-4543-9061-f7e25bef7bbb","Type":"ContainerStarted","Data":"7fdc28ee0bdf5345ca8efc10859df6bd113c71302884ca4e11761570695a9143"}
Sep 16 18:52:59.587756 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:52:59.587718    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-87rph" event={"ID":"f37977cb-a549-4543-9061-f7e25bef7bbb","Type":"ContainerStarted","Data":"f0d53c2798e82279080af08f0854160022cc7934f8a69c46dc0fc14657f43be7"}
Sep 16 18:53:01.598408 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:53:01.598371    2557 generic.go:356] "Generic (PLEG): container finished" podID="f37977cb-a549-4543-9061-f7e25bef7bbb" containerID="f0d53c2798e82279080af08f0854160022cc7934f8a69c46dc0fc14657f43be7" exitCode=0
Sep 16 18:53:01.598782 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:53:01.598420    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-87rph" event={"ID":"f37977cb-a549-4543-9061-f7e25bef7bbb","Type":"ContainerDied","Data":"f0d53c2798e82279080af08f0854160022cc7934f8a69c46dc0fc14657f43be7"}
Sep 16 18:53:06.624323 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:53:06.624287    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-87rph" event={"ID":"f37977cb-a549-4543-9061-f7e25bef7bbb","Type":"ContainerStarted","Data":"8be7f7eca7ecf67b0e12c2cf46250363e0c7c48006e63c7e63ee228dad5c73af"}
Sep 16 18:53:06.645182 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:53:06.645122    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-87rph" podStartSLOduration=2.494098313 podStartE2EDuration="11.645101511s" totalImagesPullingTime="7.782038809s" totalInitContainerRuntime="1.368964389s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:52:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:53:06.643071647 +0000 UTC m=+500.720176504" watchObservedRunningTime="2026-09-16 18:53:06.645101511 +0000 UTC m=+500.722206372"
Sep 16 18:54:11.883445 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:11.883409    2557 generic.go:356] "Generic (PLEG): container finished" podID="f37977cb-a549-4543-9061-f7e25bef7bbb" containerID="8be7f7eca7ecf67b0e12c2cf46250363e0c7c48006e63c7e63ee228dad5c73af" exitCode=0
Sep 16 18:54:11.883867 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:11.883480    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-87rph" event={"ID":"f37977cb-a549-4543-9061-f7e25bef7bbb","Type":"ContainerDied","Data":"8be7f7eca7ecf67b0e12c2cf46250363e0c7c48006e63c7e63ee228dad5c73af"}
Sep 16 18:54:13.004819 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:13.004791    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-87rph"
Sep 16 18:54:13.117285 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:13.117257    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f37977cb-a549-4543-9061-f7e25bef7bbb-model-cache\") pod \"f37977cb-a549-4543-9061-f7e25bef7bbb\" (UID: \"f37977cb-a549-4543-9061-f7e25bef7bbb\") "
Sep 16 18:54:13.117444 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:13.117297    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9t6n4\" (UniqueName: \"kubernetes.io/projected/f37977cb-a549-4543-9061-f7e25bef7bbb-kube-api-access-9t6n4\") pod \"f37977cb-a549-4543-9061-f7e25bef7bbb\" (UID: \"f37977cb-a549-4543-9061-f7e25bef7bbb\") "
Sep 16 18:54:13.119425 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:13.119389    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f37977cb-a549-4543-9061-f7e25bef7bbb-kube-api-access-9t6n4" (OuterVolumeSpecName: "kube-api-access-9t6n4") pod "f37977cb-a549-4543-9061-f7e25bef7bbb" (UID: "f37977cb-a549-4543-9061-f7e25bef7bbb"). InnerVolumeSpecName "kube-api-access-9t6n4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:54:13.135440 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:13.135414    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f37977cb-a549-4543-9061-f7e25bef7bbb-model-cache" (OuterVolumeSpecName: "model-cache") pod "f37977cb-a549-4543-9061-f7e25bef7bbb" (UID: "f37977cb-a549-4543-9061-f7e25bef7bbb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:54:13.218491 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:13.218423    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f37977cb-a549-4543-9061-f7e25bef7bbb-model-cache\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:54:13.218491 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:13.218450    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9t6n4\" (UniqueName: \"kubernetes.io/projected/f37977cb-a549-4543-9061-f7e25bef7bbb-kube-api-access-9t6n4\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:54:13.893571 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:13.893533    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-87rph" event={"ID":"f37977cb-a549-4543-9061-f7e25bef7bbb","Type":"ContainerDied","Data":"7fdc28ee0bdf5345ca8efc10859df6bd113c71302884ca4e11761570695a9143"}
Sep 16 18:54:13.893571 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:13.893556    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-87rph"
Sep 16 18:54:13.893571 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:13.893572    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7fdc28ee0bdf5345ca8efc10859df6bd113c71302884ca4e11761570695a9143"
Sep 16 18:54:25.618789 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.618747    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq"]
Sep 16 18:54:25.619151 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.619064    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f37977cb-a549-4543-9061-f7e25bef7bbb" containerName="download-hf-model"
Sep 16 18:54:25.619151 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.619076    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f37977cb-a549-4543-9061-f7e25bef7bbb" containerName="download-hf-model"
Sep 16 18:54:25.619151 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.619092    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f37977cb-a549-4543-9061-f7e25bef7bbb" containerName="s3-init"
Sep 16 18:54:25.619151 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.619098    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f37977cb-a549-4543-9061-f7e25bef7bbb" containerName="s3-init"
Sep 16 18:54:25.619314 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.619167    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f37977cb-a549-4543-9061-f7e25bef7bbb" containerName="s3-init"
Sep 16 18:54:25.621962 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.621945    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq"
Sep 16 18:54:25.625103 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.625079    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-predictor-serving-cert\""
Sep 16 18:54:25.625103 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.625086    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 16 18:54:25.625282 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.625160    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Sep 16 18:54:25.626053 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.626029    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-hvpnn\""
Sep 16 18:54:25.626172 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.626102    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 16 18:54:25.626172 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.626122    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Sep 16 18:54:25.626281 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.626201    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-kube-rbac-proxy-sar-config\""
Sep 16 18:54:25.638565 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.638544    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq"]
Sep 16 18:54:25.730454 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.730421    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/311b9459-414f-4afe-b06a-3e0e3f001570-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-595dc58456-t89gq\" (UID: \"311b9459-414f-4afe-b06a-3e0e3f001570\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq"
Sep 16 18:54:25.730623 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.730466    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/311b9459-414f-4afe-b06a-3e0e3f001570-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-595dc58456-t89gq\" (UID: \"311b9459-414f-4afe-b06a-3e0e3f001570\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq"
Sep 16 18:54:25.730743 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.730581    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/311b9459-414f-4afe-b06a-3e0e3f001570-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-595dc58456-t89gq\" (UID: \"311b9459-414f-4afe-b06a-3e0e3f001570\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq"
Sep 16 18:54:25.730743 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.730669    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smhcv\" (UniqueName: \"kubernetes.io/projected/311b9459-414f-4afe-b06a-3e0e3f001570-kube-api-access-smhcv\") pod \"isvc-sklearn-graph-1-predictor-595dc58456-t89gq\" (UID: \"311b9459-414f-4afe-b06a-3e0e3f001570\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq"
Sep 16 18:54:25.730872 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.730783    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/311b9459-414f-4afe-b06a-3e0e3f001570-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-595dc58456-t89gq\" (UID: \"311b9459-414f-4afe-b06a-3e0e3f001570\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq"
Sep 16 18:54:25.778105 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.778071    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm"]
Sep 16 18:54:25.781918 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.781897    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm"
Sep 16 18:54:25.784746 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.784701    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-aa795-predictor-serving-cert\""
Sep 16 18:54:25.784914 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.784897    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-aa795-kube-rbac-proxy-sar-config\""
Sep 16 18:54:25.794130 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.794109    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm"]
Sep 16 18:54:25.831551 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.831524    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/311b9459-414f-4afe-b06a-3e0e3f001570-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-595dc58456-t89gq\" (UID: \"311b9459-414f-4afe-b06a-3e0e3f001570\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq"
Sep 16 18:54:25.831692 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.831565    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smhcv\" (UniqueName: \"kubernetes.io/projected/311b9459-414f-4afe-b06a-3e0e3f001570-kube-api-access-smhcv\") pod \"isvc-sklearn-graph-1-predictor-595dc58456-t89gq\" (UID: \"311b9459-414f-4afe-b06a-3e0e3f001570\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq"
Sep 16 18:54:25.831692 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.831596    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/31051676-c5cc-45f1-9a97-41e8e2eb8687-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-aa795-predictor-6676fd456f-w6vxm\" (UID: \"31051676-c5cc-45f1-9a97-41e8e2eb8687\") " pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm"
Sep 16 18:54:25.831692 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.831624    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/311b9459-414f-4afe-b06a-3e0e3f001570-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-595dc58456-t89gq\" (UID: \"311b9459-414f-4afe-b06a-3e0e3f001570\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq"
Sep 16 18:54:25.831692 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.831669    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/31051676-c5cc-45f1-9a97-41e8e2eb8687-proxy-tls\") pod \"success-200-isvc-aa795-predictor-6676fd456f-w6vxm\" (UID: \"31051676-c5cc-45f1-9a97-41e8e2eb8687\") " pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm"
Sep 16 18:54:25.831945 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.831695    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78znf\" (UniqueName: \"kubernetes.io/projected/31051676-c5cc-45f1-9a97-41e8e2eb8687-kube-api-access-78znf\") pod \"success-200-isvc-aa795-predictor-6676fd456f-w6vxm\" (UID: \"31051676-c5cc-45f1-9a97-41e8e2eb8687\") " pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm"
Sep 16 18:54:25.831945 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.831761    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/311b9459-414f-4afe-b06a-3e0e3f001570-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-595dc58456-t89gq\" (UID: \"311b9459-414f-4afe-b06a-3e0e3f001570\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq"
Sep 16 18:54:25.831945 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.831852    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/311b9459-414f-4afe-b06a-3e0e3f001570-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-595dc58456-t89gq\" (UID: \"311b9459-414f-4afe-b06a-3e0e3f001570\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq"
Sep 16 18:54:25.832309 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.832266    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/311b9459-414f-4afe-b06a-3e0e3f001570-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-595dc58456-t89gq\" (UID: \"311b9459-414f-4afe-b06a-3e0e3f001570\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq"
Sep 16 18:54:25.832666 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.832643    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/311b9459-414f-4afe-b06a-3e0e3f001570-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-595dc58456-t89gq\" (UID: \"311b9459-414f-4afe-b06a-3e0e3f001570\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq"
Sep 16 18:54:25.832749 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.832643    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/311b9459-414f-4afe-b06a-3e0e3f001570-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-595dc58456-t89gq\" (UID: \"311b9459-414f-4afe-b06a-3e0e3f001570\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq"
Sep 16 18:54:25.834676 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.834655    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/311b9459-414f-4afe-b06a-3e0e3f001570-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-595dc58456-t89gq\" (UID: \"311b9459-414f-4afe-b06a-3e0e3f001570\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq"
Sep 16 18:54:25.842490 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.842460    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smhcv\" (UniqueName: \"kubernetes.io/projected/311b9459-414f-4afe-b06a-3e0e3f001570-kube-api-access-smhcv\") pod \"isvc-sklearn-graph-1-predictor-595dc58456-t89gq\" (UID: \"311b9459-414f-4afe-b06a-3e0e3f001570\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq"
Sep 16 18:54:25.932809 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.932700    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq"
Sep 16 18:54:25.932955 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.932892    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/31051676-c5cc-45f1-9a97-41e8e2eb8687-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-aa795-predictor-6676fd456f-w6vxm\" (UID: \"31051676-c5cc-45f1-9a97-41e8e2eb8687\") " pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm"
Sep 16 18:54:25.932955 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.932943    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/31051676-c5cc-45f1-9a97-41e8e2eb8687-proxy-tls\") pod \"success-200-isvc-aa795-predictor-6676fd456f-w6vxm\" (UID: \"31051676-c5cc-45f1-9a97-41e8e2eb8687\") " pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm"
Sep 16 18:54:25.933042 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.932962    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78znf\" (UniqueName: \"kubernetes.io/projected/31051676-c5cc-45f1-9a97-41e8e2eb8687-kube-api-access-78znf\") pod \"success-200-isvc-aa795-predictor-6676fd456f-w6vxm\" (UID: \"31051676-c5cc-45f1-9a97-41e8e2eb8687\") " pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm"
Sep 16 18:54:25.933508 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.933482    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/31051676-c5cc-45f1-9a97-41e8e2eb8687-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-aa795-predictor-6676fd456f-w6vxm\" (UID: \"31051676-c5cc-45f1-9a97-41e8e2eb8687\") " pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm"
Sep 16 18:54:25.935281 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.935257    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/31051676-c5cc-45f1-9a97-41e8e2eb8687-proxy-tls\") pod \"success-200-isvc-aa795-predictor-6676fd456f-w6vxm\" (UID: \"31051676-c5cc-45f1-9a97-41e8e2eb8687\") " pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm"
Sep 16 18:54:25.943341 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:25.943317    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78znf\" (UniqueName: \"kubernetes.io/projected/31051676-c5cc-45f1-9a97-41e8e2eb8687-kube-api-access-78znf\") pod \"success-200-isvc-aa795-predictor-6676fd456f-w6vxm\" (UID: \"31051676-c5cc-45f1-9a97-41e8e2eb8687\") " pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm"
Sep 16 18:54:26.089310 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.089279    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs"]
Sep 16 18:54:26.091812 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.091764    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm"
Sep 16 18:54:26.094100 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.094086    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs"
Sep 16 18:54:26.096943 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.096923    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-kube-rbac-proxy-sar-config\""
Sep 16 18:54:26.097149 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.097131    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-predictor-serving-cert\""
Sep 16 18:54:26.105129 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.105105    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs"]
Sep 16 18:54:26.135686 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.135649    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9c79e632-ba98-4a09-ad37-0992ed0e90f1-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-4pzzs\" (UID: \"9c79e632-ba98-4a09-ad37-0992ed0e90f1\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs"
Sep 16 18:54:26.135980 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.135954    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9c79e632-ba98-4a09-ad37-0992ed0e90f1-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-4pzzs\" (UID: \"9c79e632-ba98-4a09-ad37-0992ed0e90f1\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs"
Sep 16 18:54:26.136110 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.135993    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c79e632-ba98-4a09-ad37-0992ed0e90f1-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-4pzzs\" (UID: \"9c79e632-ba98-4a09-ad37-0992ed0e90f1\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs"
Sep 16 18:54:26.136110 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.136018    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9285x\" (UniqueName: \"kubernetes.io/projected/9c79e632-ba98-4a09-ad37-0992ed0e90f1-kube-api-access-9285x\") pod \"isvc-xgboost-graph-predictor-555fc69d56-4pzzs\" (UID: \"9c79e632-ba98-4a09-ad37-0992ed0e90f1\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs"
Sep 16 18:54:26.136110 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.136082    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9c79e632-ba98-4a09-ad37-0992ed0e90f1-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-4pzzs\" (UID: \"9c79e632-ba98-4a09-ad37-0992ed0e90f1\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs"
Sep 16 18:54:26.191112 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.191053    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq"]
Sep 16 18:54:26.195650 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.195626    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq"
Sep 16 18:54:26.198404 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.198381    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-predictor-serving-cert\""
Sep 16 18:54:26.198404 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.198402    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\""
Sep 16 18:54:26.205673 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.205650    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq"]
Sep 16 18:54:26.220048 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.220024    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq"]
Sep 16 18:54:26.223618 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:54:26.223594    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod311b9459_414f_4afe_b06a_3e0e3f001570.slice/crio-a6bf9d46a3df66d798560e3c8a12164e06122940413bc6a2c3720fc3b7f38feb WatchSource:0}: Error finding container a6bf9d46a3df66d798560e3c8a12164e06122940413bc6a2c3720fc3b7f38feb: Status 404 returned error can't find the container with id a6bf9d46a3df66d798560e3c8a12164e06122940413bc6a2c3720fc3b7f38feb
Sep 16 18:54:26.237034 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.237010    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq\" (UID: \"f3a3a43a-76bc-4511-8c2e-db0346cbbaef\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq"
Sep 16 18:54:26.237128 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.237042    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9c79e632-ba98-4a09-ad37-0992ed0e90f1-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-4pzzs\" (UID: \"9c79e632-ba98-4a09-ad37-0992ed0e90f1\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs"
Sep 16 18:54:26.237128 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.237071    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq\" (UID: \"f3a3a43a-76bc-4511-8c2e-db0346cbbaef\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq"
Sep 16 18:54:26.237128 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.237090    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c79e632-ba98-4a09-ad37-0992ed0e90f1-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-4pzzs\" (UID: \"9c79e632-ba98-4a09-ad37-0992ed0e90f1\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs"
Sep 16 18:54:26.237128 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.237114    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9285x\" (UniqueName: \"kubernetes.io/projected/9c79e632-ba98-4a09-ad37-0992ed0e90f1-kube-api-access-9285x\") pod \"isvc-xgboost-graph-predictor-555fc69d56-4pzzs\" (UID: \"9c79e632-ba98-4a09-ad37-0992ed0e90f1\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs"
Sep 16 18:54:26.237322 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.237142    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq\" (UID: \"f3a3a43a-76bc-4511-8c2e-db0346cbbaef\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq"
Sep 16 18:54:26.237322 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.237186    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9c79e632-ba98-4a09-ad37-0992ed0e90f1-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-4pzzs\" (UID: \"9c79e632-ba98-4a09-ad37-0992ed0e90f1\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs"
Sep 16 18:54:26.237322 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.237257    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9c79e632-ba98-4a09-ad37-0992ed0e90f1-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-4pzzs\" (UID: \"9c79e632-ba98-4a09-ad37-0992ed0e90f1\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs"
Sep 16 18:54:26.237322 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.237285    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq\" (UID: \"f3a3a43a-76bc-4511-8c2e-db0346cbbaef\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq"
Sep 16 18:54:26.237518 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.237332    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4tb7\" (UniqueName: \"kubernetes.io/projected/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-kube-api-access-h4tb7\") pod \"isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq\" (UID: \"f3a3a43a-76bc-4511-8c2e-db0346cbbaef\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq"
Sep 16 18:54:26.237640 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.237615    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9c79e632-ba98-4a09-ad37-0992ed0e90f1-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-4pzzs\" (UID: \"9c79e632-ba98-4a09-ad37-0992ed0e90f1\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs"
Sep 16 18:54:26.238039 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.238017    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9c79e632-ba98-4a09-ad37-0992ed0e90f1-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-4pzzs\" (UID: \"9c79e632-ba98-4a09-ad37-0992ed0e90f1\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs"
Sep 16 18:54:26.238120 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.238017    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9c79e632-ba98-4a09-ad37-0992ed0e90f1-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-4pzzs\" (UID: \"9c79e632-ba98-4a09-ad37-0992ed0e90f1\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs"
Sep 16 18:54:26.239600 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.239577    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c79e632-ba98-4a09-ad37-0992ed0e90f1-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-4pzzs\" (UID: \"9c79e632-ba98-4a09-ad37-0992ed0e90f1\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs"
Sep 16 18:54:26.245149 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.245128    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9285x\" (UniqueName: \"kubernetes.io/projected/9c79e632-ba98-4a09-ad37-0992ed0e90f1-kube-api-access-9285x\") pod \"isvc-xgboost-graph-predictor-555fc69d56-4pzzs\" (UID: \"9c79e632-ba98-4a09-ad37-0992ed0e90f1\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs"
Sep 16 18:54:26.337931 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.337893    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq\" (UID: \"f3a3a43a-76bc-4511-8c2e-db0346cbbaef\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq"
Sep 16 18:54:26.338165 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.337946    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4tb7\" (UniqueName: \"kubernetes.io/projected/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-kube-api-access-h4tb7\") pod \"isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq\" (UID: \"f3a3a43a-76bc-4511-8c2e-db0346cbbaef\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq"
Sep 16 18:54:26.338165 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.337979    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq\" (UID: \"f3a3a43a-76bc-4511-8c2e-db0346cbbaef\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq"
Sep 16 18:54:26.338165 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.338005    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq\" (UID: \"f3a3a43a-76bc-4511-8c2e-db0346cbbaef\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq"
Sep 16 18:54:26.338165 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.338027    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq\" (UID: \"f3a3a43a-76bc-4511-8c2e-db0346cbbaef\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq"
Sep 16 18:54:26.338383 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.338337    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq\" (UID: \"f3a3a43a-76bc-4511-8c2e-db0346cbbaef\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq"
Sep 16 18:54:26.338649 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.338626    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq\" (UID: \"f3a3a43a-76bc-4511-8c2e-db0346cbbaef\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq"
Sep 16 18:54:26.338747 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.338724    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq\" (UID: \"f3a3a43a-76bc-4511-8c2e-db0346cbbaef\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq"
Sep 16 18:54:26.340595 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.340569    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq\" (UID: \"f3a3a43a-76bc-4511-8c2e-db0346cbbaef\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq"
Sep 16 18:54:26.346890 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.346872    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4tb7\" (UniqueName: \"kubernetes.io/projected/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-kube-api-access-h4tb7\") pod \"isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq\" (UID: \"f3a3a43a-76bc-4511-8c2e-db0346cbbaef\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq"
Sep 16 18:54:26.370008 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.369989    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm"]
Sep 16 18:54:26.372419 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:54:26.372397    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31051676_c5cc_45f1_9a97_41e8e2eb8687.slice/crio-013668fd44195703a91769f2121e6468413715b6dff5088caba854c8b1424364 WatchSource:0}: Error finding container 013668fd44195703a91769f2121e6468413715b6dff5088caba854c8b1424364: Status 404 returned error can't find the container with id 013668fd44195703a91769f2121e6468413715b6dff5088caba854c8b1424364
Sep 16 18:54:26.407338 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.407304    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs"
Sep 16 18:54:26.508619 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.508594    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq"
Sep 16 18:54:26.682157 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.682125    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs"]
Sep 16 18:54:26.684120 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:54:26.684091    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c79e632_ba98_4a09_ad37_0992ed0e90f1.slice/crio-2fa3c40bceab79bf3f938666c95fc19bb763a66e61c6bfe4668887551d53c671 WatchSource:0}: Error finding container 2fa3c40bceab79bf3f938666c95fc19bb763a66e61c6bfe4668887551d53c671: Status 404 returned error can't find the container with id 2fa3c40bceab79bf3f938666c95fc19bb763a66e61c6bfe4668887551d53c671
Sep 16 18:54:26.787291 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.787263    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq"]
Sep 16 18:54:26.790567 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:54:26.790540    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3a3a43a_76bc_4511_8c2e_db0346cbbaef.slice/crio-f75cff4f3ba2f683520a0578a18f2dfcb67ca26c58271f4e06d45db9195846c7 WatchSource:0}: Error finding container f75cff4f3ba2f683520a0578a18f2dfcb67ca26c58271f4e06d45db9195846c7: Status 404 returned error can't find the container with id f75cff4f3ba2f683520a0578a18f2dfcb67ca26c58271f4e06d45db9195846c7
Sep 16 18:54:26.948902 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.948865    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq" event={"ID":"f3a3a43a-76bc-4511-8c2e-db0346cbbaef","Type":"ContainerStarted","Data":"46ea8c6ea25a4bbc3ec7f8ebd3b65b83899d82e823411bab541647cae9c4659e"}
Sep 16 18:54:26.949089 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.948938    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq" event={"ID":"f3a3a43a-76bc-4511-8c2e-db0346cbbaef","Type":"ContainerStarted","Data":"f75cff4f3ba2f683520a0578a18f2dfcb67ca26c58271f4e06d45db9195846c7"}
Sep 16 18:54:26.950666 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.950635    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq" event={"ID":"311b9459-414f-4afe-b06a-3e0e3f001570","Type":"ContainerStarted","Data":"4874aad570dba80727a363e693c357139a0d4c8b0dc43cdc4cba2220f8975f83"}
Sep 16 18:54:26.950796 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.950691    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq" event={"ID":"311b9459-414f-4afe-b06a-3e0e3f001570","Type":"ContainerStarted","Data":"a6bf9d46a3df66d798560e3c8a12164e06122940413bc6a2c3720fc3b7f38feb"}
Sep 16 18:54:26.952453 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.952411    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs" event={"ID":"9c79e632-ba98-4a09-ad37-0992ed0e90f1","Type":"ContainerStarted","Data":"858c41f1dd9d1d83940cf348886fe102021d1cc9612dcdd912e1768ba4464889"}
Sep 16 18:54:26.952560 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.952470    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs" event={"ID":"9c79e632-ba98-4a09-ad37-0992ed0e90f1","Type":"ContainerStarted","Data":"2fa3c40bceab79bf3f938666c95fc19bb763a66e61c6bfe4668887551d53c671"}
Sep 16 18:54:26.953680 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:26.953662    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm" event={"ID":"31051676-c5cc-45f1-9a97-41e8e2eb8687","Type":"ContainerStarted","Data":"013668fd44195703a91769f2121e6468413715b6dff5088caba854c8b1424364"}
Sep 16 18:54:31.985733 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:31.985694    2557 generic.go:356] "Generic (PLEG): container finished" podID="f3a3a43a-76bc-4511-8c2e-db0346cbbaef" containerID="46ea8c6ea25a4bbc3ec7f8ebd3b65b83899d82e823411bab541647cae9c4659e" exitCode=0
Sep 16 18:54:31.986192 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:31.985780    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq" event={"ID":"f3a3a43a-76bc-4511-8c2e-db0346cbbaef","Type":"ContainerDied","Data":"46ea8c6ea25a4bbc3ec7f8ebd3b65b83899d82e823411bab541647cae9c4659e"}
Sep 16 18:54:31.987835 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:31.987807    2557 generic.go:356] "Generic (PLEG): container finished" podID="311b9459-414f-4afe-b06a-3e0e3f001570" containerID="4874aad570dba80727a363e693c357139a0d4c8b0dc43cdc4cba2220f8975f83" exitCode=0
Sep 16 18:54:31.987970 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:31.987903    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq" event={"ID":"311b9459-414f-4afe-b06a-3e0e3f001570","Type":"ContainerDied","Data":"4874aad570dba80727a363e693c357139a0d4c8b0dc43cdc4cba2220f8975f83"}
Sep 16 18:54:31.990683 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:31.990588    2557 generic.go:356] "Generic (PLEG): container finished" podID="9c79e632-ba98-4a09-ad37-0992ed0e90f1" containerID="858c41f1dd9d1d83940cf348886fe102021d1cc9612dcdd912e1768ba4464889" exitCode=0
Sep 16 18:54:31.990683 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:31.990658    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs" event={"ID":"9c79e632-ba98-4a09-ad37-0992ed0e90f1","Type":"ContainerDied","Data":"858c41f1dd9d1d83940cf348886fe102021d1cc9612dcdd912e1768ba4464889"}
Sep 16 18:54:43.065218 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:43.065124    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq" event={"ID":"f3a3a43a-76bc-4511-8c2e-db0346cbbaef","Type":"ContainerStarted","Data":"bc420eabde412becfe8fe25a7e15f542eaa6a4c9c5d6bcee0fbb7e7220918a63"}
Sep 16 18:54:43.069356 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:43.069274    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq" event={"ID":"311b9459-414f-4afe-b06a-3e0e3f001570","Type":"ContainerStarted","Data":"ad9acd1562bf66f3a461110b4411b025f9b5786a9624aa850508b042fd8cb502"}
Sep 16 18:54:43.071453 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:43.071407    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm" event={"ID":"31051676-c5cc-45f1-9a97-41e8e2eb8687","Type":"ContainerStarted","Data":"f7200c39c4188bd884171cfae5fa3df79cf830bbd8e0d57f01db839b127b2756"}
Sep 16 18:54:48.110029 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:48.109913    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq" event={"ID":"f3a3a43a-76bc-4511-8c2e-db0346cbbaef","Type":"ContainerStarted","Data":"bc1c7a81c9dac1adbe373fc14bb9c87faeb644d7b54dae889f4ffd0958528c0b"}
Sep 16 18:54:48.110528 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:48.110042    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq"
Sep 16 18:54:48.112665 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:48.112354    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq" event={"ID":"311b9459-414f-4afe-b06a-3e0e3f001570","Type":"ContainerStarted","Data":"08147d628d30f2bab4b4be98d6ad6cc36142e390aabd05fba62042632ea5d20f"}
Sep 16 18:54:48.112665 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:48.112416    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq"
Sep 16 18:54:48.112665 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:48.112448    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq"
Sep 16 18:54:48.113579 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:48.113554    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq" podUID="311b9459-414f-4afe-b06a-3e0e3f001570" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Sep 16 18:54:48.114887 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:48.114821    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm" event={"ID":"31051676-c5cc-45f1-9a97-41e8e2eb8687","Type":"ContainerStarted","Data":"404b8a209d788014bed8be29f3fee22d24aab62a541918474b586755a37a4fc0"}
Sep 16 18:54:48.115174 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:48.115151    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm"
Sep 16 18:54:48.115273 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:48.115207    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm"
Sep 16 18:54:48.116259 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:48.116235    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm" podUID="31051676-c5cc-45f1-9a97-41e8e2eb8687" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Sep 16 18:54:48.132815 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:48.132676    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq" podStartSLOduration=3.014399227 podStartE2EDuration="22.132658217s" totalImagesPullingTime="14.963305133s" totalInitContainerRuntime="4.154953857s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:54:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:54:48.132100397 +0000 UTC m=+602.209205269" watchObservedRunningTime="2026-09-16 18:54:48.132658217 +0000 UTC m=+602.209763079"
Sep 16 18:54:48.186267 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:48.186210    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq" podStartSLOduration=4.209872223 podStartE2EDuration="23.186194763s" totalImagesPullingTime="15.004791771s" totalInitContainerRuntime="3.971530769s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:54:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:54:48.163890757 +0000 UTC m=+602.240995617" watchObservedRunningTime="2026-09-16 18:54:48.186194763 +0000 UTC m=+602.263299623"
Sep 16 18:54:48.186427 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:48.186337    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm" podStartSLOduration=2.585101236 podStartE2EDuration="23.186329192s" totalImagesPullingTime="20.601227956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:54:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:54:48.184607982 +0000 UTC m=+602.261712842" watchObservedRunningTime="2026-09-16 18:54:48.186329192 +0000 UTC m=+602.263434052"
Sep 16 18:54:49.120415 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:49.119948    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq" podUID="311b9459-414f-4afe-b06a-3e0e3f001570" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Sep 16 18:54:49.120415 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:49.120356    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm" podUID="31051676-c5cc-45f1-9a97-41e8e2eb8687" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Sep 16 18:54:49.120894 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:49.120581    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq"
Sep 16 18:54:49.121177 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:49.121148    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq" podUID="f3a3a43a-76bc-4511-8c2e-db0346cbbaef" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Sep 16 18:54:50.124172 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:50.124133    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq" podUID="f3a3a43a-76bc-4511-8c2e-db0346cbbaef" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Sep 16 18:54:54.124271 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:54.124234    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq"
Sep 16 18:54:54.124746 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:54.124715    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm"
Sep 16 18:54:54.124859 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:54.124761    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq" podUID="311b9459-414f-4afe-b06a-3e0e3f001570" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Sep 16 18:54:54.125216 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:54.125186    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm" podUID="31051676-c5cc-45f1-9a97-41e8e2eb8687" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Sep 16 18:54:55.129156 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:55.129128    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq"
Sep 16 18:54:55.129663 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:54:55.129631    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq" podUID="f3a3a43a-76bc-4511-8c2e-db0346cbbaef" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Sep 16 18:55:03.188427 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:03.188390    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs" event={"ID":"9c79e632-ba98-4a09-ad37-0992ed0e90f1","Type":"ContainerStarted","Data":"2e76ab5918d6678c534446e383e981ee6dd6ff9cf0069545ed3c1ec13ddc9e58"}
Sep 16 18:55:03.188815 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:03.188457    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs" event={"ID":"9c79e632-ba98-4a09-ad37-0992ed0e90f1","Type":"ContainerStarted","Data":"b48ac351afd90805d4ff60d41b84db76ab3d1d6ad9be553084423c5a31908174"}
Sep 16 18:55:03.188815 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:03.188646    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs"
Sep 16 18:55:03.209258 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:03.209195    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs" podStartSLOduration=2.837822407 podStartE2EDuration="37.209175674s" totalImagesPullingTime="30.489252533s" totalInitContainerRuntime="3.882100734s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:54:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:55:03.207580665 +0000 UTC m=+617.284685524" watchObservedRunningTime="2026-09-16 18:55:03.209175674 +0000 UTC m=+617.286280535"
Sep 16 18:55:04.125395 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:04.125353    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq" podUID="311b9459-414f-4afe-b06a-3e0e3f001570" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Sep 16 18:55:04.125566 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:04.125353    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm" podUID="31051676-c5cc-45f1-9a97-41e8e2eb8687" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Sep 16 18:55:04.192623 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:04.192585    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs"
Sep 16 18:55:04.193621 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:04.193599    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs" podUID="9c79e632-ba98-4a09-ad37-0992ed0e90f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Sep 16 18:55:05.129725 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:05.129688    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq" podUID="f3a3a43a-76bc-4511-8c2e-db0346cbbaef" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Sep 16 18:55:05.196514 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:05.196470    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs" podUID="9c79e632-ba98-4a09-ad37-0992ed0e90f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Sep 16 18:55:10.200985 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:10.200911    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs"
Sep 16 18:55:10.201364 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:10.201337    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs" podUID="9c79e632-ba98-4a09-ad37-0992ed0e90f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Sep 16 18:55:14.125046 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:14.125003    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq" podUID="311b9459-414f-4afe-b06a-3e0e3f001570" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Sep 16 18:55:14.125521 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:14.125150    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm" podUID="31051676-c5cc-45f1-9a97-41e8e2eb8687" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Sep 16 18:55:15.129864 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:15.129823    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq" podUID="f3a3a43a-76bc-4511-8c2e-db0346cbbaef" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Sep 16 18:55:20.201806 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:20.201750    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs" podUID="9c79e632-ba98-4a09-ad37-0992ed0e90f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Sep 16 18:55:24.124812 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:24.124753    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq" podUID="311b9459-414f-4afe-b06a-3e0e3f001570" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Sep 16 18:55:24.125190 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:24.125164    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm" podUID="31051676-c5cc-45f1-9a97-41e8e2eb8687" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Sep 16 18:55:25.130352 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:25.130306    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq" podUID="f3a3a43a-76bc-4511-8c2e-db0346cbbaef" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Sep 16 18:55:30.201739 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:30.201695    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs" podUID="9c79e632-ba98-4a09-ad37-0992ed0e90f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Sep 16 18:55:34.124862 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:34.124822    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq" podUID="311b9459-414f-4afe-b06a-3e0e3f001570" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Sep 16 18:55:34.125294 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:34.125165    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm" podUID="31051676-c5cc-45f1-9a97-41e8e2eb8687" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Sep 16 18:55:35.129751 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:35.129714    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq" podUID="f3a3a43a-76bc-4511-8c2e-db0346cbbaef" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Sep 16 18:55:40.202292 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:40.202249    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs" podUID="9c79e632-ba98-4a09-ad37-0992ed0e90f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Sep 16 18:55:44.124718 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:44.124681    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq" podUID="311b9459-414f-4afe-b06a-3e0e3f001570" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Sep 16 18:55:44.125677 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:44.125658    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm"
Sep 16 18:55:45.130311 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:45.130276    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq" podUID="f3a3a43a-76bc-4511-8c2e-db0346cbbaef" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Sep 16 18:55:50.202279 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:50.202237    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs" podUID="9c79e632-ba98-4a09-ad37-0992ed0e90f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Sep 16 18:55:54.124818 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:54.124756    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq" podUID="311b9459-414f-4afe-b06a-3e0e3f001570" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Sep 16 18:55:55.130472 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:55.130434    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq" podUID="f3a3a43a-76bc-4511-8c2e-db0346cbbaef" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Sep 16 18:55:56.463640 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:56.463589    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq" podUID="f3a3a43a-76bc-4511-8c2e-db0346cbbaef" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Sep 16 18:55:57.461178 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:55:57.461135    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq" podUID="311b9459-414f-4afe-b06a-3e0e3f001570" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Sep 16 18:56:00.201359 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:00.201317    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs" podUID="9c79e632-ba98-4a09-ad37-0992ed0e90f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Sep 16 18:56:05.821248 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:05.821206    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm"]
Sep 16 18:56:05.821693 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:05.821490    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm" podUID="31051676-c5cc-45f1-9a97-41e8e2eb8687" containerName="kserve-container" containerID="cri-o://f7200c39c4188bd884171cfae5fa3df79cf830bbd8e0d57f01db839b127b2756" gracePeriod=30
Sep 16 18:56:05.821693 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:05.821556    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm" podUID="31051676-c5cc-45f1-9a97-41e8e2eb8687" containerName="kube-rbac-proxy" containerID="cri-o://404b8a209d788014bed8be29f3fee22d24aab62a541918474b586755a37a4fc0" gracePeriod=30
Sep 16 18:56:05.973222 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:05.973187    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5"]
Sep 16 18:56:05.976740 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:05.976720    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5"
Sep 16 18:56:05.979361 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:05.979341    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-eb09c-predictor-serving-cert\""
Sep 16 18:56:05.979476 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:05.979383    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-eb09c-kube-rbac-proxy-sar-config\""
Sep 16 18:56:05.995955 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:05.995930    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5"]
Sep 16 18:56:06.058859 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:06.058821    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/42ce6703-df67-490f-90dc-671f7e90ce29-proxy-tls\") pod \"success-200-isvc-eb09c-predictor-555759ccc-6z7w5\" (UID: \"42ce6703-df67-490f-90dc-671f7e90ce29\") " pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5"
Sep 16 18:56:06.059022 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:06.058876    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/42ce6703-df67-490f-90dc-671f7e90ce29-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-eb09c-predictor-555759ccc-6z7w5\" (UID: \"42ce6703-df67-490f-90dc-671f7e90ce29\") " pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5"
Sep 16 18:56:06.059022 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:06.058925    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psnq9\" (UniqueName: \"kubernetes.io/projected/42ce6703-df67-490f-90dc-671f7e90ce29-kube-api-access-psnq9\") pod \"success-200-isvc-eb09c-predictor-555759ccc-6z7w5\" (UID: \"42ce6703-df67-490f-90dc-671f7e90ce29\") " pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5"
Sep 16 18:56:06.160376 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:06.160282    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/42ce6703-df67-490f-90dc-671f7e90ce29-proxy-tls\") pod \"success-200-isvc-eb09c-predictor-555759ccc-6z7w5\" (UID: \"42ce6703-df67-490f-90dc-671f7e90ce29\") " pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5"
Sep 16 18:56:06.160376 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:06.160344    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/42ce6703-df67-490f-90dc-671f7e90ce29-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-eb09c-predictor-555759ccc-6z7w5\" (UID: \"42ce6703-df67-490f-90dc-671f7e90ce29\") " pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5"
Sep 16 18:56:06.160376 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:06.160371    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psnq9\" (UniqueName: \"kubernetes.io/projected/42ce6703-df67-490f-90dc-671f7e90ce29-kube-api-access-psnq9\") pod \"success-200-isvc-eb09c-predictor-555759ccc-6z7w5\" (UID: \"42ce6703-df67-490f-90dc-671f7e90ce29\") " pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5"
Sep 16 18:56:06.161286 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:06.161253    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/42ce6703-df67-490f-90dc-671f7e90ce29-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-eb09c-predictor-555759ccc-6z7w5\" (UID: \"42ce6703-df67-490f-90dc-671f7e90ce29\") " pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5"
Sep 16 18:56:06.162832 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:06.162802    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/42ce6703-df67-490f-90dc-671f7e90ce29-proxy-tls\") pod \"success-200-isvc-eb09c-predictor-555759ccc-6z7w5\" (UID: \"42ce6703-df67-490f-90dc-671f7e90ce29\") " pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5"
Sep 16 18:56:06.170040 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:06.170018    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psnq9\" (UniqueName: \"kubernetes.io/projected/42ce6703-df67-490f-90dc-671f7e90ce29-kube-api-access-psnq9\") pod \"success-200-isvc-eb09c-predictor-555759ccc-6z7w5\" (UID: \"42ce6703-df67-490f-90dc-671f7e90ce29\") " pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5"
Sep 16 18:56:06.286680 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:06.286643    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5"
Sep 16 18:56:06.447051 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:06.446966    2557 generic.go:356] "Generic (PLEG): container finished" podID="31051676-c5cc-45f1-9a97-41e8e2eb8687" containerID="404b8a209d788014bed8be29f3fee22d24aab62a541918474b586755a37a4fc0" exitCode=2
Sep 16 18:56:06.447051 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:06.447022    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm" event={"ID":"31051676-c5cc-45f1-9a97-41e8e2eb8687","Type":"ContainerDied","Data":"404b8a209d788014bed8be29f3fee22d24aab62a541918474b586755a37a4fc0"}
Sep 16 18:56:06.466224 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:06.466198    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq"
Sep 16 18:56:06.563380 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:06.563350    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5"]
Sep 16 18:56:06.567187 ip-10-0-130-26 kubenswrapper[2557]: W0916 18:56:06.567147    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42ce6703_df67_490f_90dc_671f7e90ce29.slice/crio-ef68ceef6e964e834482a27cc5bd5800cb92d1b79ba73263eca9f09babfae95d WatchSource:0}: Error finding container ef68ceef6e964e834482a27cc5bd5800cb92d1b79ba73263eca9f09babfae95d: Status 404 returned error can't find the container with id ef68ceef6e964e834482a27cc5bd5800cb92d1b79ba73263eca9f09babfae95d
Sep 16 18:56:07.453232 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:07.453197    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5" event={"ID":"42ce6703-df67-490f-90dc-671f7e90ce29","Type":"ContainerStarted","Data":"d863f5493b098ae4600a59b0e28f6b6d8edbff7d7b238ea663aff87f083d1847"}
Sep 16 18:56:07.453703 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:07.453294    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5" event={"ID":"42ce6703-df67-490f-90dc-671f7e90ce29","Type":"ContainerStarted","Data":"5edfee8fae54fb3cdeb5e9ce75215c30fbd78211e6bc63aec14ac1c8d63a4eff"}
Sep 16 18:56:07.453703 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:07.453336    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5"
Sep 16 18:56:07.453703 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:07.453370    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5" event={"ID":"42ce6703-df67-490f-90dc-671f7e90ce29","Type":"ContainerStarted","Data":"ef68ceef6e964e834482a27cc5bd5800cb92d1b79ba73263eca9f09babfae95d"}
Sep 16 18:56:07.453703 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:07.453408    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5"
Sep 16 18:56:07.454844 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:07.454809    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5" podUID="42ce6703-df67-490f-90dc-671f7e90ce29" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Sep 16 18:56:07.461633 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:07.461612    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq"
Sep 16 18:56:07.472626 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:07.472575    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5" podStartSLOduration=2.4725626050000002 podStartE2EDuration="2.472562605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:56:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:56:07.471106091 +0000 UTC m=+681.548210953" watchObservedRunningTime="2026-09-16 18:56:07.472562605 +0000 UTC m=+681.549667465"
Sep 16 18:56:08.459809 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:08.459745    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5" podUID="42ce6703-df67-490f-90dc-671f7e90ce29" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Sep 16 18:56:08.971819 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:08.971792    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm"
Sep 16 18:56:09.084466 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:09.084432    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/31051676-c5cc-45f1-9a97-41e8e2eb8687-proxy-tls\") pod \"31051676-c5cc-45f1-9a97-41e8e2eb8687\" (UID: \"31051676-c5cc-45f1-9a97-41e8e2eb8687\") "
Sep 16 18:56:09.084642 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:09.084569    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/31051676-c5cc-45f1-9a97-41e8e2eb8687-kube-rbac-proxy-sar-config\") pod \"31051676-c5cc-45f1-9a97-41e8e2eb8687\" (UID: \"31051676-c5cc-45f1-9a97-41e8e2eb8687\") "
Sep 16 18:56:09.084642 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:09.084592    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78znf\" (UniqueName: \"kubernetes.io/projected/31051676-c5cc-45f1-9a97-41e8e2eb8687-kube-api-access-78znf\") pod \"31051676-c5cc-45f1-9a97-41e8e2eb8687\" (UID: \"31051676-c5cc-45f1-9a97-41e8e2eb8687\") "
Sep 16 18:56:09.084969 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:09.084938    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31051676-c5cc-45f1-9a97-41e8e2eb8687-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "31051676-c5cc-45f1-9a97-41e8e2eb8687" (UID: "31051676-c5cc-45f1-9a97-41e8e2eb8687"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:56:09.086692 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:09.086669    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31051676-c5cc-45f1-9a97-41e8e2eb8687-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "31051676-c5cc-45f1-9a97-41e8e2eb8687" (UID: "31051676-c5cc-45f1-9a97-41e8e2eb8687"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:56:09.086781 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:09.086725    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31051676-c5cc-45f1-9a97-41e8e2eb8687-kube-api-access-78znf" (OuterVolumeSpecName: "kube-api-access-78znf") pod "31051676-c5cc-45f1-9a97-41e8e2eb8687" (UID: "31051676-c5cc-45f1-9a97-41e8e2eb8687"). InnerVolumeSpecName "kube-api-access-78znf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:56:09.185364 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:09.185332    2557 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/31051676-c5cc-45f1-9a97-41e8e2eb8687-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:56:09.185364 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:09.185359    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78znf\" (UniqueName: \"kubernetes.io/projected/31051676-c5cc-45f1-9a97-41e8e2eb8687-kube-api-access-78znf\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:56:09.185364 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:09.185369    2557 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/31051676-c5cc-45f1-9a97-41e8e2eb8687-proxy-tls\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:56:09.465706 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:09.465633    2557 generic.go:356] "Generic (PLEG): container finished" podID="31051676-c5cc-45f1-9a97-41e8e2eb8687" containerID="f7200c39c4188bd884171cfae5fa3df79cf830bbd8e0d57f01db839b127b2756" exitCode=0
Sep 16 18:56:09.465706 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:09.465682    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm"
Sep 16 18:56:09.465706 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:09.465689    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm" event={"ID":"31051676-c5cc-45f1-9a97-41e8e2eb8687","Type":"ContainerDied","Data":"f7200c39c4188bd884171cfae5fa3df79cf830bbd8e0d57f01db839b127b2756"}
Sep 16 18:56:09.466269 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:09.465729    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm" event={"ID":"31051676-c5cc-45f1-9a97-41e8e2eb8687","Type":"ContainerDied","Data":"013668fd44195703a91769f2121e6468413715b6dff5088caba854c8b1424364"}
Sep 16 18:56:09.466269 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:09.465757    2557 scope.go:117] "RemoveContainer" containerID="404b8a209d788014bed8be29f3fee22d24aab62a541918474b586755a37a4fc0"
Sep 16 18:56:09.474755 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:09.474738    2557 scope.go:117] "RemoveContainer" containerID="f7200c39c4188bd884171cfae5fa3df79cf830bbd8e0d57f01db839b127b2756"
Sep 16 18:56:09.481973 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:09.481957    2557 scope.go:117] "RemoveContainer" containerID="404b8a209d788014bed8be29f3fee22d24aab62a541918474b586755a37a4fc0"
Sep 16 18:56:09.482221 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:56:09.482199    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"404b8a209d788014bed8be29f3fee22d24aab62a541918474b586755a37a4fc0\": container with ID starting with 404b8a209d788014bed8be29f3fee22d24aab62a541918474b586755a37a4fc0 not found: ID does not exist" containerID="404b8a209d788014bed8be29f3fee22d24aab62a541918474b586755a37a4fc0"
Sep 16 18:56:09.482320 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:09.482233    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"404b8a209d788014bed8be29f3fee22d24aab62a541918474b586755a37a4fc0"} err="failed to get container status \"404b8a209d788014bed8be29f3fee22d24aab62a541918474b586755a37a4fc0\": rpc error: code = NotFound desc = could not find container \"404b8a209d788014bed8be29f3fee22d24aab62a541918474b586755a37a4fc0\": container with ID starting with 404b8a209d788014bed8be29f3fee22d24aab62a541918474b586755a37a4fc0 not found: ID does not exist"
Sep 16 18:56:09.482320 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:09.482253    2557 scope.go:117] "RemoveContainer" containerID="f7200c39c4188bd884171cfae5fa3df79cf830bbd8e0d57f01db839b127b2756"
Sep 16 18:56:09.482477 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:56:09.482463    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7200c39c4188bd884171cfae5fa3df79cf830bbd8e0d57f01db839b127b2756\": container with ID starting with f7200c39c4188bd884171cfae5fa3df79cf830bbd8e0d57f01db839b127b2756 not found: ID does not exist" containerID="f7200c39c4188bd884171cfae5fa3df79cf830bbd8e0d57f01db839b127b2756"
Sep 16 18:56:09.482516 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:09.482481    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7200c39c4188bd884171cfae5fa3df79cf830bbd8e0d57f01db839b127b2756"} err="failed to get container status \"f7200c39c4188bd884171cfae5fa3df79cf830bbd8e0d57f01db839b127b2756\": rpc error: code = NotFound desc = could not find container \"f7200c39c4188bd884171cfae5fa3df79cf830bbd8e0d57f01db839b127b2756\": container with ID starting with f7200c39c4188bd884171cfae5fa3df79cf830bbd8e0d57f01db839b127b2756 not found: ID does not exist"
Sep 16 18:56:09.488458 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:09.488436    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm"]
Sep 16 18:56:09.493835 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:09.493816    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-aa795-predictor-6676fd456f-w6vxm"]
Sep 16 18:56:10.201976 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:10.201942    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs"
Sep 16 18:56:10.470868 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:10.470780    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31051676-c5cc-45f1-9a97-41e8e2eb8687" path="/var/lib/kubelet/pods/31051676-c5cc-45f1-9a97-41e8e2eb8687/volumes"
Sep 16 18:56:13.464148 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:13.464117    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5"
Sep 16 18:56:13.464601 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:13.464573    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5" podUID="42ce6703-df67-490f-90dc-671f7e90ce29" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Sep 16 18:56:23.465198 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:23.465158    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5" podUID="42ce6703-df67-490f-90dc-671f7e90ce29" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Sep 16 18:56:33.465118 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:33.465071    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5" podUID="42ce6703-df67-490f-90dc-671f7e90ce29" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Sep 16 18:56:43.465017 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:43.464974    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5" podUID="42ce6703-df67-490f-90dc-671f7e90ce29" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Sep 16 18:56:45.763506 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:45.763469    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq"]
Sep 16 18:56:45.763886 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:45.763809    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq" podUID="f3a3a43a-76bc-4511-8c2e-db0346cbbaef" containerName="kserve-container" containerID="cri-o://bc420eabde412becfe8fe25a7e15f542eaa6a4c9c5d6bcee0fbb7e7220918a63" gracePeriod=30
Sep 16 18:56:45.763945 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:45.763847    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq" podUID="f3a3a43a-76bc-4511-8c2e-db0346cbbaef" containerName="kube-rbac-proxy" containerID="cri-o://bc1c7a81c9dac1adbe373fc14bb9c87faeb644d7b54dae889f4ffd0958528c0b" gracePeriod=30
Sep 16 18:56:45.809662 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:45.809634    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq"]
Sep 16 18:56:45.809944 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:45.809899    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq" podUID="311b9459-414f-4afe-b06a-3e0e3f001570" containerName="kserve-container" containerID="cri-o://ad9acd1562bf66f3a461110b4411b025f9b5786a9624aa850508b042fd8cb502" gracePeriod=30
Sep 16 18:56:45.810012 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:45.809939    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq" podUID="311b9459-414f-4afe-b06a-3e0e3f001570" containerName="kube-rbac-proxy" containerID="cri-o://08147d628d30f2bab4b4be98d6ad6cc36142e390aabd05fba62042632ea5d20f" gracePeriod=30
Sep 16 18:56:45.852252 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:45.852225    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs"]
Sep 16 18:56:45.852469 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:45.852445    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs" podUID="9c79e632-ba98-4a09-ad37-0992ed0e90f1" containerName="kserve-container" containerID="cri-o://b48ac351afd90805d4ff60d41b84db76ab3d1d6ad9be553084423c5a31908174" gracePeriod=30
Sep 16 18:56:45.852531 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:45.852480    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs" podUID="9c79e632-ba98-4a09-ad37-0992ed0e90f1" containerName="kube-rbac-proxy" containerID="cri-o://2e76ab5918d6678c534446e383e981ee6dd6ff9cf0069545ed3c1ec13ddc9e58" gracePeriod=30
Sep 16 18:56:45.935791 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:45.935740    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx"]
Sep 16 18:56:45.936123 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:45.936108    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31051676-c5cc-45f1-9a97-41e8e2eb8687" containerName="kube-rbac-proxy"
Sep 16 18:56:45.936171 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:45.936124    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="31051676-c5cc-45f1-9a97-41e8e2eb8687" containerName="kube-rbac-proxy"
Sep 16 18:56:45.936171 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:45.936141    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31051676-c5cc-45f1-9a97-41e8e2eb8687" containerName="kserve-container"
Sep 16 18:56:45.936171 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:45.936147    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="31051676-c5cc-45f1-9a97-41e8e2eb8687" containerName="kserve-container"
Sep 16 18:56:45.936265 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:45.936202    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="31051676-c5cc-45f1-9a97-41e8e2eb8687" containerName="kserve-container"
Sep 16 18:56:45.936265 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:45.936208    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="31051676-c5cc-45f1-9a97-41e8e2eb8687" containerName="kube-rbac-proxy"
Sep 16 18:56:45.939302 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:45.939285    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx"
Sep 16 18:56:45.941884 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:45.941860    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-2796f-kube-rbac-proxy-sar-config\""
Sep 16 18:56:45.942132 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:45.942111    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-2796f-predictor-serving-cert\""
Sep 16 18:56:45.954857 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:45.954743    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx"]
Sep 16 18:56:46.000194 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:46.000164    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbt4s\" (UniqueName: \"kubernetes.io/projected/b64e0069-f0b8-4e72-a7b2-caf708191082-kube-api-access-nbt4s\") pod \"success-200-isvc-2796f-predictor-b7d84f855-8qlkx\" (UID: \"b64e0069-f0b8-4e72-a7b2-caf708191082\") " pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx"
Sep 16 18:56:46.000346 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:46.000240    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b64e0069-f0b8-4e72-a7b2-caf708191082-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2796f-predictor-b7d84f855-8qlkx\" (UID: \"b64e0069-f0b8-4e72-a7b2-caf708191082\") " pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx"
Sep 16 18:56:46.000346 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:46.000277    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b64e0069-f0b8-4e72-a7b2-caf708191082-proxy-tls\") pod \"success-200-isvc-2796f-predictor-b7d84f855-8qlkx\" (UID: \"b64e0069-f0b8-4e72-a7b2-caf708191082\") " pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx"
Sep 16 18:56:46.100862 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:46.100823    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b64e0069-f0b8-4e72-a7b2-caf708191082-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2796f-predictor-b7d84f855-8qlkx\" (UID: \"b64e0069-f0b8-4e72-a7b2-caf708191082\") " pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx"
Sep 16 18:56:46.101046 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:46.100881    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b64e0069-f0b8-4e72-a7b2-caf708191082-proxy-tls\") pod \"success-200-isvc-2796f-predictor-b7d84f855-8qlkx\" (UID: \"b64e0069-f0b8-4e72-a7b2-caf708191082\") " pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx"
Sep 16 18:56:46.101046 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:46.100919    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbt4s\" (UniqueName: \"kubernetes.io/projected/b64e0069-f0b8-4e72-a7b2-caf708191082-kube-api-access-nbt4s\") pod \"success-200-isvc-2796f-predictor-b7d84f855-8qlkx\" (UID: \"b64e0069-f0b8-4e72-a7b2-caf708191082\") " pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx"
Sep 16 18:56:46.101152 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:56:46.101042    2557 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-2796f-predictor-serving-cert: secret "success-200-isvc-2796f-predictor-serving-cert" not found
Sep 16 18:56:46.101152 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:56:46.101113    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b64e0069-f0b8-4e72-a7b2-caf708191082-proxy-tls podName:b64e0069-f0b8-4e72-a7b2-caf708191082 nodeName:}" failed. No retries permitted until 2026-09-16 18:56:46.601092821 +0000 UTC m=+720.678197674 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/b64e0069-f0b8-4e72-a7b2-caf708191082-proxy-tls") pod "success-200-isvc-2796f-predictor-b7d84f855-8qlkx" (UID: "b64e0069-f0b8-4e72-a7b2-caf708191082") : secret "success-200-isvc-2796f-predictor-serving-cert" not found
Sep 16 18:56:46.101477 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:46.101459    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b64e0069-f0b8-4e72-a7b2-caf708191082-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2796f-predictor-b7d84f855-8qlkx\" (UID: \"b64e0069-f0b8-4e72-a7b2-caf708191082\") " pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx"
Sep 16 18:56:46.109627 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:46.109599    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbt4s\" (UniqueName: \"kubernetes.io/projected/b64e0069-f0b8-4e72-a7b2-caf708191082-kube-api-access-nbt4s\") pod \"success-200-isvc-2796f-predictor-b7d84f855-8qlkx\" (UID: \"b64e0069-f0b8-4e72-a7b2-caf708191082\") " pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx"
Sep 16 18:56:46.463646 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:46.463596    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq" podUID="f3a3a43a-76bc-4511-8c2e-db0346cbbaef" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Sep 16 18:56:46.605544 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:46.605509    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b64e0069-f0b8-4e72-a7b2-caf708191082-proxy-tls\") pod \"success-200-isvc-2796f-predictor-b7d84f855-8qlkx\" (UID: \"b64e0069-f0b8-4e72-a7b2-caf708191082\") " pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx"
Sep 16 18:56:46.607786 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:46.607744    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b64e0069-f0b8-4e72-a7b2-caf708191082-proxy-tls\") pod \"success-200-isvc-2796f-predictor-b7d84f855-8qlkx\" (UID: \"b64e0069-f0b8-4e72-a7b2-caf708191082\") " pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx"
Sep 16 18:56:46.631075 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:46.631039    2557 generic.go:356] "Generic (PLEG): container finished" podID="9c79e632-ba98-4a09-ad37-0992ed0e90f1" containerID="2e76ab5918d6678c534446e383e981ee6dd6ff9cf0069545ed3c1ec13ddc9e58" exitCode=2
Sep 16 18:56:46.631212 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:46.631117    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs" event={"ID":"9c79e632-ba98-4a09-ad37-0992ed0e90f1","Type":"ContainerDied","Data":"2e76ab5918d6678c534446e383e981ee6dd6ff9cf0069545ed3c1ec13ddc9e58"}
Sep 16 18:56:46.633123 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:46.633100    2557 generic.go:356] "Generic (PLEG): container finished" podID="f3a3a43a-76bc-4511-8c2e-db0346cbbaef" containerID="bc1c7a81c9dac1adbe373fc14bb9c87faeb644d7b54dae889f4ffd0958528c0b" exitCode=2
Sep 16 18:56:46.633231 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:46.633171    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq" event={"ID":"f3a3a43a-76bc-4511-8c2e-db0346cbbaef","Type":"ContainerDied","Data":"bc1c7a81c9dac1adbe373fc14bb9c87faeb644d7b54dae889f4ffd0958528c0b"}
Sep 16 18:56:46.635053 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:46.635029    2557 generic.go:356] "Generic (PLEG): container finished" podID="311b9459-414f-4afe-b06a-3e0e3f001570" containerID="08147d628d30f2bab4b4be98d6ad6cc36142e390aabd05fba62042632ea5d20f" exitCode=2
Sep 16 18:56:46.635151 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:46.635099    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq" event={"ID":"311b9459-414f-4afe-b06a-3e0e3f001570","Type":"ContainerDied","Data":"08147d628d30f2bab4b4be98d6ad6cc36142e390aabd05fba62042632ea5d20f"}
Sep 16 18:56:46.849875 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:46.849845    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx"
Sep 16 18:56:47.163951 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:47.163744    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx"]
Sep 16 18:56:47.461467 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:47.461368    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq" podUID="311b9459-414f-4afe-b06a-3e0e3f001570" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Sep 16 18:56:47.641550 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:47.641509    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx" event={"ID":"b64e0069-f0b8-4e72-a7b2-caf708191082","Type":"ContainerStarted","Data":"fdb27a544a35d1daceeae99273cbf5d2538ab0fa41c164560842686162ab0489"}
Sep 16 18:56:47.641820 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:47.641569    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx" event={"ID":"b64e0069-f0b8-4e72-a7b2-caf708191082","Type":"ContainerStarted","Data":"5f5a49fbcc714827fc72bdf8f29061f1eb1a1b7d091241a7df696e6323a6c632"}
Sep 16 18:56:47.641820 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:47.641737    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx"
Sep 16 18:56:47.641820 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:47.641809    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx"
Sep 16 18:56:47.641996 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:47.641841    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx" event={"ID":"b64e0069-f0b8-4e72-a7b2-caf708191082","Type":"ContainerStarted","Data":"ee2451d8296a04cc23fad99a31c45623f6c5da7da6b78b6ab46564d649608bb9"}
Sep 16 18:56:47.642973 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:47.642945    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx" podUID="b64e0069-f0b8-4e72-a7b2-caf708191082" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Sep 16 18:56:47.682872 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:47.682818    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx" podStartSLOduration=2.682790131 podStartE2EDuration="2.682790131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:56:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:56:47.679744842 +0000 UTC m=+721.756849701" watchObservedRunningTime="2026-09-16 18:56:47.682790131 +0000 UTC m=+721.759894982"
Sep 16 18:56:48.646885 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:48.646842    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx" podUID="b64e0069-f0b8-4e72-a7b2-caf708191082" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Sep 16 18:56:49.120503 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:49.120462    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq" podUID="311b9459-414f-4afe-b06a-3e0e3f001570" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.25:8643/healthz\": dial tcp 10.134.0.25:8643: connect: connection refused"
Sep 16 18:56:49.813407 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:49.813384    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs"
Sep 16 18:56:49.836816 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:49.836792    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9c79e632-ba98-4a09-ad37-0992ed0e90f1-cabundle-cert\") pod \"9c79e632-ba98-4a09-ad37-0992ed0e90f1\" (UID: \"9c79e632-ba98-4a09-ad37-0992ed0e90f1\") "
Sep 16 18:56:49.836911 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:49.836828    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9c79e632-ba98-4a09-ad37-0992ed0e90f1-kube-rbac-proxy-sar-config\") pod \"9c79e632-ba98-4a09-ad37-0992ed0e90f1\" (UID: \"9c79e632-ba98-4a09-ad37-0992ed0e90f1\") "
Sep 16 18:56:49.836953 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:49.836934    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9285x\" (UniqueName: \"kubernetes.io/projected/9c79e632-ba98-4a09-ad37-0992ed0e90f1-kube-api-access-9285x\") pod \"9c79e632-ba98-4a09-ad37-0992ed0e90f1\" (UID: \"9c79e632-ba98-4a09-ad37-0992ed0e90f1\") "
Sep 16 18:56:49.836994 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:49.836982    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c79e632-ba98-4a09-ad37-0992ed0e90f1-proxy-tls\") pod \"9c79e632-ba98-4a09-ad37-0992ed0e90f1\" (UID: \"9c79e632-ba98-4a09-ad37-0992ed0e90f1\") "
Sep 16 18:56:49.837078 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:49.837062    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9c79e632-ba98-4a09-ad37-0992ed0e90f1-kserve-provision-location\") pod \"9c79e632-ba98-4a09-ad37-0992ed0e90f1\" (UID: \"9c79e632-ba98-4a09-ad37-0992ed0e90f1\") "
Sep 16 18:56:49.837164 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:49.837083    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c79e632-ba98-4a09-ad37-0992ed0e90f1-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "9c79e632-ba98-4a09-ad37-0992ed0e90f1" (UID: "9c79e632-ba98-4a09-ad37-0992ed0e90f1"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:56:49.837164 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:49.837144    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c79e632-ba98-4a09-ad37-0992ed0e90f1-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "9c79e632-ba98-4a09-ad37-0992ed0e90f1" (UID: "9c79e632-ba98-4a09-ad37-0992ed0e90f1"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:56:49.837426 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:49.837401    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c79e632-ba98-4a09-ad37-0992ed0e90f1-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "9c79e632-ba98-4a09-ad37-0992ed0e90f1" (UID: "9c79e632-ba98-4a09-ad37-0992ed0e90f1"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:56:49.837426 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:49.837422    2557 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9c79e632-ba98-4a09-ad37-0992ed0e90f1-cabundle-cert\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:56:49.837598 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:49.837438    2557 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9c79e632-ba98-4a09-ad37-0992ed0e90f1-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:56:49.838982 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:49.838954    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c79e632-ba98-4a09-ad37-0992ed0e90f1-kube-api-access-9285x" (OuterVolumeSpecName: "kube-api-access-9285x") pod "9c79e632-ba98-4a09-ad37-0992ed0e90f1" (UID: "9c79e632-ba98-4a09-ad37-0992ed0e90f1"). InnerVolumeSpecName "kube-api-access-9285x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:56:49.839676 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:49.839588    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c79e632-ba98-4a09-ad37-0992ed0e90f1-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "9c79e632-ba98-4a09-ad37-0992ed0e90f1" (UID: "9c79e632-ba98-4a09-ad37-0992ed0e90f1"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:56:49.938165 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:49.938092    2557 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9c79e632-ba98-4a09-ad37-0992ed0e90f1-kserve-provision-location\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:56:49.938165 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:49.938119    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9285x\" (UniqueName: \"kubernetes.io/projected/9c79e632-ba98-4a09-ad37-0992ed0e90f1-kube-api-access-9285x\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:56:49.938165 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:49.938129    2557 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c79e632-ba98-4a09-ad37-0992ed0e90f1-proxy-tls\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:56:50.124537 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:50.124490    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq" podUID="f3a3a43a-76bc-4511-8c2e-db0346cbbaef" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.28:8643/healthz\": dial tcp 10.134.0.28:8643: connect: connection refused"
Sep 16 18:56:50.657347 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:50.657306    2557 generic.go:356] "Generic (PLEG): container finished" podID="9c79e632-ba98-4a09-ad37-0992ed0e90f1" containerID="b48ac351afd90805d4ff60d41b84db76ab3d1d6ad9be553084423c5a31908174" exitCode=0
Sep 16 18:56:50.657540 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:50.657377    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs"
Sep 16 18:56:50.657540 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:50.657386    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs" event={"ID":"9c79e632-ba98-4a09-ad37-0992ed0e90f1","Type":"ContainerDied","Data":"b48ac351afd90805d4ff60d41b84db76ab3d1d6ad9be553084423c5a31908174"}
Sep 16 18:56:50.657540 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:50.657467    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs" event={"ID":"9c79e632-ba98-4a09-ad37-0992ed0e90f1","Type":"ContainerDied","Data":"2fa3c40bceab79bf3f938666c95fc19bb763a66e61c6bfe4668887551d53c671"}
Sep 16 18:56:50.657540 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:50.657506    2557 scope.go:117] "RemoveContainer" containerID="2e76ab5918d6678c534446e383e981ee6dd6ff9cf0069545ed3c1ec13ddc9e58"
Sep 16 18:56:50.666713 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:50.666693    2557 scope.go:117] "RemoveContainer" containerID="b48ac351afd90805d4ff60d41b84db76ab3d1d6ad9be553084423c5a31908174"
Sep 16 18:56:50.680365 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:50.680340    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs"]
Sep 16 18:56:50.684035 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:50.684012    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-4pzzs"]
Sep 16 18:56:50.722610 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:50.722497    2557 scope.go:117] "RemoveContainer" containerID="858c41f1dd9d1d83940cf348886fe102021d1cc9612dcdd912e1768ba4464889"
Sep 16 18:56:50.730248 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:50.730230    2557 scope.go:117] "RemoveContainer" containerID="2e76ab5918d6678c534446e383e981ee6dd6ff9cf0069545ed3c1ec13ddc9e58"
Sep 16 18:56:50.730516 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:56:50.730497    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e76ab5918d6678c534446e383e981ee6dd6ff9cf0069545ed3c1ec13ddc9e58\": container with ID starting with 2e76ab5918d6678c534446e383e981ee6dd6ff9cf0069545ed3c1ec13ddc9e58 not found: ID does not exist" containerID="2e76ab5918d6678c534446e383e981ee6dd6ff9cf0069545ed3c1ec13ddc9e58"
Sep 16 18:56:50.730553 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:50.730525    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e76ab5918d6678c534446e383e981ee6dd6ff9cf0069545ed3c1ec13ddc9e58"} err="failed to get container status \"2e76ab5918d6678c534446e383e981ee6dd6ff9cf0069545ed3c1ec13ddc9e58\": rpc error: code = NotFound desc = could not find container \"2e76ab5918d6678c534446e383e981ee6dd6ff9cf0069545ed3c1ec13ddc9e58\": container with ID starting with 2e76ab5918d6678c534446e383e981ee6dd6ff9cf0069545ed3c1ec13ddc9e58 not found: ID does not exist"
Sep 16 18:56:50.730553 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:50.730544    2557 scope.go:117] "RemoveContainer" containerID="b48ac351afd90805d4ff60d41b84db76ab3d1d6ad9be553084423c5a31908174"
Sep 16 18:56:50.730898 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:56:50.730876    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b48ac351afd90805d4ff60d41b84db76ab3d1d6ad9be553084423c5a31908174\": container with ID starting with b48ac351afd90805d4ff60d41b84db76ab3d1d6ad9be553084423c5a31908174 not found: ID does not exist" containerID="b48ac351afd90805d4ff60d41b84db76ab3d1d6ad9be553084423c5a31908174"
Sep 16 18:56:50.730963 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:50.730904    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b48ac351afd90805d4ff60d41b84db76ab3d1d6ad9be553084423c5a31908174"} err="failed to get container status \"b48ac351afd90805d4ff60d41b84db76ab3d1d6ad9be553084423c5a31908174\": rpc error: code = NotFound desc = could not find container \"b48ac351afd90805d4ff60d41b84db76ab3d1d6ad9be553084423c5a31908174\": container with ID starting with b48ac351afd90805d4ff60d41b84db76ab3d1d6ad9be553084423c5a31908174 not found: ID does not exist"
Sep 16 18:56:50.730963 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:50.730920    2557 scope.go:117] "RemoveContainer" containerID="858c41f1dd9d1d83940cf348886fe102021d1cc9612dcdd912e1768ba4464889"
Sep 16 18:56:50.731161 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:56:50.731145    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"858c41f1dd9d1d83940cf348886fe102021d1cc9612dcdd912e1768ba4464889\": container with ID starting with 858c41f1dd9d1d83940cf348886fe102021d1cc9612dcdd912e1768ba4464889 not found: ID does not exist" containerID="858c41f1dd9d1d83940cf348886fe102021d1cc9612dcdd912e1768ba4464889"
Sep 16 18:56:50.731204 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:50.731166    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"858c41f1dd9d1d83940cf348886fe102021d1cc9612dcdd912e1768ba4464889"} err="failed to get container status \"858c41f1dd9d1d83940cf348886fe102021d1cc9612dcdd912e1768ba4464889\": rpc error: code = NotFound desc = could not find container \"858c41f1dd9d1d83940cf348886fe102021d1cc9612dcdd912e1768ba4464889\": container with ID starting with 858c41f1dd9d1d83940cf348886fe102021d1cc9612dcdd912e1768ba4464889 not found: ID does not exist"
Sep 16 18:56:50.866383 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:50.866355    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq"
Sep 16 18:56:50.945941 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:50.945909    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/311b9459-414f-4afe-b06a-3e0e3f001570-proxy-tls\") pod \"311b9459-414f-4afe-b06a-3e0e3f001570\" (UID: \"311b9459-414f-4afe-b06a-3e0e3f001570\") "
Sep 16 18:56:50.946116 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:50.945948    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smhcv\" (UniqueName: \"kubernetes.io/projected/311b9459-414f-4afe-b06a-3e0e3f001570-kube-api-access-smhcv\") pod \"311b9459-414f-4afe-b06a-3e0e3f001570\" (UID: \"311b9459-414f-4afe-b06a-3e0e3f001570\") "
Sep 16 18:56:50.946116 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:50.945975    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/311b9459-414f-4afe-b06a-3e0e3f001570-cabundle-cert\") pod \"311b9459-414f-4afe-b06a-3e0e3f001570\" (UID: \"311b9459-414f-4afe-b06a-3e0e3f001570\") "
Sep 16 18:56:50.946230 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:50.946115    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/311b9459-414f-4afe-b06a-3e0e3f001570-kserve-provision-location\") pod \"311b9459-414f-4afe-b06a-3e0e3f001570\" (UID: \"311b9459-414f-4afe-b06a-3e0e3f001570\") "
Sep 16 18:56:50.946230 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:50.946178    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/311b9459-414f-4afe-b06a-3e0e3f001570-kube-rbac-proxy-sar-config\") pod \"311b9459-414f-4afe-b06a-3e0e3f001570\" (UID: \"311b9459-414f-4afe-b06a-3e0e3f001570\") "
Sep 16 18:56:50.946332 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:50.946241    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/311b9459-414f-4afe-b06a-3e0e3f001570-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "311b9459-414f-4afe-b06a-3e0e3f001570" (UID: "311b9459-414f-4afe-b06a-3e0e3f001570"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:56:50.946453 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:50.946427    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/311b9459-414f-4afe-b06a-3e0e3f001570-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "311b9459-414f-4afe-b06a-3e0e3f001570" (UID: "311b9459-414f-4afe-b06a-3e0e3f001570"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:56:50.946506 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:50.946454    2557 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/311b9459-414f-4afe-b06a-3e0e3f001570-cabundle-cert\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:56:50.946506 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:50.946496    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/311b9459-414f-4afe-b06a-3e0e3f001570-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "311b9459-414f-4afe-b06a-3e0e3f001570" (UID: "311b9459-414f-4afe-b06a-3e0e3f001570"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:56:50.948100 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:50.948082    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/311b9459-414f-4afe-b06a-3e0e3f001570-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "311b9459-414f-4afe-b06a-3e0e3f001570" (UID: "311b9459-414f-4afe-b06a-3e0e3f001570"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:56:50.948181 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:50.948155    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/311b9459-414f-4afe-b06a-3e0e3f001570-kube-api-access-smhcv" (OuterVolumeSpecName: "kube-api-access-smhcv") pod "311b9459-414f-4afe-b06a-3e0e3f001570" (UID: "311b9459-414f-4afe-b06a-3e0e3f001570"). InnerVolumeSpecName "kube-api-access-smhcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:56:51.047094 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.047059    2557 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/311b9459-414f-4afe-b06a-3e0e3f001570-proxy-tls\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:56:51.047094 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.047090    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smhcv\" (UniqueName: \"kubernetes.io/projected/311b9459-414f-4afe-b06a-3e0e3f001570-kube-api-access-smhcv\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:56:51.047094 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.047100    2557 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/311b9459-414f-4afe-b06a-3e0e3f001570-kserve-provision-location\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:56:51.047314 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.047108    2557 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/311b9459-414f-4afe-b06a-3e0e3f001570-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:56:51.499057 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.499035    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq"
Sep 16 18:56:51.551151 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.551118    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-proxy-tls\") pod \"f3a3a43a-76bc-4511-8c2e-db0346cbbaef\" (UID: \"f3a3a43a-76bc-4511-8c2e-db0346cbbaef\") "
Sep 16 18:56:51.551151 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.551154    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-kube-rbac-proxy-sar-config\") pod \"f3a3a43a-76bc-4511-8c2e-db0346cbbaef\" (UID: \"f3a3a43a-76bc-4511-8c2e-db0346cbbaef\") "
Sep 16 18:56:51.551324 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.551214    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-kserve-provision-location\") pod \"f3a3a43a-76bc-4511-8c2e-db0346cbbaef\" (UID: \"f3a3a43a-76bc-4511-8c2e-db0346cbbaef\") "
Sep 16 18:56:51.551324 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.551262    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-cabundle-cert\") pod \"f3a3a43a-76bc-4511-8c2e-db0346cbbaef\" (UID: \"f3a3a43a-76bc-4511-8c2e-db0346cbbaef\") "
Sep 16 18:56:51.551324 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.551306    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h4tb7\" (UniqueName: \"kubernetes.io/projected/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-kube-api-access-h4tb7\") pod \"f3a3a43a-76bc-4511-8c2e-db0346cbbaef\" (UID: \"f3a3a43a-76bc-4511-8c2e-db0346cbbaef\") "
Sep 16 18:56:51.551568 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.551530    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "f3a3a43a-76bc-4511-8c2e-db0346cbbaef" (UID: "f3a3a43a-76bc-4511-8c2e-db0346cbbaef"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:56:51.551784 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.551573    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "f3a3a43a-76bc-4511-8c2e-db0346cbbaef" (UID: "f3a3a43a-76bc-4511-8c2e-db0346cbbaef"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:56:51.551784 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.551616    2557 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-kserve-provision-location\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:56:51.551784 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.551668    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "f3a3a43a-76bc-4511-8c2e-db0346cbbaef" (UID: "f3a3a43a-76bc-4511-8c2e-db0346cbbaef"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:56:51.553304 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.553287    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "f3a3a43a-76bc-4511-8c2e-db0346cbbaef" (UID: "f3a3a43a-76bc-4511-8c2e-db0346cbbaef"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:56:51.553375 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.553357    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-kube-api-access-h4tb7" (OuterVolumeSpecName: "kube-api-access-h4tb7") pod "f3a3a43a-76bc-4511-8c2e-db0346cbbaef" (UID: "f3a3a43a-76bc-4511-8c2e-db0346cbbaef"). InnerVolumeSpecName "kube-api-access-h4tb7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:56:51.652126 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.652050    2557 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-cabundle-cert\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:56:51.652126 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.652086    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4tb7\" (UniqueName: \"kubernetes.io/projected/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-kube-api-access-h4tb7\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:56:51.652126 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.652096    2557 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-proxy-tls\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:56:51.652126 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.652106    2557 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f3a3a43a-76bc-4511-8c2e-db0346cbbaef-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 18:56:51.664808 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.664758    2557 generic.go:356] "Generic (PLEG): container finished" podID="f3a3a43a-76bc-4511-8c2e-db0346cbbaef" containerID="bc420eabde412becfe8fe25a7e15f542eaa6a4c9c5d6bcee0fbb7e7220918a63" exitCode=0
Sep 16 18:56:51.664930 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.664830    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq"
Sep 16 18:56:51.664930 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.664848    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq" event={"ID":"f3a3a43a-76bc-4511-8c2e-db0346cbbaef","Type":"ContainerDied","Data":"bc420eabde412becfe8fe25a7e15f542eaa6a4c9c5d6bcee0fbb7e7220918a63"}
Sep 16 18:56:51.665028 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.664928    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq" event={"ID":"f3a3a43a-76bc-4511-8c2e-db0346cbbaef","Type":"ContainerDied","Data":"f75cff4f3ba2f683520a0578a18f2dfcb67ca26c58271f4e06d45db9195846c7"}
Sep 16 18:56:51.665028 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.664969    2557 scope.go:117] "RemoveContainer" containerID="bc1c7a81c9dac1adbe373fc14bb9c87faeb644d7b54dae889f4ffd0958528c0b"
Sep 16 18:56:51.667028 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.667002    2557 generic.go:356] "Generic (PLEG): container finished" podID="311b9459-414f-4afe-b06a-3e0e3f001570" containerID="ad9acd1562bf66f3a461110b4411b025f9b5786a9624aa850508b042fd8cb502" exitCode=0
Sep 16 18:56:51.667144 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.667030    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq" event={"ID":"311b9459-414f-4afe-b06a-3e0e3f001570","Type":"ContainerDied","Data":"ad9acd1562bf66f3a461110b4411b025f9b5786a9624aa850508b042fd8cb502"}
Sep 16 18:56:51.667144 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.667061    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq"
Sep 16 18:56:51.667144 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.667085    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq" event={"ID":"311b9459-414f-4afe-b06a-3e0e3f001570","Type":"ContainerDied","Data":"a6bf9d46a3df66d798560e3c8a12164e06122940413bc6a2c3720fc3b7f38feb"}
Sep 16 18:56:51.682738 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.682722    2557 scope.go:117] "RemoveContainer" containerID="bc420eabde412becfe8fe25a7e15f542eaa6a4c9c5d6bcee0fbb7e7220918a63"
Sep 16 18:56:51.689756 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.689740    2557 scope.go:117] "RemoveContainer" containerID="46ea8c6ea25a4bbc3ec7f8ebd3b65b83899d82e823411bab541647cae9c4659e"
Sep 16 18:56:51.696659 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.696624    2557 scope.go:117] "RemoveContainer" containerID="bc1c7a81c9dac1adbe373fc14bb9c87faeb644d7b54dae889f4ffd0958528c0b"
Sep 16 18:56:51.697245 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:56:51.697218    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc1c7a81c9dac1adbe373fc14bb9c87faeb644d7b54dae889f4ffd0958528c0b\": container with ID starting with bc1c7a81c9dac1adbe373fc14bb9c87faeb644d7b54dae889f4ffd0958528c0b not found: ID does not exist" containerID="bc1c7a81c9dac1adbe373fc14bb9c87faeb644d7b54dae889f4ffd0958528c0b"
Sep 16 18:56:51.697335 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.697249    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc1c7a81c9dac1adbe373fc14bb9c87faeb644d7b54dae889f4ffd0958528c0b"} err="failed to get container status \"bc1c7a81c9dac1adbe373fc14bb9c87faeb644d7b54dae889f4ffd0958528c0b\": rpc error: code = NotFound desc = could not find container \"bc1c7a81c9dac1adbe373fc14bb9c87faeb644d7b54dae889f4ffd0958528c0b\": container with ID starting with bc1c7a81c9dac1adbe373fc14bb9c87faeb644d7b54dae889f4ffd0958528c0b not found: ID does not exist"
Sep 16 18:56:51.697335 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.697268    2557 scope.go:117] "RemoveContainer" containerID="bc420eabde412becfe8fe25a7e15f542eaa6a4c9c5d6bcee0fbb7e7220918a63"
Sep 16 18:56:51.697610 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:56:51.697589    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc420eabde412becfe8fe25a7e15f542eaa6a4c9c5d6bcee0fbb7e7220918a63\": container with ID starting with bc420eabde412becfe8fe25a7e15f542eaa6a4c9c5d6bcee0fbb7e7220918a63 not found: ID does not exist" containerID="bc420eabde412becfe8fe25a7e15f542eaa6a4c9c5d6bcee0fbb7e7220918a63"
Sep 16 18:56:51.697663 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.697616    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc420eabde412becfe8fe25a7e15f542eaa6a4c9c5d6bcee0fbb7e7220918a63"} err="failed to get container status \"bc420eabde412becfe8fe25a7e15f542eaa6a4c9c5d6bcee0fbb7e7220918a63\": rpc error: code = NotFound desc = could not find container \"bc420eabde412becfe8fe25a7e15f542eaa6a4c9c5d6bcee0fbb7e7220918a63\": container with ID starting with bc420eabde412becfe8fe25a7e15f542eaa6a4c9c5d6bcee0fbb7e7220918a63 not found: ID does not exist"
Sep 16 18:56:51.697663 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.697632    2557 scope.go:117] "RemoveContainer" containerID="46ea8c6ea25a4bbc3ec7f8ebd3b65b83899d82e823411bab541647cae9c4659e"
Sep 16 18:56:51.697893 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:56:51.697870    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46ea8c6ea25a4bbc3ec7f8ebd3b65b83899d82e823411bab541647cae9c4659e\": container with ID starting with 46ea8c6ea25a4bbc3ec7f8ebd3b65b83899d82e823411bab541647cae9c4659e not found: ID does not exist" containerID="46ea8c6ea25a4bbc3ec7f8ebd3b65b83899d82e823411bab541647cae9c4659e"
Sep 16 18:56:51.697933 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.697901    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46ea8c6ea25a4bbc3ec7f8ebd3b65b83899d82e823411bab541647cae9c4659e"} err="failed to get container status \"46ea8c6ea25a4bbc3ec7f8ebd3b65b83899d82e823411bab541647cae9c4659e\": rpc error: code = NotFound desc = could not find container \"46ea8c6ea25a4bbc3ec7f8ebd3b65b83899d82e823411bab541647cae9c4659e\": container with ID starting with 46ea8c6ea25a4bbc3ec7f8ebd3b65b83899d82e823411bab541647cae9c4659e not found: ID does not exist"
Sep 16 18:56:51.697933 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.697920    2557 scope.go:117] "RemoveContainer" containerID="08147d628d30f2bab4b4be98d6ad6cc36142e390aabd05fba62042632ea5d20f"
Sep 16 18:56:51.699358 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.699339    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq"]
Sep 16 18:56:51.703737 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.703714    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-6dfc766449-5r5nq"]
Sep 16 18:56:51.705376 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.705358    2557 scope.go:117] "RemoveContainer" containerID="ad9acd1562bf66f3a461110b4411b025f9b5786a9624aa850508b042fd8cb502"
Sep 16 18:56:51.712074 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.712060    2557 scope.go:117] "RemoveContainer" containerID="4874aad570dba80727a363e693c357139a0d4c8b0dc43cdc4cba2220f8975f83"
Sep 16 18:56:51.721063 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.718012    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq"]
Sep 16 18:56:51.723936 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.723910    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-595dc58456-t89gq"]
Sep 16 18:56:51.725565 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.725454    2557 scope.go:117] "RemoveContainer" containerID="08147d628d30f2bab4b4be98d6ad6cc36142e390aabd05fba62042632ea5d20f"
Sep 16 18:56:51.725747 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:56:51.725724    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08147d628d30f2bab4b4be98d6ad6cc36142e390aabd05fba62042632ea5d20f\": container with ID starting with 08147d628d30f2bab4b4be98d6ad6cc36142e390aabd05fba62042632ea5d20f not found: ID does not exist" containerID="08147d628d30f2bab4b4be98d6ad6cc36142e390aabd05fba62042632ea5d20f"
Sep 16 18:56:51.725802 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.725753    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08147d628d30f2bab4b4be98d6ad6cc36142e390aabd05fba62042632ea5d20f"} err="failed to get container status \"08147d628d30f2bab4b4be98d6ad6cc36142e390aabd05fba62042632ea5d20f\": rpc error: code = NotFound desc = could not find container \"08147d628d30f2bab4b4be98d6ad6cc36142e390aabd05fba62042632ea5d20f\": container with ID starting with 08147d628d30f2bab4b4be98d6ad6cc36142e390aabd05fba62042632ea5d20f not found: ID does not exist"
Sep 16 18:56:51.725802 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.725786    2557 scope.go:117] "RemoveContainer" containerID="ad9acd1562bf66f3a461110b4411b025f9b5786a9624aa850508b042fd8cb502"
Sep 16 18:56:51.726034 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:56:51.726016    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad9acd1562bf66f3a461110b4411b025f9b5786a9624aa850508b042fd8cb502\": container with ID starting with ad9acd1562bf66f3a461110b4411b025f9b5786a9624aa850508b042fd8cb502 not found: ID does not exist" containerID="ad9acd1562bf66f3a461110b4411b025f9b5786a9624aa850508b042fd8cb502"
Sep 16 18:56:51.726101 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.726042    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad9acd1562bf66f3a461110b4411b025f9b5786a9624aa850508b042fd8cb502"} err="failed to get container status \"ad9acd1562bf66f3a461110b4411b025f9b5786a9624aa850508b042fd8cb502\": rpc error: code = NotFound desc = could not find container \"ad9acd1562bf66f3a461110b4411b025f9b5786a9624aa850508b042fd8cb502\": container with ID starting with ad9acd1562bf66f3a461110b4411b025f9b5786a9624aa850508b042fd8cb502 not found: ID does not exist"
Sep 16 18:56:51.726101 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.726062    2557 scope.go:117] "RemoveContainer" containerID="4874aad570dba80727a363e693c357139a0d4c8b0dc43cdc4cba2220f8975f83"
Sep 16 18:56:51.726436 ip-10-0-130-26 kubenswrapper[2557]: E0916 18:56:51.726274    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4874aad570dba80727a363e693c357139a0d4c8b0dc43cdc4cba2220f8975f83\": container with ID starting with 4874aad570dba80727a363e693c357139a0d4c8b0dc43cdc4cba2220f8975f83 not found: ID does not exist" containerID="4874aad570dba80727a363e693c357139a0d4c8b0dc43cdc4cba2220f8975f83"
Sep 16 18:56:51.726436 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:51.726291    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4874aad570dba80727a363e693c357139a0d4c8b0dc43cdc4cba2220f8975f83"} err="failed to get container status \"4874aad570dba80727a363e693c357139a0d4c8b0dc43cdc4cba2220f8975f83\": rpc error: code = NotFound desc = could not find container \"4874aad570dba80727a363e693c357139a0d4c8b0dc43cdc4cba2220f8975f83\": container with ID starting with 4874aad570dba80727a363e693c357139a0d4c8b0dc43cdc4cba2220f8975f83 not found: ID does not exist"
Sep 16 18:56:52.466063 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:52.466030    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="311b9459-414f-4afe-b06a-3e0e3f001570" path="/var/lib/kubelet/pods/311b9459-414f-4afe-b06a-3e0e3f001570/volumes"
Sep 16 18:56:52.466519 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:52.466505    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c79e632-ba98-4a09-ad37-0992ed0e90f1" path="/var/lib/kubelet/pods/9c79e632-ba98-4a09-ad37-0992ed0e90f1/volumes"
Sep 16 18:56:52.467239 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:52.467223    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3a3a43a-76bc-4511-8c2e-db0346cbbaef" path="/var/lib/kubelet/pods/f3a3a43a-76bc-4511-8c2e-db0346cbbaef/volumes"
Sep 16 18:56:53.465634 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:53.465604    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5"
Sep 16 18:56:53.651038 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:53.651009    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx"
Sep 16 18:56:53.651479 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:56:53.651456    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx" podUID="b64e0069-f0b8-4e72-a7b2-caf708191082" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Sep 16 18:57:03.652372 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:57:03.652333    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx" podUID="b64e0069-f0b8-4e72-a7b2-caf708191082" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Sep 16 18:57:13.651724 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:57:13.651680    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx" podUID="b64e0069-f0b8-4e72-a7b2-caf708191082" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Sep 16 18:57:23.652083 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:57:23.652044    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx" podUID="b64e0069-f0b8-4e72-a7b2-caf708191082" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Sep 16 18:57:33.651909 ip-10-0-130-26 kubenswrapper[2557]: I0916 18:57:33.651882    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx"
Sep 16 19:05:20.788783 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.788744    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5"]
Sep 16 19:05:20.789290 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.788993    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5" podUID="42ce6703-df67-490f-90dc-671f7e90ce29" containerName="kserve-container" containerID="cri-o://5edfee8fae54fb3cdeb5e9ce75215c30fbd78211e6bc63aec14ac1c8d63a4eff" gracePeriod=30
Sep 16 19:05:20.789290 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.789028    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5" podUID="42ce6703-df67-490f-90dc-671f7e90ce29" containerName="kube-rbac-proxy" containerID="cri-o://d863f5493b098ae4600a59b0e28f6b6d8edbff7d7b238ea663aff87f083d1847" gracePeriod=30
Sep 16 19:05:20.889453 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.889411    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx"]
Sep 16 19:05:20.889973 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.889952    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="311b9459-414f-4afe-b06a-3e0e3f001570" containerName="kserve-container"
Sep 16 19:05:20.890077 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.889975    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="311b9459-414f-4afe-b06a-3e0e3f001570" containerName="kserve-container"
Sep 16 19:05:20.890077 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.889997    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c79e632-ba98-4a09-ad37-0992ed0e90f1" containerName="storage-initializer"
Sep 16 19:05:20.890077 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.890007    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c79e632-ba98-4a09-ad37-0992ed0e90f1" containerName="storage-initializer"
Sep 16 19:05:20.890077 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.890019    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3a3a43a-76bc-4511-8c2e-db0346cbbaef" containerName="storage-initializer"
Sep 16 19:05:20.890077 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.890027    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3a3a43a-76bc-4511-8c2e-db0346cbbaef" containerName="storage-initializer"
Sep 16 19:05:20.890077 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.890038    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="311b9459-414f-4afe-b06a-3e0e3f001570" containerName="kube-rbac-proxy"
Sep 16 19:05:20.890077 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.890047    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="311b9459-414f-4afe-b06a-3e0e3f001570" containerName="kube-rbac-proxy"
Sep 16 19:05:20.890077 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.890056    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c79e632-ba98-4a09-ad37-0992ed0e90f1" containerName="kserve-container"
Sep 16 19:05:20.890077 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.890065    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c79e632-ba98-4a09-ad37-0992ed0e90f1" containerName="kserve-container"
Sep 16 19:05:20.890077 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.890075    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c79e632-ba98-4a09-ad37-0992ed0e90f1" containerName="kube-rbac-proxy"
Sep 16 19:05:20.890431 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.890086    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c79e632-ba98-4a09-ad37-0992ed0e90f1" containerName="kube-rbac-proxy"
Sep 16 19:05:20.890431 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.890100    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="311b9459-414f-4afe-b06a-3e0e3f001570" containerName="storage-initializer"
Sep 16 19:05:20.890431 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.890108    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="311b9459-414f-4afe-b06a-3e0e3f001570" containerName="storage-initializer"
Sep 16 19:05:20.890431 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.890122    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3a3a43a-76bc-4511-8c2e-db0346cbbaef" containerName="kube-rbac-proxy"
Sep 16 19:05:20.890431 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.890130    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3a3a43a-76bc-4511-8c2e-db0346cbbaef" containerName="kube-rbac-proxy"
Sep 16 19:05:20.890431 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.890146    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3a3a43a-76bc-4511-8c2e-db0346cbbaef" containerName="kserve-container"
Sep 16 19:05:20.890431 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.890154    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3a3a43a-76bc-4511-8c2e-db0346cbbaef" containerName="kserve-container"
Sep 16 19:05:20.890431 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.890240    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3a3a43a-76bc-4511-8c2e-db0346cbbaef" containerName="kube-rbac-proxy"
Sep 16 19:05:20.890431 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.890251    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="311b9459-414f-4afe-b06a-3e0e3f001570" containerName="kserve-container"
Sep 16 19:05:20.890431 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.890259    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c79e632-ba98-4a09-ad37-0992ed0e90f1" containerName="kserve-container"
Sep 16 19:05:20.890431 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.890268    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c79e632-ba98-4a09-ad37-0992ed0e90f1" containerName="kube-rbac-proxy"
Sep 16 19:05:20.890431 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.890277    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3a3a43a-76bc-4511-8c2e-db0346cbbaef" containerName="kserve-container"
Sep 16 19:05:20.890431 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.890284    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="311b9459-414f-4afe-b06a-3e0e3f001570" containerName="kube-rbac-proxy"
Sep 16 19:05:20.893965 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.893929    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx"
Sep 16 19:05:20.899539 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.899520    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-c6de0-kube-rbac-proxy-sar-config\""
Sep 16 19:05:20.902161 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.902139    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-c6de0-predictor-serving-cert\""
Sep 16 19:05:20.913239 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.913215    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx"]
Sep 16 19:05:20.962399 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.962368    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/44f9db22-17eb-42e7-8d88-1c26d286ef85-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx\" (UID: \"44f9db22-17eb-42e7-8d88-1c26d286ef85\") " pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx"
Sep 16 19:05:20.962551 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.962428    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/44f9db22-17eb-42e7-8d88-1c26d286ef85-proxy-tls\") pod \"success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx\" (UID: \"44f9db22-17eb-42e7-8d88-1c26d286ef85\") " pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx"
Sep 16 19:05:20.962600 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:20.962540    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjcpg\" (UniqueName: \"kubernetes.io/projected/44f9db22-17eb-42e7-8d88-1c26d286ef85-kube-api-access-tjcpg\") pod \"success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx\" (UID: \"44f9db22-17eb-42e7-8d88-1c26d286ef85\") " pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx"
Sep 16 19:05:21.063338 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:21.063244    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/44f9db22-17eb-42e7-8d88-1c26d286ef85-proxy-tls\") pod \"success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx\" (UID: \"44f9db22-17eb-42e7-8d88-1c26d286ef85\") " pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx"
Sep 16 19:05:21.063338 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:21.063295    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjcpg\" (UniqueName: \"kubernetes.io/projected/44f9db22-17eb-42e7-8d88-1c26d286ef85-kube-api-access-tjcpg\") pod \"success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx\" (UID: \"44f9db22-17eb-42e7-8d88-1c26d286ef85\") " pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx"
Sep 16 19:05:21.063338 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:21.063325    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/44f9db22-17eb-42e7-8d88-1c26d286ef85-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx\" (UID: \"44f9db22-17eb-42e7-8d88-1c26d286ef85\") " pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx"
Sep 16 19:05:21.064034 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:21.064012    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/44f9db22-17eb-42e7-8d88-1c26d286ef85-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx\" (UID: \"44f9db22-17eb-42e7-8d88-1c26d286ef85\") " pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx"
Sep 16 19:05:21.065890 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:21.065863    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/44f9db22-17eb-42e7-8d88-1c26d286ef85-proxy-tls\") pod \"success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx\" (UID: \"44f9db22-17eb-42e7-8d88-1c26d286ef85\") " pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx"
Sep 16 19:05:21.084053 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:21.084025    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjcpg\" (UniqueName: \"kubernetes.io/projected/44f9db22-17eb-42e7-8d88-1c26d286ef85-kube-api-access-tjcpg\") pod \"success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx\" (UID: \"44f9db22-17eb-42e7-8d88-1c26d286ef85\") " pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx"
Sep 16 19:05:21.205213 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:21.205176    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx"
Sep 16 19:05:21.480836 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:21.480811    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx"]
Sep 16 19:05:21.482991 ip-10-0-130-26 kubenswrapper[2557]: W0916 19:05:21.482953    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44f9db22_17eb_42e7_8d88_1c26d286ef85.slice/crio-7f95f812bfac6034a8575f80b631127ac57f94e1f96ada4a00f5b798feda1725 WatchSource:0}: Error finding container 7f95f812bfac6034a8575f80b631127ac57f94e1f96ada4a00f5b798feda1725: Status 404 returned error can't find the container with id 7f95f812bfac6034a8575f80b631127ac57f94e1f96ada4a00f5b798feda1725
Sep 16 19:05:21.711605 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:21.711560    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx" event={"ID":"44f9db22-17eb-42e7-8d88-1c26d286ef85","Type":"ContainerStarted","Data":"82f4421c95a60f737e09733f4741a3af07ad133ff6e8bb6ec552fa20338a1274"}
Sep 16 19:05:21.711821 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:21.711641    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx"
Sep 16 19:05:21.711821 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:21.711677    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx"
Sep 16 19:05:21.711821 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:21.711706    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx" event={"ID":"44f9db22-17eb-42e7-8d88-1c26d286ef85","Type":"ContainerStarted","Data":"efd655ad1454a1458374b723562c002ceea03ed762b01778cb36e95a8027b66e"}
Sep 16 19:05:21.711821 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:21.711735    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx" event={"ID":"44f9db22-17eb-42e7-8d88-1c26d286ef85","Type":"ContainerStarted","Data":"7f95f812bfac6034a8575f80b631127ac57f94e1f96ada4a00f5b798feda1725"}
Sep 16 19:05:21.713442 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:21.713403    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx" podUID="44f9db22-17eb-42e7-8d88-1c26d286ef85" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Sep 16 19:05:21.713573 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:21.713546    2557 generic.go:356] "Generic (PLEG): container finished" podID="42ce6703-df67-490f-90dc-671f7e90ce29" containerID="d863f5493b098ae4600a59b0e28f6b6d8edbff7d7b238ea663aff87f083d1847" exitCode=2
Sep 16 19:05:21.713806 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:21.713597    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5" event={"ID":"42ce6703-df67-490f-90dc-671f7e90ce29","Type":"ContainerDied","Data":"d863f5493b098ae4600a59b0e28f6b6d8edbff7d7b238ea663aff87f083d1847"}
Sep 16 19:05:21.733634 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:21.733553    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx" podStartSLOduration=1.733540096 podStartE2EDuration="1.733540096s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:05:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:05:21.732041952 +0000 UTC m=+1235.809146814" watchObservedRunningTime="2026-09-16 19:05:21.733540096 +0000 UTC m=+1235.810645023"
Sep 16 19:05:22.718236 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:22.718198    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx" podUID="44f9db22-17eb-42e7-8d88-1c26d286ef85" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Sep 16 19:05:23.460886 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:23.460844    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5" podUID="42ce6703-df67-490f-90dc-671f7e90ce29" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.29:8643/healthz\": dial tcp 10.134.0.29:8643: connect: connection refused"
Sep 16 19:05:23.465376 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:23.465350    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5" podUID="42ce6703-df67-490f-90dc-671f7e90ce29" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Sep 16 19:05:24.030441 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:24.030417    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5"
Sep 16 19:05:24.088486 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:24.088456    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-psnq9\" (UniqueName: \"kubernetes.io/projected/42ce6703-df67-490f-90dc-671f7e90ce29-kube-api-access-psnq9\") pod \"42ce6703-df67-490f-90dc-671f7e90ce29\" (UID: \"42ce6703-df67-490f-90dc-671f7e90ce29\") "
Sep 16 19:05:24.088656 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:24.088504    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/42ce6703-df67-490f-90dc-671f7e90ce29-proxy-tls\") pod \"42ce6703-df67-490f-90dc-671f7e90ce29\" (UID: \"42ce6703-df67-490f-90dc-671f7e90ce29\") "
Sep 16 19:05:24.088656 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:24.088524    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/42ce6703-df67-490f-90dc-671f7e90ce29-kube-rbac-proxy-sar-config\") pod \"42ce6703-df67-490f-90dc-671f7e90ce29\" (UID: \"42ce6703-df67-490f-90dc-671f7e90ce29\") "
Sep 16 19:05:24.088992 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:24.088965    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42ce6703-df67-490f-90dc-671f7e90ce29-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "42ce6703-df67-490f-90dc-671f7e90ce29" (UID: "42ce6703-df67-490f-90dc-671f7e90ce29"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:05:24.090556 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:24.090527    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42ce6703-df67-490f-90dc-671f7e90ce29-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "42ce6703-df67-490f-90dc-671f7e90ce29" (UID: "42ce6703-df67-490f-90dc-671f7e90ce29"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:05:24.090658 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:24.090551    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42ce6703-df67-490f-90dc-671f7e90ce29-kube-api-access-psnq9" (OuterVolumeSpecName: "kube-api-access-psnq9") pod "42ce6703-df67-490f-90dc-671f7e90ce29" (UID: "42ce6703-df67-490f-90dc-671f7e90ce29"). InnerVolumeSpecName "kube-api-access-psnq9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:05:24.189123 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:24.189038    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-psnq9\" (UniqueName: \"kubernetes.io/projected/42ce6703-df67-490f-90dc-671f7e90ce29-kube-api-access-psnq9\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 19:05:24.189123 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:24.189076    2557 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/42ce6703-df67-490f-90dc-671f7e90ce29-proxy-tls\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 19:05:24.189123 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:24.189088    2557 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/42ce6703-df67-490f-90dc-671f7e90ce29-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 19:05:24.734743 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:24.734647    2557 generic.go:356] "Generic (PLEG): container finished" podID="42ce6703-df67-490f-90dc-671f7e90ce29" containerID="5edfee8fae54fb3cdeb5e9ce75215c30fbd78211e6bc63aec14ac1c8d63a4eff" exitCode=0
Sep 16 19:05:24.734743 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:24.734715    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5"
Sep 16 19:05:24.734947 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:24.734740    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5" event={"ID":"42ce6703-df67-490f-90dc-671f7e90ce29","Type":"ContainerDied","Data":"5edfee8fae54fb3cdeb5e9ce75215c30fbd78211e6bc63aec14ac1c8d63a4eff"}
Sep 16 19:05:24.734947 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:24.734855    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5" event={"ID":"42ce6703-df67-490f-90dc-671f7e90ce29","Type":"ContainerDied","Data":"ef68ceef6e964e834482a27cc5bd5800cb92d1b79ba73263eca9f09babfae95d"}
Sep 16 19:05:24.734947 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:24.734894    2557 scope.go:117] "RemoveContainer" containerID="d863f5493b098ae4600a59b0e28f6b6d8edbff7d7b238ea663aff87f083d1847"
Sep 16 19:05:24.742755 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:24.742737    2557 scope.go:117] "RemoveContainer" containerID="5edfee8fae54fb3cdeb5e9ce75215c30fbd78211e6bc63aec14ac1c8d63a4eff"
Sep 16 19:05:24.749966 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:24.749946    2557 scope.go:117] "RemoveContainer" containerID="d863f5493b098ae4600a59b0e28f6b6d8edbff7d7b238ea663aff87f083d1847"
Sep 16 19:05:24.750234 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:05:24.750216    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d863f5493b098ae4600a59b0e28f6b6d8edbff7d7b238ea663aff87f083d1847\": container with ID starting with d863f5493b098ae4600a59b0e28f6b6d8edbff7d7b238ea663aff87f083d1847 not found: ID does not exist" containerID="d863f5493b098ae4600a59b0e28f6b6d8edbff7d7b238ea663aff87f083d1847"
Sep 16 19:05:24.750283 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:24.750240    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d863f5493b098ae4600a59b0e28f6b6d8edbff7d7b238ea663aff87f083d1847"} err="failed to get container status \"d863f5493b098ae4600a59b0e28f6b6d8edbff7d7b238ea663aff87f083d1847\": rpc error: code = NotFound desc = could not find container \"d863f5493b098ae4600a59b0e28f6b6d8edbff7d7b238ea663aff87f083d1847\": container with ID starting with d863f5493b098ae4600a59b0e28f6b6d8edbff7d7b238ea663aff87f083d1847 not found: ID does not exist"
Sep 16 19:05:24.750283 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:24.750256    2557 scope.go:117] "RemoveContainer" containerID="5edfee8fae54fb3cdeb5e9ce75215c30fbd78211e6bc63aec14ac1c8d63a4eff"
Sep 16 19:05:24.750479 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:05:24.750464    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5edfee8fae54fb3cdeb5e9ce75215c30fbd78211e6bc63aec14ac1c8d63a4eff\": container with ID starting with 5edfee8fae54fb3cdeb5e9ce75215c30fbd78211e6bc63aec14ac1c8d63a4eff not found: ID does not exist" containerID="5edfee8fae54fb3cdeb5e9ce75215c30fbd78211e6bc63aec14ac1c8d63a4eff"
Sep 16 19:05:24.750565 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:24.750482    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5edfee8fae54fb3cdeb5e9ce75215c30fbd78211e6bc63aec14ac1c8d63a4eff"} err="failed to get container status \"5edfee8fae54fb3cdeb5e9ce75215c30fbd78211e6bc63aec14ac1c8d63a4eff\": rpc error: code = NotFound desc = could not find container \"5edfee8fae54fb3cdeb5e9ce75215c30fbd78211e6bc63aec14ac1c8d63a4eff\": container with ID starting with 5edfee8fae54fb3cdeb5e9ce75215c30fbd78211e6bc63aec14ac1c8d63a4eff not found: ID does not exist"
Sep 16 19:05:24.757253 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:24.757229    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5"]
Sep 16 19:05:24.762118 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:24.762093    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-eb09c-predictor-555759ccc-6z7w5"]
Sep 16 19:05:26.466786 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:26.466743    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42ce6703-df67-490f-90dc-671f7e90ce29" path="/var/lib/kubelet/pods/42ce6703-df67-490f-90dc-671f7e90ce29/volumes"
Sep 16 19:05:27.723738 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:27.723712    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx"
Sep 16 19:05:27.724227 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:27.724200    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx" podUID="44f9db22-17eb-42e7-8d88-1c26d286ef85" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Sep 16 19:05:37.724568 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:37.724524    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx" podUID="44f9db22-17eb-42e7-8d88-1c26d286ef85" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Sep 16 19:05:47.724572 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:47.724524    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx" podUID="44f9db22-17eb-42e7-8d88-1c26d286ef85" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Sep 16 19:05:57.724335 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:05:57.724287    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx" podUID="44f9db22-17eb-42e7-8d88-1c26d286ef85" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Sep 16 19:06:00.635005 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:00.634975    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx"]
Sep 16 19:06:00.635426 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:00.635195    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx" podUID="b64e0069-f0b8-4e72-a7b2-caf708191082" containerName="kserve-container" containerID="cri-o://5f5a49fbcc714827fc72bdf8f29061f1eb1a1b7d091241a7df696e6323a6c632" gracePeriod=30
Sep 16 19:06:00.635426 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:00.635227    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx" podUID="b64e0069-f0b8-4e72-a7b2-caf708191082" containerName="kube-rbac-proxy" containerID="cri-o://fdb27a544a35d1daceeae99273cbf5d2538ab0fa41c164560842686162ab0489" gracePeriod=30
Sep 16 19:06:00.752110 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:00.752080    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n"]
Sep 16 19:06:00.752444 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:00.752430    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42ce6703-df67-490f-90dc-671f7e90ce29" containerName="kube-rbac-proxy"
Sep 16 19:06:00.752444 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:00.752445    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="42ce6703-df67-490f-90dc-671f7e90ce29" containerName="kube-rbac-proxy"
Sep 16 19:06:00.752558 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:00.752462    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42ce6703-df67-490f-90dc-671f7e90ce29" containerName="kserve-container"
Sep 16 19:06:00.752558 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:00.752468    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="42ce6703-df67-490f-90dc-671f7e90ce29" containerName="kserve-container"
Sep 16 19:06:00.752558 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:00.752525    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="42ce6703-df67-490f-90dc-671f7e90ce29" containerName="kserve-container"
Sep 16 19:06:00.752558 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:00.752534    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="42ce6703-df67-490f-90dc-671f7e90ce29" containerName="kube-rbac-proxy"
Sep 16 19:06:00.755645 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:00.755624    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n"
Sep 16 19:06:00.759005 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:00.758980    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-f64b9-predictor-serving-cert\""
Sep 16 19:06:00.759141 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:00.758986    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-f64b9-kube-rbac-proxy-sar-config\""
Sep 16 19:06:00.785317 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:00.785288    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n"]
Sep 16 19:06:00.879270 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:00.879238    2557 generic.go:356] "Generic (PLEG): container finished" podID="b64e0069-f0b8-4e72-a7b2-caf708191082" containerID="fdb27a544a35d1daceeae99273cbf5d2538ab0fa41c164560842686162ab0489" exitCode=2
Sep 16 19:06:00.879426 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:00.879316    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx" event={"ID":"b64e0069-f0b8-4e72-a7b2-caf708191082","Type":"ContainerDied","Data":"fdb27a544a35d1daceeae99273cbf5d2538ab0fa41c164560842686162ab0489"}
Sep 16 19:06:00.892765 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:00.892707    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8xl7\" (UniqueName: \"kubernetes.io/projected/73c06bb6-1195-4d3a-9a58-d01702b61f73-kube-api-access-h8xl7\") pod \"success-200-isvc-f64b9-predictor-ffc69556f-g6c6n\" (UID: \"73c06bb6-1195-4d3a-9a58-d01702b61f73\") " pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n"
Sep 16 19:06:00.892765 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:00.892753    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/73c06bb6-1195-4d3a-9a58-d01702b61f73-proxy-tls\") pod \"success-200-isvc-f64b9-predictor-ffc69556f-g6c6n\" (UID: \"73c06bb6-1195-4d3a-9a58-d01702b61f73\") " pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n"
Sep 16 19:06:00.892878 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:00.892854    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/73c06bb6-1195-4d3a-9a58-d01702b61f73-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-f64b9-predictor-ffc69556f-g6c6n\" (UID: \"73c06bb6-1195-4d3a-9a58-d01702b61f73\") " pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n"
Sep 16 19:06:00.993516 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:00.993482    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8xl7\" (UniqueName: \"kubernetes.io/projected/73c06bb6-1195-4d3a-9a58-d01702b61f73-kube-api-access-h8xl7\") pod \"success-200-isvc-f64b9-predictor-ffc69556f-g6c6n\" (UID: \"73c06bb6-1195-4d3a-9a58-d01702b61f73\") " pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n"
Sep 16 19:06:00.993516 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:00.993521    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/73c06bb6-1195-4d3a-9a58-d01702b61f73-proxy-tls\") pod \"success-200-isvc-f64b9-predictor-ffc69556f-g6c6n\" (UID: \"73c06bb6-1195-4d3a-9a58-d01702b61f73\") " pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n"
Sep 16 19:06:00.993764 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:06:00.993687    2557 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-serving-cert: secret "success-200-isvc-f64b9-predictor-serving-cert" not found
Sep 16 19:06:00.993764 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:00.993731    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/73c06bb6-1195-4d3a-9a58-d01702b61f73-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-f64b9-predictor-ffc69556f-g6c6n\" (UID: \"73c06bb6-1195-4d3a-9a58-d01702b61f73\") " pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n"
Sep 16 19:06:00.993764 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:06:00.993739    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73c06bb6-1195-4d3a-9a58-d01702b61f73-proxy-tls podName:73c06bb6-1195-4d3a-9a58-d01702b61f73 nodeName:}" failed. No retries permitted until 2026-09-16 19:06:01.493720388 +0000 UTC m=+1275.570825232 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/73c06bb6-1195-4d3a-9a58-d01702b61f73-proxy-tls") pod "success-200-isvc-f64b9-predictor-ffc69556f-g6c6n" (UID: "73c06bb6-1195-4d3a-9a58-d01702b61f73") : secret "success-200-isvc-f64b9-predictor-serving-cert" not found
Sep 16 19:06:00.994482 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:00.994462    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/73c06bb6-1195-4d3a-9a58-d01702b61f73-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-f64b9-predictor-ffc69556f-g6c6n\" (UID: \"73c06bb6-1195-4d3a-9a58-d01702b61f73\") " pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n"
Sep 16 19:06:01.011805 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:01.011758    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8xl7\" (UniqueName: \"kubernetes.io/projected/73c06bb6-1195-4d3a-9a58-d01702b61f73-kube-api-access-h8xl7\") pod \"success-200-isvc-f64b9-predictor-ffc69556f-g6c6n\" (UID: \"73c06bb6-1195-4d3a-9a58-d01702b61f73\") " pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n"
Sep 16 19:06:01.498581 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:01.498540    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/73c06bb6-1195-4d3a-9a58-d01702b61f73-proxy-tls\") pod \"success-200-isvc-f64b9-predictor-ffc69556f-g6c6n\" (UID: \"73c06bb6-1195-4d3a-9a58-d01702b61f73\") " pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n"
Sep 16 19:06:01.500989 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:01.500968    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/73c06bb6-1195-4d3a-9a58-d01702b61f73-proxy-tls\") pod \"success-200-isvc-f64b9-predictor-ffc69556f-g6c6n\" (UID: \"73c06bb6-1195-4d3a-9a58-d01702b61f73\") " pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n"
Sep 16 19:06:01.666586 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:01.666537    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n"
Sep 16 19:06:01.940373 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:01.940345    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n"]
Sep 16 19:06:01.942787 ip-10-0-130-26 kubenswrapper[2557]: W0916 19:06:01.942739    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73c06bb6_1195_4d3a_9a58_d01702b61f73.slice/crio-7ebba0208869d7dadc75ca73632e85b9284f8096d1022a848a876983246df11f WatchSource:0}: Error finding container 7ebba0208869d7dadc75ca73632e85b9284f8096d1022a848a876983246df11f: Status 404 returned error can't find the container with id 7ebba0208869d7dadc75ca73632e85b9284f8096d1022a848a876983246df11f
Sep 16 19:06:02.888600 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:02.888568    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n" event={"ID":"73c06bb6-1195-4d3a-9a58-d01702b61f73","Type":"ContainerStarted","Data":"aea34a29e5e82c665e9ea876ffa974a7d385e2ceebeec2ec87eb8b27b14167e3"}
Sep 16 19:06:02.889051 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:02.888631    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n" event={"ID":"73c06bb6-1195-4d3a-9a58-d01702b61f73","Type":"ContainerStarted","Data":"00214748509d04f65669fb2f94e908c02d217eeb6a747de4e17144ce65136845"}
Sep 16 19:06:02.889051 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:02.888652    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n" event={"ID":"73c06bb6-1195-4d3a-9a58-d01702b61f73","Type":"ContainerStarted","Data":"7ebba0208869d7dadc75ca73632e85b9284f8096d1022a848a876983246df11f"}
Sep 16 19:06:02.889051 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:02.888830    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n"
Sep 16 19:06:02.907951 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:02.907904    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n" podStartSLOduration=2.907888297 podStartE2EDuration="2.907888297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:06:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:06:02.906439723 +0000 UTC m=+1276.983544595" watchObservedRunningTime="2026-09-16 19:06:02.907888297 +0000 UTC m=+1276.984993158"
Sep 16 19:06:03.647909 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:03.647868    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx" podUID="b64e0069-f0b8-4e72-a7b2-caf708191082" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.30:8643/healthz\": dial tcp 10.134.0.30:8643: connect: connection refused"
Sep 16 19:06:03.652151 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:03.652132    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx" podUID="b64e0069-f0b8-4e72-a7b2-caf708191082" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Sep 16 19:06:03.890786 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:03.890752    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx"
Sep 16 19:06:03.894543 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:03.894517    2557 generic.go:356] "Generic (PLEG): container finished" podID="b64e0069-f0b8-4e72-a7b2-caf708191082" containerID="5f5a49fbcc714827fc72bdf8f29061f1eb1a1b7d091241a7df696e6323a6c632" exitCode=0
Sep 16 19:06:03.894670 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:03.894557    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx"
Sep 16 19:06:03.894670 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:03.894594    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx" event={"ID":"b64e0069-f0b8-4e72-a7b2-caf708191082","Type":"ContainerDied","Data":"5f5a49fbcc714827fc72bdf8f29061f1eb1a1b7d091241a7df696e6323a6c632"}
Sep 16 19:06:03.894670 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:03.894652    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx" event={"ID":"b64e0069-f0b8-4e72-a7b2-caf708191082","Type":"ContainerDied","Data":"ee2451d8296a04cc23fad99a31c45623f6c5da7da6b78b6ab46564d649608bb9"}
Sep 16 19:06:03.894884 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:03.894681    2557 scope.go:117] "RemoveContainer" containerID="fdb27a544a35d1daceeae99273cbf5d2538ab0fa41c164560842686162ab0489"
Sep 16 19:06:03.894993 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:03.894964    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n"
Sep 16 19:06:03.896552 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:03.896520    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n" podUID="73c06bb6-1195-4d3a-9a58-d01702b61f73" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Sep 16 19:06:03.902413 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:03.902394    2557 scope.go:117] "RemoveContainer" containerID="5f5a49fbcc714827fc72bdf8f29061f1eb1a1b7d091241a7df696e6323a6c632"
Sep 16 19:06:03.909812 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:03.909786    2557 scope.go:117] "RemoveContainer" containerID="fdb27a544a35d1daceeae99273cbf5d2538ab0fa41c164560842686162ab0489"
Sep 16 19:06:03.910195 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:06:03.910174    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdb27a544a35d1daceeae99273cbf5d2538ab0fa41c164560842686162ab0489\": container with ID starting with fdb27a544a35d1daceeae99273cbf5d2538ab0fa41c164560842686162ab0489 not found: ID does not exist" containerID="fdb27a544a35d1daceeae99273cbf5d2538ab0fa41c164560842686162ab0489"
Sep 16 19:06:03.910284 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:03.910201    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdb27a544a35d1daceeae99273cbf5d2538ab0fa41c164560842686162ab0489"} err="failed to get container status \"fdb27a544a35d1daceeae99273cbf5d2538ab0fa41c164560842686162ab0489\": rpc error: code = NotFound desc = could not find container \"fdb27a544a35d1daceeae99273cbf5d2538ab0fa41c164560842686162ab0489\": container with ID starting with fdb27a544a35d1daceeae99273cbf5d2538ab0fa41c164560842686162ab0489 not found: ID does not exist"
Sep 16 19:06:03.910284 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:03.910217    2557 scope.go:117] "RemoveContainer" containerID="5f5a49fbcc714827fc72bdf8f29061f1eb1a1b7d091241a7df696e6323a6c632"
Sep 16 19:06:03.910508 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:06:03.910487    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f5a49fbcc714827fc72bdf8f29061f1eb1a1b7d091241a7df696e6323a6c632\": container with ID starting with 5f5a49fbcc714827fc72bdf8f29061f1eb1a1b7d091241a7df696e6323a6c632 not found: ID does not exist" containerID="5f5a49fbcc714827fc72bdf8f29061f1eb1a1b7d091241a7df696e6323a6c632"
Sep 16 19:06:03.910562 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:03.910518    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f5a49fbcc714827fc72bdf8f29061f1eb1a1b7d091241a7df696e6323a6c632"} err="failed to get container status \"5f5a49fbcc714827fc72bdf8f29061f1eb1a1b7d091241a7df696e6323a6c632\": rpc error: code = NotFound desc = could not find container \"5f5a49fbcc714827fc72bdf8f29061f1eb1a1b7d091241a7df696e6323a6c632\": container with ID starting with 5f5a49fbcc714827fc72bdf8f29061f1eb1a1b7d091241a7df696e6323a6c632 not found: ID does not exist"
Sep 16 19:06:04.019255 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:04.019223    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b64e0069-f0b8-4e72-a7b2-caf708191082-kube-rbac-proxy-sar-config\") pod \"b64e0069-f0b8-4e72-a7b2-caf708191082\" (UID: \"b64e0069-f0b8-4e72-a7b2-caf708191082\") "
Sep 16 19:06:04.019255 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:04.019260    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbt4s\" (UniqueName: \"kubernetes.io/projected/b64e0069-f0b8-4e72-a7b2-caf708191082-kube-api-access-nbt4s\") pod \"b64e0069-f0b8-4e72-a7b2-caf708191082\" (UID: \"b64e0069-f0b8-4e72-a7b2-caf708191082\") "
Sep 16 19:06:04.019473 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:04.019278    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b64e0069-f0b8-4e72-a7b2-caf708191082-proxy-tls\") pod \"b64e0069-f0b8-4e72-a7b2-caf708191082\" (UID: \"b64e0069-f0b8-4e72-a7b2-caf708191082\") "
Sep 16 19:06:04.019663 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:04.019627    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b64e0069-f0b8-4e72-a7b2-caf708191082-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "b64e0069-f0b8-4e72-a7b2-caf708191082" (UID: "b64e0069-f0b8-4e72-a7b2-caf708191082"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:06:04.021904 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:04.021883    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b64e0069-f0b8-4e72-a7b2-caf708191082-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b64e0069-f0b8-4e72-a7b2-caf708191082" (UID: "b64e0069-f0b8-4e72-a7b2-caf708191082"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:06:04.021904 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:04.021895    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b64e0069-f0b8-4e72-a7b2-caf708191082-kube-api-access-nbt4s" (OuterVolumeSpecName: "kube-api-access-nbt4s") pod "b64e0069-f0b8-4e72-a7b2-caf708191082" (UID: "b64e0069-f0b8-4e72-a7b2-caf708191082"). InnerVolumeSpecName "kube-api-access-nbt4s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:06:04.120526 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:04.120492    2557 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b64e0069-f0b8-4e72-a7b2-caf708191082-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 19:06:04.120526 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:04.120523    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbt4s\" (UniqueName: \"kubernetes.io/projected/b64e0069-f0b8-4e72-a7b2-caf708191082-kube-api-access-nbt4s\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 19:06:04.120717 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:04.120549    2557 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b64e0069-f0b8-4e72-a7b2-caf708191082-proxy-tls\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 19:06:04.217927 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:04.217893    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx"]
Sep 16 19:06:04.220641 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:04.220616    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2796f-predictor-b7d84f855-8qlkx"]
Sep 16 19:06:04.466292 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:04.466207    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b64e0069-f0b8-4e72-a7b2-caf708191082" path="/var/lib/kubelet/pods/b64e0069-f0b8-4e72-a7b2-caf708191082/volumes"
Sep 16 19:06:04.900441 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:04.900404    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n" podUID="73c06bb6-1195-4d3a-9a58-d01702b61f73" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Sep 16 19:06:07.724943 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:07.724910    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx"
Sep 16 19:06:09.904701 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:09.904671    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n"
Sep 16 19:06:09.905188 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:09.905163    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n" podUID="73c06bb6-1195-4d3a-9a58-d01702b61f73" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Sep 16 19:06:19.905674 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:19.905635    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n" podUID="73c06bb6-1195-4d3a-9a58-d01702b61f73" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Sep 16 19:06:29.905141 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:29.905097    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n" podUID="73c06bb6-1195-4d3a-9a58-d01702b61f73" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Sep 16 19:06:30.959696 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:30.959650    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx"]
Sep 16 19:06:30.960118 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:30.959972    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx" podUID="44f9db22-17eb-42e7-8d88-1c26d286ef85" containerName="kserve-container" containerID="cri-o://efd655ad1454a1458374b723562c002ceea03ed762b01778cb36e95a8027b66e" gracePeriod=30
Sep 16 19:06:30.960118 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:30.959994    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx" podUID="44f9db22-17eb-42e7-8d88-1c26d286ef85" containerName="kube-rbac-proxy" containerID="cri-o://82f4421c95a60f737e09733f4741a3af07ad133ff6e8bb6ec552fa20338a1274" gracePeriod=30
Sep 16 19:06:31.077503 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:31.077471    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5"]
Sep 16 19:06:31.077824 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:31.077810    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b64e0069-f0b8-4e72-a7b2-caf708191082" containerName="kube-rbac-proxy"
Sep 16 19:06:31.077896 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:31.077825    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b64e0069-f0b8-4e72-a7b2-caf708191082" containerName="kube-rbac-proxy"
Sep 16 19:06:31.077896 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:31.077837    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b64e0069-f0b8-4e72-a7b2-caf708191082" containerName="kserve-container"
Sep 16 19:06:31.077896 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:31.077842    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b64e0069-f0b8-4e72-a7b2-caf708191082" containerName="kserve-container"
Sep 16 19:06:31.078053 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:31.077902    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b64e0069-f0b8-4e72-a7b2-caf708191082" containerName="kserve-container"
Sep 16 19:06:31.078053 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:31.077909    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b64e0069-f0b8-4e72-a7b2-caf708191082" containerName="kube-rbac-proxy"
Sep 16 19:06:31.080997 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:31.080981    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5"
Sep 16 19:06:31.084461 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:31.084438    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-a3448-predictor-serving-cert\""
Sep 16 19:06:31.084589 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:31.084442    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-a3448-kube-rbac-proxy-sar-config\""
Sep 16 19:06:31.092843 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:31.092823    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5"]
Sep 16 19:06:31.138633 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:31.138603    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a446052c-f917-4296-8587-13e21bb06dea-proxy-tls\") pod \"success-200-isvc-a3448-predictor-779f587bc9-5zwl5\" (UID: \"a446052c-f917-4296-8587-13e21bb06dea\") " pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5"
Sep 16 19:06:31.138827 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:31.138642    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dftb4\" (UniqueName: \"kubernetes.io/projected/a446052c-f917-4296-8587-13e21bb06dea-kube-api-access-dftb4\") pod \"success-200-isvc-a3448-predictor-779f587bc9-5zwl5\" (UID: \"a446052c-f917-4296-8587-13e21bb06dea\") " pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5"
Sep 16 19:06:31.138827 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:31.138782    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a446052c-f917-4296-8587-13e21bb06dea-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-a3448-predictor-779f587bc9-5zwl5\" (UID: \"a446052c-f917-4296-8587-13e21bb06dea\") " pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5"
Sep 16 19:06:31.239615 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:31.239537    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a446052c-f917-4296-8587-13e21bb06dea-proxy-tls\") pod \"success-200-isvc-a3448-predictor-779f587bc9-5zwl5\" (UID: \"a446052c-f917-4296-8587-13e21bb06dea\") " pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5"
Sep 16 19:06:31.239615 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:31.239575    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dftb4\" (UniqueName: \"kubernetes.io/projected/a446052c-f917-4296-8587-13e21bb06dea-kube-api-access-dftb4\") pod \"success-200-isvc-a3448-predictor-779f587bc9-5zwl5\" (UID: \"a446052c-f917-4296-8587-13e21bb06dea\") " pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5"
Sep 16 19:06:31.239854 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:31.239635    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a446052c-f917-4296-8587-13e21bb06dea-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-a3448-predictor-779f587bc9-5zwl5\" (UID: \"a446052c-f917-4296-8587-13e21bb06dea\") " pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5"
Sep 16 19:06:31.240302 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:31.240283    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a446052c-f917-4296-8587-13e21bb06dea-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-a3448-predictor-779f587bc9-5zwl5\" (UID: \"a446052c-f917-4296-8587-13e21bb06dea\") " pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5"
Sep 16 19:06:31.241933 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:31.241905    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a446052c-f917-4296-8587-13e21bb06dea-proxy-tls\") pod \"success-200-isvc-a3448-predictor-779f587bc9-5zwl5\" (UID: \"a446052c-f917-4296-8587-13e21bb06dea\") " pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5"
Sep 16 19:06:31.249454 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:31.249432    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dftb4\" (UniqueName: \"kubernetes.io/projected/a446052c-f917-4296-8587-13e21bb06dea-kube-api-access-dftb4\") pod \"success-200-isvc-a3448-predictor-779f587bc9-5zwl5\" (UID: \"a446052c-f917-4296-8587-13e21bb06dea\") " pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5"
Sep 16 19:06:31.393722 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:31.393681    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5"
Sep 16 19:06:31.667278 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:31.667252    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5"]
Sep 16 19:06:31.669163 ip-10-0-130-26 kubenswrapper[2557]: W0916 19:06:31.669136    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda446052c_f917_4296_8587_13e21bb06dea.slice/crio-c9823f0511cbb7ce0f3ff574edf69d4d31fe3b16bf6b817e7469b9443e7e330e WatchSource:0}: Error finding container c9823f0511cbb7ce0f3ff574edf69d4d31fe3b16bf6b817e7469b9443e7e330e: Status 404 returned error can't find the container with id c9823f0511cbb7ce0f3ff574edf69d4d31fe3b16bf6b817e7469b9443e7e330e
Sep 16 19:06:32.021407 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:32.021311    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5" event={"ID":"a446052c-f917-4296-8587-13e21bb06dea","Type":"ContainerStarted","Data":"5893a7b9dfefe624ba5e8ff4d58fccadeb88334a2b547e9c93761f543f9caed4"}
Sep 16 19:06:32.021407 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:32.021391    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5"
Sep 16 19:06:32.021906 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:32.021427    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5"
Sep 16 19:06:32.021906 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:32.021454    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5" event={"ID":"a446052c-f917-4296-8587-13e21bb06dea","Type":"ContainerStarted","Data":"539107b73b2f25c3375132235d26642ba9fd7f2ff7426439be293a1263651529"}
Sep 16 19:06:32.021906 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:32.021485    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5" event={"ID":"a446052c-f917-4296-8587-13e21bb06dea","Type":"ContainerStarted","Data":"c9823f0511cbb7ce0f3ff574edf69d4d31fe3b16bf6b817e7469b9443e7e330e"}
Sep 16 19:06:32.023007 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:32.022977    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5" podUID="a446052c-f917-4296-8587-13e21bb06dea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Sep 16 19:06:32.023296 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:32.023270    2557 generic.go:356] "Generic (PLEG): container finished" podID="44f9db22-17eb-42e7-8d88-1c26d286ef85" containerID="82f4421c95a60f737e09733f4741a3af07ad133ff6e8bb6ec552fa20338a1274" exitCode=2
Sep 16 19:06:32.023353 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:32.023309    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx" event={"ID":"44f9db22-17eb-42e7-8d88-1c26d286ef85","Type":"ContainerDied","Data":"82f4421c95a60f737e09733f4741a3af07ad133ff6e8bb6ec552fa20338a1274"}
Sep 16 19:06:32.048988 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:32.048941    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5" podStartSLOduration=1.048924556 podStartE2EDuration="1.048924556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:06:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:06:32.046001925 +0000 UTC m=+1306.123106785" watchObservedRunningTime="2026-09-16 19:06:32.048924556 +0000 UTC m=+1306.126029415"
Sep 16 19:06:32.719213 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:32.719164    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx" podUID="44f9db22-17eb-42e7-8d88-1c26d286ef85" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.31:8643/healthz\": dial tcp 10.134.0.31:8643: connect: connection refused"
Sep 16 19:06:33.027880 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:33.027783    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5" podUID="a446052c-f917-4296-8587-13e21bb06dea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Sep 16 19:06:34.413386 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:34.413363    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx"
Sep 16 19:06:34.466549 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:34.466524    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjcpg\" (UniqueName: \"kubernetes.io/projected/44f9db22-17eb-42e7-8d88-1c26d286ef85-kube-api-access-tjcpg\") pod \"44f9db22-17eb-42e7-8d88-1c26d286ef85\" (UID: \"44f9db22-17eb-42e7-8d88-1c26d286ef85\") "
Sep 16 19:06:34.466725 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:34.466616    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/44f9db22-17eb-42e7-8d88-1c26d286ef85-proxy-tls\") pod \"44f9db22-17eb-42e7-8d88-1c26d286ef85\" (UID: \"44f9db22-17eb-42e7-8d88-1c26d286ef85\") "
Sep 16 19:06:34.466725 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:34.466675    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/44f9db22-17eb-42e7-8d88-1c26d286ef85-kube-rbac-proxy-sar-config\") pod \"44f9db22-17eb-42e7-8d88-1c26d286ef85\" (UID: \"44f9db22-17eb-42e7-8d88-1c26d286ef85\") "
Sep 16 19:06:34.467112 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:34.467083    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44f9db22-17eb-42e7-8d88-1c26d286ef85-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "44f9db22-17eb-42e7-8d88-1c26d286ef85" (UID: "44f9db22-17eb-42e7-8d88-1c26d286ef85"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:06:34.468593 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:34.468569    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44f9db22-17eb-42e7-8d88-1c26d286ef85-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "44f9db22-17eb-42e7-8d88-1c26d286ef85" (UID: "44f9db22-17eb-42e7-8d88-1c26d286ef85"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:06:34.469043 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:34.469022    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44f9db22-17eb-42e7-8d88-1c26d286ef85-kube-api-access-tjcpg" (OuterVolumeSpecName: "kube-api-access-tjcpg") pod "44f9db22-17eb-42e7-8d88-1c26d286ef85" (UID: "44f9db22-17eb-42e7-8d88-1c26d286ef85"). InnerVolumeSpecName "kube-api-access-tjcpg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:06:34.567421 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:34.567391    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjcpg\" (UniqueName: \"kubernetes.io/projected/44f9db22-17eb-42e7-8d88-1c26d286ef85-kube-api-access-tjcpg\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 19:06:34.567421 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:34.567418    2557 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/44f9db22-17eb-42e7-8d88-1c26d286ef85-proxy-tls\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 19:06:34.567596 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:34.567429    2557 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/44f9db22-17eb-42e7-8d88-1c26d286ef85-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 19:06:35.037685 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:35.037654    2557 generic.go:356] "Generic (PLEG): container finished" podID="44f9db22-17eb-42e7-8d88-1c26d286ef85" containerID="efd655ad1454a1458374b723562c002ceea03ed762b01778cb36e95a8027b66e" exitCode=0
Sep 16 19:06:35.037859 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:35.037705    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx"
Sep 16 19:06:35.037859 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:35.037734    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx" event={"ID":"44f9db22-17eb-42e7-8d88-1c26d286ef85","Type":"ContainerDied","Data":"efd655ad1454a1458374b723562c002ceea03ed762b01778cb36e95a8027b66e"}
Sep 16 19:06:35.037859 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:35.037818    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx" event={"ID":"44f9db22-17eb-42e7-8d88-1c26d286ef85","Type":"ContainerDied","Data":"7f95f812bfac6034a8575f80b631127ac57f94e1f96ada4a00f5b798feda1725"}
Sep 16 19:06:35.037859 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:35.037850    2557 scope.go:117] "RemoveContainer" containerID="82f4421c95a60f737e09733f4741a3af07ad133ff6e8bb6ec552fa20338a1274"
Sep 16 19:06:35.048335 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:35.048311    2557 scope.go:117] "RemoveContainer" containerID="efd655ad1454a1458374b723562c002ceea03ed762b01778cb36e95a8027b66e"
Sep 16 19:06:35.055946 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:35.055929    2557 scope.go:117] "RemoveContainer" containerID="82f4421c95a60f737e09733f4741a3af07ad133ff6e8bb6ec552fa20338a1274"
Sep 16 19:06:35.056261 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:06:35.056240    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82f4421c95a60f737e09733f4741a3af07ad133ff6e8bb6ec552fa20338a1274\": container with ID starting with 82f4421c95a60f737e09733f4741a3af07ad133ff6e8bb6ec552fa20338a1274 not found: ID does not exist" containerID="82f4421c95a60f737e09733f4741a3af07ad133ff6e8bb6ec552fa20338a1274"
Sep 16 19:06:35.056360 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:35.056267    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82f4421c95a60f737e09733f4741a3af07ad133ff6e8bb6ec552fa20338a1274"} err="failed to get container status \"82f4421c95a60f737e09733f4741a3af07ad133ff6e8bb6ec552fa20338a1274\": rpc error: code = NotFound desc = could not find container \"82f4421c95a60f737e09733f4741a3af07ad133ff6e8bb6ec552fa20338a1274\": container with ID starting with 82f4421c95a60f737e09733f4741a3af07ad133ff6e8bb6ec552fa20338a1274 not found: ID does not exist"
Sep 16 19:06:35.056360 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:35.056286    2557 scope.go:117] "RemoveContainer" containerID="efd655ad1454a1458374b723562c002ceea03ed762b01778cb36e95a8027b66e"
Sep 16 19:06:35.056548 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:06:35.056522    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efd655ad1454a1458374b723562c002ceea03ed762b01778cb36e95a8027b66e\": container with ID starting with efd655ad1454a1458374b723562c002ceea03ed762b01778cb36e95a8027b66e not found: ID does not exist" containerID="efd655ad1454a1458374b723562c002ceea03ed762b01778cb36e95a8027b66e"
Sep 16 19:06:35.056628 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:35.056550    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efd655ad1454a1458374b723562c002ceea03ed762b01778cb36e95a8027b66e"} err="failed to get container status \"efd655ad1454a1458374b723562c002ceea03ed762b01778cb36e95a8027b66e\": rpc error: code = NotFound desc = could not find container \"efd655ad1454a1458374b723562c002ceea03ed762b01778cb36e95a8027b66e\": container with ID starting with efd655ad1454a1458374b723562c002ceea03ed762b01778cb36e95a8027b66e not found: ID does not exist"
Sep 16 19:06:35.065128 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:35.065107    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx"]
Sep 16 19:06:35.066454 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:35.066435    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c6de0-predictor-654f67bf9f-7f7cx"]
Sep 16 19:06:36.467330 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:36.467301    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44f9db22-17eb-42e7-8d88-1c26d286ef85" path="/var/lib/kubelet/pods/44f9db22-17eb-42e7-8d88-1c26d286ef85/volumes"
Sep 16 19:06:38.033437 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:38.033403    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5"
Sep 16 19:06:38.033915 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:38.033884    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5" podUID="a446052c-f917-4296-8587-13e21bb06dea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Sep 16 19:06:39.905974 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:39.905937    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n" podUID="73c06bb6-1195-4d3a-9a58-d01702b61f73" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Sep 16 19:06:48.034545 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:48.034501    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5" podUID="a446052c-f917-4296-8587-13e21bb06dea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Sep 16 19:06:49.906131 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:49.906098    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n"
Sep 16 19:06:58.034656 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:06:58.034616    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5" podUID="a446052c-f917-4296-8587-13e21bb06dea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Sep 16 19:07:08.034087 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:08.034045    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5" podUID="a446052c-f917-4296-8587-13e21bb06dea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Sep 16 19:07:10.929642 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:10.929557    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n"]
Sep 16 19:07:10.930030 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:10.929803    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n" podUID="73c06bb6-1195-4d3a-9a58-d01702b61f73" containerName="kserve-container" containerID="cri-o://00214748509d04f65669fb2f94e908c02d217eeb6a747de4e17144ce65136845" gracePeriod=30
Sep 16 19:07:10.930030 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:10.929851    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n" podUID="73c06bb6-1195-4d3a-9a58-d01702b61f73" containerName="kube-rbac-proxy" containerID="cri-o://aea34a29e5e82c665e9ea876ffa974a7d385e2ceebeec2ec87eb8b27b14167e3" gracePeriod=30
Sep 16 19:07:11.019065 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:11.019030    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf"]
Sep 16 19:07:11.019458 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:11.019433    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44f9db22-17eb-42e7-8d88-1c26d286ef85" containerName="kserve-container"
Sep 16 19:07:11.019458 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:11.019458    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="44f9db22-17eb-42e7-8d88-1c26d286ef85" containerName="kserve-container"
Sep 16 19:07:11.019644 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:11.019478    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44f9db22-17eb-42e7-8d88-1c26d286ef85" containerName="kube-rbac-proxy"
Sep 16 19:07:11.019644 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:11.019487    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="44f9db22-17eb-42e7-8d88-1c26d286ef85" containerName="kube-rbac-proxy"
Sep 16 19:07:11.019644 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:11.019558    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="44f9db22-17eb-42e7-8d88-1c26d286ef85" containerName="kube-rbac-proxy"
Sep 16 19:07:11.019644 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:11.019576    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="44f9db22-17eb-42e7-8d88-1c26d286ef85" containerName="kserve-container"
Sep 16 19:07:11.023894 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:11.023875    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf"
Sep 16 19:07:11.026624 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:11.026607    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-e99c4-predictor-serving-cert\""
Sep 16 19:07:11.026700 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:11.026606    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-e99c4-kube-rbac-proxy-sar-config\""
Sep 16 19:07:11.033189 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:11.033154    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf"]
Sep 16 19:07:11.082940 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:11.082909    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/df26cccb-22ae-4076-8ad2-e9218f84d64d-proxy-tls\") pod \"success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf\" (UID: \"df26cccb-22ae-4076-8ad2-e9218f84d64d\") " pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf"
Sep 16 19:07:11.082940 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:11.082951    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdm6f\" (UniqueName: \"kubernetes.io/projected/df26cccb-22ae-4076-8ad2-e9218f84d64d-kube-api-access-rdm6f\") pod \"success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf\" (UID: \"df26cccb-22ae-4076-8ad2-e9218f84d64d\") " pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf"
Sep 16 19:07:11.083156 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:11.083105    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/df26cccb-22ae-4076-8ad2-e9218f84d64d-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf\" (UID: \"df26cccb-22ae-4076-8ad2-e9218f84d64d\") " pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf"
Sep 16 19:07:11.184539 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:11.184443    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/df26cccb-22ae-4076-8ad2-e9218f84d64d-proxy-tls\") pod \"success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf\" (UID: \"df26cccb-22ae-4076-8ad2-e9218f84d64d\") " pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf"
Sep 16 19:07:11.184539 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:11.184497    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdm6f\" (UniqueName: \"kubernetes.io/projected/df26cccb-22ae-4076-8ad2-e9218f84d64d-kube-api-access-rdm6f\") pod \"success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf\" (UID: \"df26cccb-22ae-4076-8ad2-e9218f84d64d\") " pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf"
Sep 16 19:07:11.184796 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:07:11.184559    2557 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-serving-cert: secret "success-200-isvc-e99c4-predictor-serving-cert" not found
Sep 16 19:07:11.184796 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:11.184581    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/df26cccb-22ae-4076-8ad2-e9218f84d64d-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf\" (UID: \"df26cccb-22ae-4076-8ad2-e9218f84d64d\") " pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf"
Sep 16 19:07:11.184796 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:07:11.184638    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df26cccb-22ae-4076-8ad2-e9218f84d64d-proxy-tls podName:df26cccb-22ae-4076-8ad2-e9218f84d64d nodeName:}" failed. No retries permitted until 2026-09-16 19:07:11.68461646 +0000 UTC m=+1345.761721311 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/df26cccb-22ae-4076-8ad2-e9218f84d64d-proxy-tls") pod "success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf" (UID: "df26cccb-22ae-4076-8ad2-e9218f84d64d") : secret "success-200-isvc-e99c4-predictor-serving-cert" not found
Sep 16 19:07:11.185351 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:11.185328    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/df26cccb-22ae-4076-8ad2-e9218f84d64d-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf\" (UID: \"df26cccb-22ae-4076-8ad2-e9218f84d64d\") " pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf"
Sep 16 19:07:11.190319 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:11.190289    2557 generic.go:356] "Generic (PLEG): container finished" podID="73c06bb6-1195-4d3a-9a58-d01702b61f73" containerID="aea34a29e5e82c665e9ea876ffa974a7d385e2ceebeec2ec87eb8b27b14167e3" exitCode=2
Sep 16 19:07:11.190437 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:11.190335    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n" event={"ID":"73c06bb6-1195-4d3a-9a58-d01702b61f73","Type":"ContainerDied","Data":"aea34a29e5e82c665e9ea876ffa974a7d385e2ceebeec2ec87eb8b27b14167e3"}
Sep 16 19:07:11.194369 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:11.194346    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdm6f\" (UniqueName: \"kubernetes.io/projected/df26cccb-22ae-4076-8ad2-e9218f84d64d-kube-api-access-rdm6f\") pod \"success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf\" (UID: \"df26cccb-22ae-4076-8ad2-e9218f84d64d\") " pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf"
Sep 16 19:07:11.689296 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:11.689257    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/df26cccb-22ae-4076-8ad2-e9218f84d64d-proxy-tls\") pod \"success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf\" (UID: \"df26cccb-22ae-4076-8ad2-e9218f84d64d\") " pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf"
Sep 16 19:07:11.691662 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:11.691642    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/df26cccb-22ae-4076-8ad2-e9218f84d64d-proxy-tls\") pod \"success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf\" (UID: \"df26cccb-22ae-4076-8ad2-e9218f84d64d\") " pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf"
Sep 16 19:07:11.935920 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:11.935886    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf"
Sep 16 19:07:12.208135 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:12.208113    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf"]
Sep 16 19:07:12.209873 ip-10-0-130-26 kubenswrapper[2557]: W0916 19:07:12.209850    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf26cccb_22ae_4076_8ad2_e9218f84d64d.slice/crio-8ebaa8d39c1e2f24b766e19371e7e22846110462d1898d781b2a0f39b59c64bd WatchSource:0}: Error finding container 8ebaa8d39c1e2f24b766e19371e7e22846110462d1898d781b2a0f39b59c64bd: Status 404 returned error can't find the container with id 8ebaa8d39c1e2f24b766e19371e7e22846110462d1898d781b2a0f39b59c64bd
Sep 16 19:07:13.199798 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:13.199756    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf" event={"ID":"df26cccb-22ae-4076-8ad2-e9218f84d64d","Type":"ContainerStarted","Data":"85a348e18e1f6f7520515229172c6824012a2d5baed4b91fa6fabb261117ce00"}
Sep 16 19:07:13.200245 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:13.199822    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf" event={"ID":"df26cccb-22ae-4076-8ad2-e9218f84d64d","Type":"ContainerStarted","Data":"d77c762278b786e9ed95fa5fe1ff4a9c810a20da59083c805b1ebad756bec86d"}
Sep 16 19:07:13.200245 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:13.199840    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf" event={"ID":"df26cccb-22ae-4076-8ad2-e9218f84d64d","Type":"ContainerStarted","Data":"8ebaa8d39c1e2f24b766e19371e7e22846110462d1898d781b2a0f39b59c64bd"}
Sep 16 19:07:13.200245 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:13.200042    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf"
Sep 16 19:07:13.200245 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:13.200083    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf"
Sep 16 19:07:13.201361 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:13.201337    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf" podUID="df26cccb-22ae-4076-8ad2-e9218f84d64d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Sep 16 19:07:13.222482 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:13.222443    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf" podStartSLOduration=3.222430677 podStartE2EDuration="3.222430677s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:07:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:07:13.221367237 +0000 UTC m=+1347.298472098" watchObservedRunningTime="2026-09-16 19:07:13.222430677 +0000 UTC m=+1347.299535537"
Sep 16 19:07:14.185126 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:14.185101    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n"
Sep 16 19:07:14.204702 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:14.204669    2557 generic.go:356] "Generic (PLEG): container finished" podID="73c06bb6-1195-4d3a-9a58-d01702b61f73" containerID="00214748509d04f65669fb2f94e908c02d217eeb6a747de4e17144ce65136845" exitCode=0
Sep 16 19:07:14.205029 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:14.204705    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n"
Sep 16 19:07:14.205029 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:14.204747    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n" event={"ID":"73c06bb6-1195-4d3a-9a58-d01702b61f73","Type":"ContainerDied","Data":"00214748509d04f65669fb2f94e908c02d217eeb6a747de4e17144ce65136845"}
Sep 16 19:07:14.205029 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:14.204821    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n" event={"ID":"73c06bb6-1195-4d3a-9a58-d01702b61f73","Type":"ContainerDied","Data":"7ebba0208869d7dadc75ca73632e85b9284f8096d1022a848a876983246df11f"}
Sep 16 19:07:14.205029 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:14.204849    2557 scope.go:117] "RemoveContainer" containerID="aea34a29e5e82c665e9ea876ffa974a7d385e2ceebeec2ec87eb8b27b14167e3"
Sep 16 19:07:14.205244 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:14.205209    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf" podUID="df26cccb-22ae-4076-8ad2-e9218f84d64d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Sep 16 19:07:14.210153 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:14.210094    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8xl7\" (UniqueName: \"kubernetes.io/projected/73c06bb6-1195-4d3a-9a58-d01702b61f73-kube-api-access-h8xl7\") pod \"73c06bb6-1195-4d3a-9a58-d01702b61f73\" (UID: \"73c06bb6-1195-4d3a-9a58-d01702b61f73\") "
Sep 16 19:07:14.210268 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:14.210206    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/73c06bb6-1195-4d3a-9a58-d01702b61f73-proxy-tls\") pod \"73c06bb6-1195-4d3a-9a58-d01702b61f73\" (UID: \"73c06bb6-1195-4d3a-9a58-d01702b61f73\") "
Sep 16 19:07:14.210330 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:14.210266    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/73c06bb6-1195-4d3a-9a58-d01702b61f73-kube-rbac-proxy-sar-config\") pod \"73c06bb6-1195-4d3a-9a58-d01702b61f73\" (UID: \"73c06bb6-1195-4d3a-9a58-d01702b61f73\") "
Sep 16 19:07:14.210665 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:14.210613    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73c06bb6-1195-4d3a-9a58-d01702b61f73-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "73c06bb6-1195-4d3a-9a58-d01702b61f73" (UID: "73c06bb6-1195-4d3a-9a58-d01702b61f73"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:07:14.212344 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:14.212312    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73c06bb6-1195-4d3a-9a58-d01702b61f73-kube-api-access-h8xl7" (OuterVolumeSpecName: "kube-api-access-h8xl7") pod "73c06bb6-1195-4d3a-9a58-d01702b61f73" (UID: "73c06bb6-1195-4d3a-9a58-d01702b61f73"). InnerVolumeSpecName "kube-api-access-h8xl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:07:14.212546 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:14.212517    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73c06bb6-1195-4d3a-9a58-d01702b61f73-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "73c06bb6-1195-4d3a-9a58-d01702b61f73" (UID: "73c06bb6-1195-4d3a-9a58-d01702b61f73"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:07:14.214381 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:14.214363    2557 scope.go:117] "RemoveContainer" containerID="00214748509d04f65669fb2f94e908c02d217eeb6a747de4e17144ce65136845"
Sep 16 19:07:14.226783 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:14.226753    2557 scope.go:117] "RemoveContainer" containerID="aea34a29e5e82c665e9ea876ffa974a7d385e2ceebeec2ec87eb8b27b14167e3"
Sep 16 19:07:14.227089 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:07:14.227070    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aea34a29e5e82c665e9ea876ffa974a7d385e2ceebeec2ec87eb8b27b14167e3\": container with ID starting with aea34a29e5e82c665e9ea876ffa974a7d385e2ceebeec2ec87eb8b27b14167e3 not found: ID does not exist" containerID="aea34a29e5e82c665e9ea876ffa974a7d385e2ceebeec2ec87eb8b27b14167e3"
Sep 16 19:07:14.227167 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:14.227094    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aea34a29e5e82c665e9ea876ffa974a7d385e2ceebeec2ec87eb8b27b14167e3"} err="failed to get container status \"aea34a29e5e82c665e9ea876ffa974a7d385e2ceebeec2ec87eb8b27b14167e3\": rpc error: code = NotFound desc = could not find container \"aea34a29e5e82c665e9ea876ffa974a7d385e2ceebeec2ec87eb8b27b14167e3\": container with ID starting with aea34a29e5e82c665e9ea876ffa974a7d385e2ceebeec2ec87eb8b27b14167e3 not found: ID does not exist"
Sep 16 19:07:14.227167 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:14.227110    2557 scope.go:117] "RemoveContainer" containerID="00214748509d04f65669fb2f94e908c02d217eeb6a747de4e17144ce65136845"
Sep 16 19:07:14.227387 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:07:14.227368    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00214748509d04f65669fb2f94e908c02d217eeb6a747de4e17144ce65136845\": container with ID starting with 00214748509d04f65669fb2f94e908c02d217eeb6a747de4e17144ce65136845 not found: ID does not exist" containerID="00214748509d04f65669fb2f94e908c02d217eeb6a747de4e17144ce65136845"
Sep 16 19:07:14.227456 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:14.227391    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00214748509d04f65669fb2f94e908c02d217eeb6a747de4e17144ce65136845"} err="failed to get container status \"00214748509d04f65669fb2f94e908c02d217eeb6a747de4e17144ce65136845\": rpc error: code = NotFound desc = could not find container \"00214748509d04f65669fb2f94e908c02d217eeb6a747de4e17144ce65136845\": container with ID starting with 00214748509d04f65669fb2f94e908c02d217eeb6a747de4e17144ce65136845 not found: ID does not exist"
Sep 16 19:07:14.317979 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:14.317939    2557 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/73c06bb6-1195-4d3a-9a58-d01702b61f73-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 19:07:14.318113 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:14.317983    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8xl7\" (UniqueName: \"kubernetes.io/projected/73c06bb6-1195-4d3a-9a58-d01702b61f73-kube-api-access-h8xl7\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 19:07:14.318113 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:14.318005    2557 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/73c06bb6-1195-4d3a-9a58-d01702b61f73-proxy-tls\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 19:07:14.523473 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:14.523446    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n"]
Sep 16 19:07:14.529714 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:14.529688    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-f64b9-predictor-ffc69556f-g6c6n"]
Sep 16 19:07:16.466252 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:16.466220    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73c06bb6-1195-4d3a-9a58-d01702b61f73" path="/var/lib/kubelet/pods/73c06bb6-1195-4d3a-9a58-d01702b61f73/volumes"
Sep 16 19:07:18.034623 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:18.034596    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5"
Sep 16 19:07:19.209935 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:19.209903    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf"
Sep 16 19:07:19.210373 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:19.210348    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf" podUID="df26cccb-22ae-4076-8ad2-e9218f84d64d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Sep 16 19:07:29.210947 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:29.210904    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf" podUID="df26cccb-22ae-4076-8ad2-e9218f84d64d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Sep 16 19:07:39.210601 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:39.210561    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf" podUID="df26cccb-22ae-4076-8ad2-e9218f84d64d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Sep 16 19:07:49.210564 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:49.210519    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf" podUID="df26cccb-22ae-4076-8ad2-e9218f84d64d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Sep 16 19:07:59.210922 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:07:59.210890    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf"
Sep 16 19:15:46.019664 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:46.019625    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5"]
Sep 16 19:15:46.020194 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:46.019874    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5" podUID="a446052c-f917-4296-8587-13e21bb06dea" containerName="kserve-container" containerID="cri-o://539107b73b2f25c3375132235d26642ba9fd7f2ff7426439be293a1263651529" gracePeriod=30
Sep 16 19:15:46.020194 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:46.019923    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5" podUID="a446052c-f917-4296-8587-13e21bb06dea" containerName="kube-rbac-proxy" containerID="cri-o://5893a7b9dfefe624ba5e8ff4d58fccadeb88334a2b547e9c93761f543f9caed4" gracePeriod=30
Sep 16 19:15:46.059475 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:46.059446    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf"]
Sep 16 19:15:46.059795 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:46.059782    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73c06bb6-1195-4d3a-9a58-d01702b61f73" containerName="kserve-container"
Sep 16 19:15:46.059841 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:46.059796    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="73c06bb6-1195-4d3a-9a58-d01702b61f73" containerName="kserve-container"
Sep 16 19:15:46.059841 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:46.059819    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73c06bb6-1195-4d3a-9a58-d01702b61f73" containerName="kube-rbac-proxy"
Sep 16 19:15:46.059841 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:46.059824    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="73c06bb6-1195-4d3a-9a58-d01702b61f73" containerName="kube-rbac-proxy"
Sep 16 19:15:46.059937 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:46.059873    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="73c06bb6-1195-4d3a-9a58-d01702b61f73" containerName="kserve-container"
Sep 16 19:15:46.059937 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:46.059881    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="73c06bb6-1195-4d3a-9a58-d01702b61f73" containerName="kube-rbac-proxy"
Sep 16 19:15:46.063195 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:46.063169    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf"
Sep 16 19:15:46.066792 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:46.066747    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-da3eb-predictor-serving-cert\""
Sep 16 19:15:46.066881 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:46.066843    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-da3eb-kube-rbac-proxy-sar-config\""
Sep 16 19:15:46.080817 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:46.080790    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf"]
Sep 16 19:15:46.095169 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:46.095144    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84pv4\" (UniqueName: \"kubernetes.io/projected/d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e-kube-api-access-84pv4\") pod \"success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf\" (UID: \"d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e\") " pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf"
Sep 16 19:15:46.095287 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:46.095200    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e-proxy-tls\") pod \"success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf\" (UID: \"d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e\") " pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf"
Sep 16 19:15:46.095353 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:46.095324    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf\" (UID: \"d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e\") " pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf"
Sep 16 19:15:46.196663 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:46.196625    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84pv4\" (UniqueName: \"kubernetes.io/projected/d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e-kube-api-access-84pv4\") pod \"success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf\" (UID: \"d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e\") " pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf"
Sep 16 19:15:46.196858 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:46.196678    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e-proxy-tls\") pod \"success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf\" (UID: \"d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e\") " pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf"
Sep 16 19:15:46.196858 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:46.196731    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf\" (UID: \"d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e\") " pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf"
Sep 16 19:15:46.197374 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:46.197344    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf\" (UID: \"d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e\") " pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf"
Sep 16 19:15:46.199115 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:46.199093    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e-proxy-tls\") pod \"success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf\" (UID: \"d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e\") " pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf"
Sep 16 19:15:46.213308 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:46.213286    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84pv4\" (UniqueName: \"kubernetes.io/projected/d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e-kube-api-access-84pv4\") pod \"success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf\" (UID: \"d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e\") " pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf"
Sep 16 19:15:46.256264 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:46.256233    2557 generic.go:356] "Generic (PLEG): container finished" podID="a446052c-f917-4296-8587-13e21bb06dea" containerID="5893a7b9dfefe624ba5e8ff4d58fccadeb88334a2b547e9c93761f543f9caed4" exitCode=2
Sep 16 19:15:46.256391 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:46.256284    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5" event={"ID":"a446052c-f917-4296-8587-13e21bb06dea","Type":"ContainerDied","Data":"5893a7b9dfefe624ba5e8ff4d58fccadeb88334a2b547e9c93761f543f9caed4"}
Sep 16 19:15:46.374953 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:46.374924    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf"
Sep 16 19:15:46.658492 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:46.658464    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf"]
Sep 16 19:15:46.661407 ip-10-0-130-26 kubenswrapper[2557]: W0916 19:15:46.661381    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4457dd2_6c6b_42d4_b9f9_1cb5e2f35d0e.slice/crio-d0b5c8fc6cf0bb53884e674d4af5a41c4198949afbc3f64664381de7756f0361 WatchSource:0}: Error finding container d0b5c8fc6cf0bb53884e674d4af5a41c4198949afbc3f64664381de7756f0361: Status 404 returned error can't find the container with id d0b5c8fc6cf0bb53884e674d4af5a41c4198949afbc3f64664381de7756f0361
Sep 16 19:15:47.262035 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:47.261997    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf" event={"ID":"d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e","Type":"ContainerStarted","Data":"66850764cbe86e7f63bc9a897b6983d657b7572640d9b872720aefc915f3c736"}
Sep 16 19:15:47.262438 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:47.262064    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf" event={"ID":"d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e","Type":"ContainerStarted","Data":"ddffb19acb56a430aaf9dc9e4b52045b52469dc7bcdb66227b8d92556b54782f"}
Sep 16 19:15:47.262438 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:47.262096    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf" event={"ID":"d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e","Type":"ContainerStarted","Data":"d0b5c8fc6cf0bb53884e674d4af5a41c4198949afbc3f64664381de7756f0361"}
Sep 16 19:15:47.262438 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:47.262133    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf"
Sep 16 19:15:47.305937 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:47.305890    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf" podStartSLOduration=1.3058736149999999 podStartE2EDuration="1.305873615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:15:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:15:47.303309558 +0000 UTC m=+1861.380414419" watchObservedRunningTime="2026-09-16 19:15:47.305873615 +0000 UTC m=+1861.382978534"
Sep 16 19:15:48.028953 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:48.028914    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5" podUID="a446052c-f917-4296-8587-13e21bb06dea" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.33:8643/healthz\": dial tcp 10.134.0.33:8643: connect: connection refused"
Sep 16 19:15:48.034301 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:48.034279    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5" podUID="a446052c-f917-4296-8587-13e21bb06dea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Sep 16 19:15:48.266466 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:48.266435    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf"
Sep 16 19:15:48.267887 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:48.267857    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf" podUID="d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Sep 16 19:15:49.168794 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:49.168752    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5"
Sep 16 19:15:49.224730 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:49.224641    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a446052c-f917-4296-8587-13e21bb06dea-proxy-tls\") pod \"a446052c-f917-4296-8587-13e21bb06dea\" (UID: \"a446052c-f917-4296-8587-13e21bb06dea\") "
Sep 16 19:15:49.224901 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:49.224791    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a446052c-f917-4296-8587-13e21bb06dea-kube-rbac-proxy-sar-config\") pod \"a446052c-f917-4296-8587-13e21bb06dea\" (UID: \"a446052c-f917-4296-8587-13e21bb06dea\") "
Sep 16 19:15:49.224901 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:49.224826    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dftb4\" (UniqueName: \"kubernetes.io/projected/a446052c-f917-4296-8587-13e21bb06dea-kube-api-access-dftb4\") pod \"a446052c-f917-4296-8587-13e21bb06dea\" (UID: \"a446052c-f917-4296-8587-13e21bb06dea\") "
Sep 16 19:15:49.225139 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:49.225115    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a446052c-f917-4296-8587-13e21bb06dea-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "a446052c-f917-4296-8587-13e21bb06dea" (UID: "a446052c-f917-4296-8587-13e21bb06dea"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:15:49.226936 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:49.226906    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a446052c-f917-4296-8587-13e21bb06dea-kube-api-access-dftb4" (OuterVolumeSpecName: "kube-api-access-dftb4") pod "a446052c-f917-4296-8587-13e21bb06dea" (UID: "a446052c-f917-4296-8587-13e21bb06dea"). InnerVolumeSpecName "kube-api-access-dftb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:15:49.227009 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:49.226917    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a446052c-f917-4296-8587-13e21bb06dea-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "a446052c-f917-4296-8587-13e21bb06dea" (UID: "a446052c-f917-4296-8587-13e21bb06dea"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:15:49.272315 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:49.272286    2557 generic.go:356] "Generic (PLEG): container finished" podID="a446052c-f917-4296-8587-13e21bb06dea" containerID="539107b73b2f25c3375132235d26642ba9fd7f2ff7426439be293a1263651529" exitCode=0
Sep 16 19:15:49.272716 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:49.272331    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5"
Sep 16 19:15:49.272716 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:49.272365    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5" event={"ID":"a446052c-f917-4296-8587-13e21bb06dea","Type":"ContainerDied","Data":"539107b73b2f25c3375132235d26642ba9fd7f2ff7426439be293a1263651529"}
Sep 16 19:15:49.272716 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:49.272418    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5" event={"ID":"a446052c-f917-4296-8587-13e21bb06dea","Type":"ContainerDied","Data":"c9823f0511cbb7ce0f3ff574edf69d4d31fe3b16bf6b817e7469b9443e7e330e"}
Sep 16 19:15:49.272716 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:49.272443    2557 scope.go:117] "RemoveContainer" containerID="5893a7b9dfefe624ba5e8ff4d58fccadeb88334a2b547e9c93761f543f9caed4"
Sep 16 19:15:49.272944 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:49.272841    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf" podUID="d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Sep 16 19:15:49.281220 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:49.281191    2557 scope.go:117] "RemoveContainer" containerID="539107b73b2f25c3375132235d26642ba9fd7f2ff7426439be293a1263651529"
Sep 16 19:15:49.288476 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:49.288457    2557 scope.go:117] "RemoveContainer" containerID="5893a7b9dfefe624ba5e8ff4d58fccadeb88334a2b547e9c93761f543f9caed4"
Sep 16 19:15:49.288725 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:15:49.288708    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5893a7b9dfefe624ba5e8ff4d58fccadeb88334a2b547e9c93761f543f9caed4\": container with ID starting with 5893a7b9dfefe624ba5e8ff4d58fccadeb88334a2b547e9c93761f543f9caed4 not found: ID does not exist" containerID="5893a7b9dfefe624ba5e8ff4d58fccadeb88334a2b547e9c93761f543f9caed4"
Sep 16 19:15:49.288810 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:49.288732    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5893a7b9dfefe624ba5e8ff4d58fccadeb88334a2b547e9c93761f543f9caed4"} err="failed to get container status \"5893a7b9dfefe624ba5e8ff4d58fccadeb88334a2b547e9c93761f543f9caed4\": rpc error: code = NotFound desc = could not find container \"5893a7b9dfefe624ba5e8ff4d58fccadeb88334a2b547e9c93761f543f9caed4\": container with ID starting with 5893a7b9dfefe624ba5e8ff4d58fccadeb88334a2b547e9c93761f543f9caed4 not found: ID does not exist"
Sep 16 19:15:49.288810 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:49.288747    2557 scope.go:117] "RemoveContainer" containerID="539107b73b2f25c3375132235d26642ba9fd7f2ff7426439be293a1263651529"
Sep 16 19:15:49.288989 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:15:49.288971    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"539107b73b2f25c3375132235d26642ba9fd7f2ff7426439be293a1263651529\": container with ID starting with 539107b73b2f25c3375132235d26642ba9fd7f2ff7426439be293a1263651529 not found: ID does not exist" containerID="539107b73b2f25c3375132235d26642ba9fd7f2ff7426439be293a1263651529"
Sep 16 19:15:49.289047 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:49.289000    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"539107b73b2f25c3375132235d26642ba9fd7f2ff7426439be293a1263651529"} err="failed to get container status \"539107b73b2f25c3375132235d26642ba9fd7f2ff7426439be293a1263651529\": rpc error: code = NotFound desc = could not find container \"539107b73b2f25c3375132235d26642ba9fd7f2ff7426439be293a1263651529\": container with ID starting with 539107b73b2f25c3375132235d26642ba9fd7f2ff7426439be293a1263651529 not found: ID does not exist"
Sep 16 19:15:49.296562 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:49.296538    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5"]
Sep 16 19:15:49.301011 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:49.300984    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-a3448-predictor-779f587bc9-5zwl5"]
Sep 16 19:15:49.326226 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:49.326204    2557 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a446052c-f917-4296-8587-13e21bb06dea-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 19:15:49.326226 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:49.326226    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dftb4\" (UniqueName: \"kubernetes.io/projected/a446052c-f917-4296-8587-13e21bb06dea-kube-api-access-dftb4\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 19:15:49.326409 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:49.326238    2557 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a446052c-f917-4296-8587-13e21bb06dea-proxy-tls\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 19:15:50.465784 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:50.465740    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a446052c-f917-4296-8587-13e21bb06dea" path="/var/lib/kubelet/pods/a446052c-f917-4296-8587-13e21bb06dea/volumes"
Sep 16 19:15:54.277380 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:54.277349    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf"
Sep 16 19:15:54.277879 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:15:54.277837    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf" podUID="d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Sep 16 19:16:04.277745 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:04.277700    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf" podUID="d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Sep 16 19:16:14.278246 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:14.278166    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf" podUID="d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Sep 16 19:16:24.277758 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:24.277719    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf" podUID="d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Sep 16 19:16:25.794704 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:25.794666    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf"]
Sep 16 19:16:25.795224 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:25.794945    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf" podUID="df26cccb-22ae-4076-8ad2-e9218f84d64d" containerName="kserve-container" containerID="cri-o://d77c762278b786e9ed95fa5fe1ff4a9c810a20da59083c805b1ebad756bec86d" gracePeriod=30
Sep 16 19:16:25.795224 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:25.795023    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf" podUID="df26cccb-22ae-4076-8ad2-e9218f84d64d" containerName="kube-rbac-proxy" containerID="cri-o://85a348e18e1f6f7520515229172c6824012a2d5baed4b91fa6fabb261117ce00" gracePeriod=30
Sep 16 19:16:25.922086 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:25.922051    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k"]
Sep 16 19:16:25.922415 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:25.922402    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a446052c-f917-4296-8587-13e21bb06dea" containerName="kube-rbac-proxy"
Sep 16 19:16:25.922463 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:25.922418    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a446052c-f917-4296-8587-13e21bb06dea" containerName="kube-rbac-proxy"
Sep 16 19:16:25.922463 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:25.922434    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a446052c-f917-4296-8587-13e21bb06dea" containerName="kserve-container"
Sep 16 19:16:25.922463 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:25.922440    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a446052c-f917-4296-8587-13e21bb06dea" containerName="kserve-container"
Sep 16 19:16:25.922561 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:25.922493    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a446052c-f917-4296-8587-13e21bb06dea" containerName="kube-rbac-proxy"
Sep 16 19:16:25.922561 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:25.922502    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a446052c-f917-4296-8587-13e21bb06dea" containerName="kserve-container"
Sep 16 19:16:25.925532 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:25.925515    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k"
Sep 16 19:16:25.928697 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:25.928674    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-ca22e-predictor-serving-cert\""
Sep 16 19:16:25.928840 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:25.928705    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-ca22e-kube-rbac-proxy-sar-config\""
Sep 16 19:16:25.931435 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:25.931404    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85gz2\" (UniqueName: \"kubernetes.io/projected/ef314c3b-f862-4b89-8a16-633b13991399-kube-api-access-85gz2\") pod \"success-200-isvc-ca22e-predictor-558b7cd765-n4t6k\" (UID: \"ef314c3b-f862-4b89-8a16-633b13991399\") " pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k"
Sep 16 19:16:25.931532 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:25.931447    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ef314c3b-f862-4b89-8a16-633b13991399-proxy-tls\") pod \"success-200-isvc-ca22e-predictor-558b7cd765-n4t6k\" (UID: \"ef314c3b-f862-4b89-8a16-633b13991399\") " pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k"
Sep 16 19:16:25.931532 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:25.931504    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ef314c3b-f862-4b89-8a16-633b13991399-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ca22e-predictor-558b7cd765-n4t6k\" (UID: \"ef314c3b-f862-4b89-8a16-633b13991399\") " pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k"
Sep 16 19:16:25.948069 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:25.948042    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k"]
Sep 16 19:16:26.032083 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:26.032036    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ef314c3b-f862-4b89-8a16-633b13991399-proxy-tls\") pod \"success-200-isvc-ca22e-predictor-558b7cd765-n4t6k\" (UID: \"ef314c3b-f862-4b89-8a16-633b13991399\") " pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k"
Sep 16 19:16:26.032288 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:26.032094    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ef314c3b-f862-4b89-8a16-633b13991399-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ca22e-predictor-558b7cd765-n4t6k\" (UID: \"ef314c3b-f862-4b89-8a16-633b13991399\") " pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k"
Sep 16 19:16:26.032288 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:16:26.032187    2557 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-serving-cert: secret "success-200-isvc-ca22e-predictor-serving-cert" not found
Sep 16 19:16:26.032288 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:26.032201    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85gz2\" (UniqueName: \"kubernetes.io/projected/ef314c3b-f862-4b89-8a16-633b13991399-kube-api-access-85gz2\") pod \"success-200-isvc-ca22e-predictor-558b7cd765-n4t6k\" (UID: \"ef314c3b-f862-4b89-8a16-633b13991399\") " pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k"
Sep 16 19:16:26.032288 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:16:26.032277    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef314c3b-f862-4b89-8a16-633b13991399-proxy-tls podName:ef314c3b-f862-4b89-8a16-633b13991399 nodeName:}" failed. No retries permitted until 2026-09-16 19:16:26.532255466 +0000 UTC m=+1900.609360327 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/ef314c3b-f862-4b89-8a16-633b13991399-proxy-tls") pod "success-200-isvc-ca22e-predictor-558b7cd765-n4t6k" (UID: "ef314c3b-f862-4b89-8a16-633b13991399") : secret "success-200-isvc-ca22e-predictor-serving-cert" not found
Sep 16 19:16:26.032786 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:26.032755    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ef314c3b-f862-4b89-8a16-633b13991399-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ca22e-predictor-558b7cd765-n4t6k\" (UID: \"ef314c3b-f862-4b89-8a16-633b13991399\") " pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k"
Sep 16 19:16:26.048446 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:26.048373    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85gz2\" (UniqueName: \"kubernetes.io/projected/ef314c3b-f862-4b89-8a16-633b13991399-kube-api-access-85gz2\") pod \"success-200-isvc-ca22e-predictor-558b7cd765-n4t6k\" (UID: \"ef314c3b-f862-4b89-8a16-633b13991399\") " pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k"
Sep 16 19:16:26.425487 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:26.425393    2557 generic.go:356] "Generic (PLEG): container finished" podID="df26cccb-22ae-4076-8ad2-e9218f84d64d" containerID="85a348e18e1f6f7520515229172c6824012a2d5baed4b91fa6fabb261117ce00" exitCode=2
Sep 16 19:16:26.425487 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:26.425451    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf" event={"ID":"df26cccb-22ae-4076-8ad2-e9218f84d64d","Type":"ContainerDied","Data":"85a348e18e1f6f7520515229172c6824012a2d5baed4b91fa6fabb261117ce00"}
Sep 16 19:16:26.534968 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:26.534932    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ef314c3b-f862-4b89-8a16-633b13991399-proxy-tls\") pod \"success-200-isvc-ca22e-predictor-558b7cd765-n4t6k\" (UID: \"ef314c3b-f862-4b89-8a16-633b13991399\") " pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k"
Sep 16 19:16:26.537322 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:26.537302    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ef314c3b-f862-4b89-8a16-633b13991399-proxy-tls\") pod \"success-200-isvc-ca22e-predictor-558b7cd765-n4t6k\" (UID: \"ef314c3b-f862-4b89-8a16-633b13991399\") " pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k"
Sep 16 19:16:26.835674 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:26.835630    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k"
Sep 16 19:16:27.124418 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:27.117659    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k"]
Sep 16 19:16:27.431725 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:27.431632    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k" event={"ID":"ef314c3b-f862-4b89-8a16-633b13991399","Type":"ContainerStarted","Data":"4eac47705a0a6b6e6247dc1c5084824a35e6e37ce5cf06352ba810a2cd0cc5ba"}
Sep 16 19:16:27.431725 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:27.431705    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k" event={"ID":"ef314c3b-f862-4b89-8a16-633b13991399","Type":"ContainerStarted","Data":"fb299558856136d10b17cefba6c4e0f1c8be4d873c65085e08211104c43a9668"}
Sep 16 19:16:27.431938 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:27.431738    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k" event={"ID":"ef314c3b-f862-4b89-8a16-633b13991399","Type":"ContainerStarted","Data":"4d6b4185757e635a2bd27f3e4ec685ac0fd4510e70391b4564508e3524909cc7"}
Sep 16 19:16:27.431938 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:27.431801    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k"
Sep 16 19:16:27.455838 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:27.455752    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k" podStartSLOduration=2.455731037 podStartE2EDuration="2.455731037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:16:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:16:27.454402298 +0000 UTC m=+1901.531507160" watchObservedRunningTime="2026-09-16 19:16:27.455731037 +0000 UTC m=+1901.532835898"
Sep 16 19:16:28.436746 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:28.436711    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k"
Sep 16 19:16:28.437797 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:28.437752    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k" podUID="ef314c3b-f862-4b89-8a16-633b13991399" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Sep 16 19:16:29.205919 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:29.205875    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf" podUID="df26cccb-22ae-4076-8ad2-e9218f84d64d" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.34:8643/healthz\": dial tcp 10.134.0.34:8643: connect: connection refused"
Sep 16 19:16:29.211339 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:29.211311    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf" podUID="df26cccb-22ae-4076-8ad2-e9218f84d64d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Sep 16 19:16:29.441251 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:29.441216    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k" podUID="ef314c3b-f862-4b89-8a16-633b13991399" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Sep 16 19:16:31.943337 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:31.943313    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf"
Sep 16 19:16:31.978622 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:31.978593    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/df26cccb-22ae-4076-8ad2-e9218f84d64d-kube-rbac-proxy-sar-config\") pod \"df26cccb-22ae-4076-8ad2-e9218f84d64d\" (UID: \"df26cccb-22ae-4076-8ad2-e9218f84d64d\") "
Sep 16 19:16:31.978762 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:31.978685    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/df26cccb-22ae-4076-8ad2-e9218f84d64d-proxy-tls\") pod \"df26cccb-22ae-4076-8ad2-e9218f84d64d\" (UID: \"df26cccb-22ae-4076-8ad2-e9218f84d64d\") "
Sep 16 19:16:31.978762 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:31.978709    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdm6f\" (UniqueName: \"kubernetes.io/projected/df26cccb-22ae-4076-8ad2-e9218f84d64d-kube-api-access-rdm6f\") pod \"df26cccb-22ae-4076-8ad2-e9218f84d64d\" (UID: \"df26cccb-22ae-4076-8ad2-e9218f84d64d\") "
Sep 16 19:16:31.979075 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:31.979035    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df26cccb-22ae-4076-8ad2-e9218f84d64d-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "df26cccb-22ae-4076-8ad2-e9218f84d64d" (UID: "df26cccb-22ae-4076-8ad2-e9218f84d64d"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:16:31.980739 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:31.980716    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df26cccb-22ae-4076-8ad2-e9218f84d64d-kube-api-access-rdm6f" (OuterVolumeSpecName: "kube-api-access-rdm6f") pod "df26cccb-22ae-4076-8ad2-e9218f84d64d" (UID: "df26cccb-22ae-4076-8ad2-e9218f84d64d"). InnerVolumeSpecName "kube-api-access-rdm6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:16:31.980835 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:31.980743    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df26cccb-22ae-4076-8ad2-e9218f84d64d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "df26cccb-22ae-4076-8ad2-e9218f84d64d" (UID: "df26cccb-22ae-4076-8ad2-e9218f84d64d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:16:32.079992 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:32.079956    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdm6f\" (UniqueName: \"kubernetes.io/projected/df26cccb-22ae-4076-8ad2-e9218f84d64d-kube-api-access-rdm6f\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 19:16:32.079992 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:32.079987    2557 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/df26cccb-22ae-4076-8ad2-e9218f84d64d-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 19:16:32.079992 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:32.079998    2557 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/df26cccb-22ae-4076-8ad2-e9218f84d64d-proxy-tls\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 19:16:32.454930 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:32.454836    2557 generic.go:356] "Generic (PLEG): container finished" podID="df26cccb-22ae-4076-8ad2-e9218f84d64d" containerID="d77c762278b786e9ed95fa5fe1ff4a9c810a20da59083c805b1ebad756bec86d" exitCode=0
Sep 16 19:16:32.454930 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:32.454908    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf"
Sep 16 19:16:32.455100 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:32.454924    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf" event={"ID":"df26cccb-22ae-4076-8ad2-e9218f84d64d","Type":"ContainerDied","Data":"d77c762278b786e9ed95fa5fe1ff4a9c810a20da59083c805b1ebad756bec86d"}
Sep 16 19:16:32.455100 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:32.455017    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf" event={"ID":"df26cccb-22ae-4076-8ad2-e9218f84d64d","Type":"ContainerDied","Data":"8ebaa8d39c1e2f24b766e19371e7e22846110462d1898d781b2a0f39b59c64bd"}
Sep 16 19:16:32.455100 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:32.455050    2557 scope.go:117] "RemoveContainer" containerID="85a348e18e1f6f7520515229172c6824012a2d5baed4b91fa6fabb261117ce00"
Sep 16 19:16:32.464722 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:32.464704    2557 scope.go:117] "RemoveContainer" containerID="d77c762278b786e9ed95fa5fe1ff4a9c810a20da59083c805b1ebad756bec86d"
Sep 16 19:16:32.472270 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:32.472250    2557 scope.go:117] "RemoveContainer" containerID="85a348e18e1f6f7520515229172c6824012a2d5baed4b91fa6fabb261117ce00"
Sep 16 19:16:32.472519 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:16:32.472496    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85a348e18e1f6f7520515229172c6824012a2d5baed4b91fa6fabb261117ce00\": container with ID starting with 85a348e18e1f6f7520515229172c6824012a2d5baed4b91fa6fabb261117ce00 not found: ID does not exist" containerID="85a348e18e1f6f7520515229172c6824012a2d5baed4b91fa6fabb261117ce00"
Sep 16 19:16:32.472595 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:32.472532    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85a348e18e1f6f7520515229172c6824012a2d5baed4b91fa6fabb261117ce00"} err="failed to get container status \"85a348e18e1f6f7520515229172c6824012a2d5baed4b91fa6fabb261117ce00\": rpc error: code = NotFound desc = could not find container \"85a348e18e1f6f7520515229172c6824012a2d5baed4b91fa6fabb261117ce00\": container with ID starting with 85a348e18e1f6f7520515229172c6824012a2d5baed4b91fa6fabb261117ce00 not found: ID does not exist"
Sep 16 19:16:32.472595 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:32.472557    2557 scope.go:117] "RemoveContainer" containerID="d77c762278b786e9ed95fa5fe1ff4a9c810a20da59083c805b1ebad756bec86d"
Sep 16 19:16:32.472823 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:16:32.472804    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d77c762278b786e9ed95fa5fe1ff4a9c810a20da59083c805b1ebad756bec86d\": container with ID starting with d77c762278b786e9ed95fa5fe1ff4a9c810a20da59083c805b1ebad756bec86d not found: ID does not exist" containerID="d77c762278b786e9ed95fa5fe1ff4a9c810a20da59083c805b1ebad756bec86d"
Sep 16 19:16:32.472868 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:32.472829    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d77c762278b786e9ed95fa5fe1ff4a9c810a20da59083c805b1ebad756bec86d"} err="failed to get container status \"d77c762278b786e9ed95fa5fe1ff4a9c810a20da59083c805b1ebad756bec86d\": rpc error: code = NotFound desc = could not find container \"d77c762278b786e9ed95fa5fe1ff4a9c810a20da59083c805b1ebad756bec86d\": container with ID starting with d77c762278b786e9ed95fa5fe1ff4a9c810a20da59083c805b1ebad756bec86d not found: ID does not exist"
Sep 16 19:16:32.477621 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:32.477600    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf"]
Sep 16 19:16:32.482598 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:32.482578    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e99c4-predictor-5dcbf77d7f-hqcjf"]
Sep 16 19:16:34.278737 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:34.278709    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf"
Sep 16 19:16:34.445239 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:34.445210    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k"
Sep 16 19:16:34.445890 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:34.445862    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k" podUID="ef314c3b-f862-4b89-8a16-633b13991399" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Sep 16 19:16:34.465578 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:34.465549    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df26cccb-22ae-4076-8ad2-e9218f84d64d" path="/var/lib/kubelet/pods/df26cccb-22ae-4076-8ad2-e9218f84d64d/volumes"
Sep 16 19:16:44.446797 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:44.446742    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k" podUID="ef314c3b-f862-4b89-8a16-633b13991399" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Sep 16 19:16:54.446113 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:54.446075    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k" podUID="ef314c3b-f862-4b89-8a16-633b13991399" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Sep 16 19:16:56.234014 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:56.233982    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf"]
Sep 16 19:16:56.234418 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:56.234196    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf" podUID="d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e" containerName="kserve-container" containerID="cri-o://ddffb19acb56a430aaf9dc9e4b52045b52469dc7bcdb66227b8d92556b54782f" gracePeriod=30
Sep 16 19:16:56.234418 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:56.234268    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf" podUID="d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e" containerName="kube-rbac-proxy" containerID="cri-o://66850764cbe86e7f63bc9a897b6983d657b7572640d9b872720aefc915f3c736" gracePeriod=30
Sep 16 19:16:56.390109 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:56.390062    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp"]
Sep 16 19:16:56.390462 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:56.390444    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df26cccb-22ae-4076-8ad2-e9218f84d64d" containerName="kserve-container"
Sep 16 19:16:56.390547 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:56.390465    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="df26cccb-22ae-4076-8ad2-e9218f84d64d" containerName="kserve-container"
Sep 16 19:16:56.390547 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:56.390484    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df26cccb-22ae-4076-8ad2-e9218f84d64d" containerName="kube-rbac-proxy"
Sep 16 19:16:56.390547 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:56.390492    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="df26cccb-22ae-4076-8ad2-e9218f84d64d" containerName="kube-rbac-proxy"
Sep 16 19:16:56.390713 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:56.390590    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="df26cccb-22ae-4076-8ad2-e9218f84d64d" containerName="kserve-container"
Sep 16 19:16:56.390713 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:56.390604    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="df26cccb-22ae-4076-8ad2-e9218f84d64d" containerName="kube-rbac-proxy"
Sep 16 19:16:56.393606 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:56.393584    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp"
Sep 16 19:16:56.396866 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:56.396842    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-4a90a-predictor-serving-cert\""
Sep 16 19:16:56.397101 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:56.397075    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-4a90a-kube-rbac-proxy-sar-config\""
Sep 16 19:16:56.415177 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:56.415154    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp"]
Sep 16 19:16:56.486595 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:56.486512    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49e28776-61dd-4e5f-8072-a0c209d00bc5-proxy-tls\") pod \"success-200-isvc-4a90a-predictor-5c5f467b97-gbppp\" (UID: \"49e28776-61dd-4e5f-8072-a0c209d00bc5\") " pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp"
Sep 16 19:16:56.486595 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:56.486568    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/49e28776-61dd-4e5f-8072-a0c209d00bc5-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-4a90a-predictor-5c5f467b97-gbppp\" (UID: \"49e28776-61dd-4e5f-8072-a0c209d00bc5\") " pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp"
Sep 16 19:16:56.486595 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:56.486588    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkgrw\" (UniqueName: \"kubernetes.io/projected/49e28776-61dd-4e5f-8072-a0c209d00bc5-kube-api-access-jkgrw\") pod \"success-200-isvc-4a90a-predictor-5c5f467b97-gbppp\" (UID: \"49e28776-61dd-4e5f-8072-a0c209d00bc5\") " pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp"
Sep 16 19:16:56.550974 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:56.550930    2557 generic.go:356] "Generic (PLEG): container finished" podID="d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e" containerID="66850764cbe86e7f63bc9a897b6983d657b7572640d9b872720aefc915f3c736" exitCode=2
Sep 16 19:16:56.551133 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:56.550990    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf" event={"ID":"d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e","Type":"ContainerDied","Data":"66850764cbe86e7f63bc9a897b6983d657b7572640d9b872720aefc915f3c736"}
Sep 16 19:16:56.588094 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:56.588052    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/49e28776-61dd-4e5f-8072-a0c209d00bc5-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-4a90a-predictor-5c5f467b97-gbppp\" (UID: \"49e28776-61dd-4e5f-8072-a0c209d00bc5\") " pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp"
Sep 16 19:16:56.588325 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:56.588099    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkgrw\" (UniqueName: \"kubernetes.io/projected/49e28776-61dd-4e5f-8072-a0c209d00bc5-kube-api-access-jkgrw\") pod \"success-200-isvc-4a90a-predictor-5c5f467b97-gbppp\" (UID: \"49e28776-61dd-4e5f-8072-a0c209d00bc5\") " pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp"
Sep 16 19:16:56.588325 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:56.588176    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49e28776-61dd-4e5f-8072-a0c209d00bc5-proxy-tls\") pod \"success-200-isvc-4a90a-predictor-5c5f467b97-gbppp\" (UID: \"49e28776-61dd-4e5f-8072-a0c209d00bc5\") " pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp"
Sep 16 19:16:56.588713 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:56.588690    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/49e28776-61dd-4e5f-8072-a0c209d00bc5-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-4a90a-predictor-5c5f467b97-gbppp\" (UID: \"49e28776-61dd-4e5f-8072-a0c209d00bc5\") " pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp"
Sep 16 19:16:56.590737 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:56.590717    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49e28776-61dd-4e5f-8072-a0c209d00bc5-proxy-tls\") pod \"success-200-isvc-4a90a-predictor-5c5f467b97-gbppp\" (UID: \"49e28776-61dd-4e5f-8072-a0c209d00bc5\") " pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp"
Sep 16 19:16:56.600857 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:56.600832    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkgrw\" (UniqueName: \"kubernetes.io/projected/49e28776-61dd-4e5f-8072-a0c209d00bc5-kube-api-access-jkgrw\") pod \"success-200-isvc-4a90a-predictor-5c5f467b97-gbppp\" (UID: \"49e28776-61dd-4e5f-8072-a0c209d00bc5\") " pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp"
Sep 16 19:16:56.708521 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:56.708488    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp"
Sep 16 19:16:56.996097 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:56.996067    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp"]
Sep 16 19:16:56.997081 ip-10-0-130-26 kubenswrapper[2557]: W0916 19:16:56.997058    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49e28776_61dd_4e5f_8072_a0c209d00bc5.slice/crio-c7592378304fe8e51ced1ff1b729d163f12582ba2cd9d71d4ba73737e13d741b WatchSource:0}: Error finding container c7592378304fe8e51ced1ff1b729d163f12582ba2cd9d71d4ba73737e13d741b: Status 404 returned error can't find the container with id c7592378304fe8e51ced1ff1b729d163f12582ba2cd9d71d4ba73737e13d741b
Sep 16 19:16:57.558660 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:57.558613    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp" event={"ID":"49e28776-61dd-4e5f-8072-a0c209d00bc5","Type":"ContainerStarted","Data":"6e5b2bd405a44c7c5c708c0af317577c445b5b33061dcfa65830057454b4bda4"}
Sep 16 19:16:57.559107 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:57.558799    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp"
Sep 16 19:16:57.559107 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:57.558856    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp" event={"ID":"49e28776-61dd-4e5f-8072-a0c209d00bc5","Type":"ContainerStarted","Data":"8d34d0960e34608b934ecb80b63dee777e1144f5c8a22edf3567e4c87cf93656"}
Sep 16 19:16:57.559107 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:57.558888    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp" event={"ID":"49e28776-61dd-4e5f-8072-a0c209d00bc5","Type":"ContainerStarted","Data":"c7592378304fe8e51ced1ff1b729d163f12582ba2cd9d71d4ba73737e13d741b"}
Sep 16 19:16:57.580689 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:57.580639    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp" podStartSLOduration=1.580624526 podStartE2EDuration="1.580624526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:16:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:16:57.578515829 +0000 UTC m=+1931.655620690" watchObservedRunningTime="2026-09-16 19:16:57.580624526 +0000 UTC m=+1931.657729385"
Sep 16 19:16:58.564094 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:58.564058    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp"
Sep 16 19:16:58.565029 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:58.565001    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp" podUID="49e28776-61dd-4e5f-8072-a0c209d00bc5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Sep 16 19:16:59.274107 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:59.274065    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf" podUID="d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.35:8643/healthz\": dial tcp 10.134.0.35:8643: connect: connection refused"
Sep 16 19:16:59.570680 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:59.570648    2557 generic.go:356] "Generic (PLEG): container finished" podID="d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e" containerID="ddffb19acb56a430aaf9dc9e4b52045b52469dc7bcdb66227b8d92556b54782f" exitCode=0
Sep 16 19:16:59.571109 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:59.570715    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf" event={"ID":"d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e","Type":"ContainerDied","Data":"ddffb19acb56a430aaf9dc9e4b52045b52469dc7bcdb66227b8d92556b54782f"}
Sep 16 19:16:59.571109 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:59.571001    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp" podUID="49e28776-61dd-4e5f-8072-a0c209d00bc5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Sep 16 19:16:59.693894 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:59.693872    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf"
Sep 16 19:16:59.818094 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:59.818006    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e-kube-rbac-proxy-sar-config\") pod \"d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e\" (UID: \"d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e\") "
Sep 16 19:16:59.818094 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:59.818055    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84pv4\" (UniqueName: \"kubernetes.io/projected/d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e-kube-api-access-84pv4\") pod \"d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e\" (UID: \"d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e\") "
Sep 16 19:16:59.818094 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:59.818083    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e-proxy-tls\") pod \"d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e\" (UID: \"d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e\") "
Sep 16 19:16:59.818384 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:59.818352    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e" (UID: "d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:16:59.820119 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:59.820087    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e-kube-api-access-84pv4" (OuterVolumeSpecName: "kube-api-access-84pv4") pod "d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e" (UID: "d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e"). InnerVolumeSpecName "kube-api-access-84pv4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:16:59.820119 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:59.820106    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e" (UID: "d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:16:59.919583 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:59.919554    2557 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 19:16:59.919583 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:59.919579    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84pv4\" (UniqueName: \"kubernetes.io/projected/d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e-kube-api-access-84pv4\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 19:16:59.919583 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:16:59.919589    2557 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e-proxy-tls\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 19:17:00.576224 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:17:00.576181    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf" event={"ID":"d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e","Type":"ContainerDied","Data":"d0b5c8fc6cf0bb53884e674d4af5a41c4198949afbc3f64664381de7756f0361"}
Sep 16 19:17:00.576224 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:17:00.576222    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf"
Sep 16 19:17:00.576692 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:17:00.576255    2557 scope.go:117] "RemoveContainer" containerID="66850764cbe86e7f63bc9a897b6983d657b7572640d9b872720aefc915f3c736"
Sep 16 19:17:00.584475 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:17:00.584453    2557 scope.go:117] "RemoveContainer" containerID="ddffb19acb56a430aaf9dc9e4b52045b52469dc7bcdb66227b8d92556b54782f"
Sep 16 19:17:00.594581 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:17:00.594556    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf"]
Sep 16 19:17:00.600957 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:17:00.600935    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-da3eb-predictor-5d58fc86fc-fchjf"]
Sep 16 19:17:02.466433 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:17:02.466403    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e" path="/var/lib/kubelet/pods/d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e/volumes"
Sep 16 19:17:04.446004 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:17:04.445964    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k" podUID="ef314c3b-f862-4b89-8a16-633b13991399" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Sep 16 19:17:04.576108 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:17:04.576079    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp"
Sep 16 19:17:04.576590 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:17:04.576563    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp" podUID="49e28776-61dd-4e5f-8072-a0c209d00bc5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Sep 16 19:17:14.446364 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:17:14.446334    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k"
Sep 16 19:17:14.577429 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:17:14.577389    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp" podUID="49e28776-61dd-4e5f-8072-a0c209d00bc5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Sep 16 19:17:24.576958 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:17:24.576918    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp" podUID="49e28776-61dd-4e5f-8072-a0c209d00bc5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Sep 16 19:17:34.577227 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:17:34.577189    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp" podUID="49e28776-61dd-4e5f-8072-a0c209d00bc5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Sep 16 19:17:44.577519 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:17:44.577489    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp"
Sep 16 19:26:11.091999 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:26:11.091963    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp"]
Sep 16 19:26:11.094854 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:26:11.092256    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp" podUID="49e28776-61dd-4e5f-8072-a0c209d00bc5" containerName="kserve-container" containerID="cri-o://8d34d0960e34608b934ecb80b63dee777e1144f5c8a22edf3567e4c87cf93656" gracePeriod=30
Sep 16 19:26:11.094854 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:26:11.092298    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp" podUID="49e28776-61dd-4e5f-8072-a0c209d00bc5" containerName="kube-rbac-proxy" containerID="cri-o://6e5b2bd405a44c7c5c708c0af317577c445b5b33061dcfa65830057454b4bda4" gracePeriod=30
Sep 16 19:26:11.738741 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:26:11.738704    2557 generic.go:356] "Generic (PLEG): container finished" podID="49e28776-61dd-4e5f-8072-a0c209d00bc5" containerID="6e5b2bd405a44c7c5c708c0af317577c445b5b33061dcfa65830057454b4bda4" exitCode=2
Sep 16 19:26:11.738931 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:26:11.738760    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp" event={"ID":"49e28776-61dd-4e5f-8072-a0c209d00bc5","Type":"ContainerDied","Data":"6e5b2bd405a44c7c5c708c0af317577c445b5b33061dcfa65830057454b4bda4"}
Sep 16 19:26:14.038317 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:26:14.038292    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp"
Sep 16 19:26:14.163442 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:26:14.163352    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49e28776-61dd-4e5f-8072-a0c209d00bc5-proxy-tls\") pod \"49e28776-61dd-4e5f-8072-a0c209d00bc5\" (UID: \"49e28776-61dd-4e5f-8072-a0c209d00bc5\") "
Sep 16 19:26:14.163442 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:26:14.163437    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/49e28776-61dd-4e5f-8072-a0c209d00bc5-kube-rbac-proxy-sar-config\") pod \"49e28776-61dd-4e5f-8072-a0c209d00bc5\" (UID: \"49e28776-61dd-4e5f-8072-a0c209d00bc5\") "
Sep 16 19:26:14.163681 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:26:14.163560    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkgrw\" (UniqueName: \"kubernetes.io/projected/49e28776-61dd-4e5f-8072-a0c209d00bc5-kube-api-access-jkgrw\") pod \"49e28776-61dd-4e5f-8072-a0c209d00bc5\" (UID: \"49e28776-61dd-4e5f-8072-a0c209d00bc5\") "
Sep 16 19:26:14.163738 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:26:14.163704    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49e28776-61dd-4e5f-8072-a0c209d00bc5-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "49e28776-61dd-4e5f-8072-a0c209d00bc5" (UID: "49e28776-61dd-4e5f-8072-a0c209d00bc5"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:26:14.163884 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:26:14.163861    2557 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/49e28776-61dd-4e5f-8072-a0c209d00bc5-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 19:26:14.165379 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:26:14.165359    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49e28776-61dd-4e5f-8072-a0c209d00bc5-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "49e28776-61dd-4e5f-8072-a0c209d00bc5" (UID: "49e28776-61dd-4e5f-8072-a0c209d00bc5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:26:14.165482 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:26:14.165457    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49e28776-61dd-4e5f-8072-a0c209d00bc5-kube-api-access-jkgrw" (OuterVolumeSpecName: "kube-api-access-jkgrw") pod "49e28776-61dd-4e5f-8072-a0c209d00bc5" (UID: "49e28776-61dd-4e5f-8072-a0c209d00bc5"). InnerVolumeSpecName "kube-api-access-jkgrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:26:14.264577 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:26:14.264524    2557 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49e28776-61dd-4e5f-8072-a0c209d00bc5-proxy-tls\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 19:26:14.264577 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:26:14.264571    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkgrw\" (UniqueName: \"kubernetes.io/projected/49e28776-61dd-4e5f-8072-a0c209d00bc5-kube-api-access-jkgrw\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 19:26:14.752187 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:26:14.752094    2557 generic.go:356] "Generic (PLEG): container finished" podID="49e28776-61dd-4e5f-8072-a0c209d00bc5" containerID="8d34d0960e34608b934ecb80b63dee777e1144f5c8a22edf3567e4c87cf93656" exitCode=0
Sep 16 19:26:14.752187 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:26:14.752157    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp"
Sep 16 19:26:14.752362 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:26:14.752180    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp" event={"ID":"49e28776-61dd-4e5f-8072-a0c209d00bc5","Type":"ContainerDied","Data":"8d34d0960e34608b934ecb80b63dee777e1144f5c8a22edf3567e4c87cf93656"}
Sep 16 19:26:14.752362 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:26:14.752244    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp" event={"ID":"49e28776-61dd-4e5f-8072-a0c209d00bc5","Type":"ContainerDied","Data":"c7592378304fe8e51ced1ff1b729d163f12582ba2cd9d71d4ba73737e13d741b"}
Sep 16 19:26:14.752362 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:26:14.752272    2557 scope.go:117] "RemoveContainer" containerID="6e5b2bd405a44c7c5c708c0af317577c445b5b33061dcfa65830057454b4bda4"
Sep 16 19:26:14.760246 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:26:14.760229    2557 scope.go:117] "RemoveContainer" containerID="8d34d0960e34608b934ecb80b63dee777e1144f5c8a22edf3567e4c87cf93656"
Sep 16 19:26:14.767153 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:26:14.767124    2557 scope.go:117] "RemoveContainer" containerID="6e5b2bd405a44c7c5c708c0af317577c445b5b33061dcfa65830057454b4bda4"
Sep 16 19:26:14.767752 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:26:14.767624    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e5b2bd405a44c7c5c708c0af317577c445b5b33061dcfa65830057454b4bda4\": container with ID starting with 6e5b2bd405a44c7c5c708c0af317577c445b5b33061dcfa65830057454b4bda4 not found: ID does not exist" containerID="6e5b2bd405a44c7c5c708c0af317577c445b5b33061dcfa65830057454b4bda4"
Sep 16 19:26:14.767752 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:26:14.767680    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e5b2bd405a44c7c5c708c0af317577c445b5b33061dcfa65830057454b4bda4"} err="failed to get container status \"6e5b2bd405a44c7c5c708c0af317577c445b5b33061dcfa65830057454b4bda4\": rpc error: code = NotFound desc = could not find container \"6e5b2bd405a44c7c5c708c0af317577c445b5b33061dcfa65830057454b4bda4\": container with ID starting with 6e5b2bd405a44c7c5c708c0af317577c445b5b33061dcfa65830057454b4bda4 not found: ID does not exist"
Sep 16 19:26:14.767752 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:26:14.767708    2557 scope.go:117] "RemoveContainer" containerID="8d34d0960e34608b934ecb80b63dee777e1144f5c8a22edf3567e4c87cf93656"
Sep 16 19:26:14.768216 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:26:14.768182    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d34d0960e34608b934ecb80b63dee777e1144f5c8a22edf3567e4c87cf93656\": container with ID starting with 8d34d0960e34608b934ecb80b63dee777e1144f5c8a22edf3567e4c87cf93656 not found: ID does not exist" containerID="8d34d0960e34608b934ecb80b63dee777e1144f5c8a22edf3567e4c87cf93656"
Sep 16 19:26:14.768305 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:26:14.768221    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d34d0960e34608b934ecb80b63dee777e1144f5c8a22edf3567e4c87cf93656"} err="failed to get container status \"8d34d0960e34608b934ecb80b63dee777e1144f5c8a22edf3567e4c87cf93656\": rpc error: code = NotFound desc = could not find container \"8d34d0960e34608b934ecb80b63dee777e1144f5c8a22edf3567e4c87cf93656\": container with ID starting with 8d34d0960e34608b934ecb80b63dee777e1144f5c8a22edf3567e4c87cf93656 not found: ID does not exist"
Sep 16 19:26:14.769728 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:26:14.769704    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp"]
Sep 16 19:26:14.776450 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:26:14.776424    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4a90a-predictor-5c5f467b97-gbppp"]
Sep 16 19:26:16.465611 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:26:16.465566    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49e28776-61dd-4e5f-8072-a0c209d00bc5" path="/var/lib/kubelet/pods/49e28776-61dd-4e5f-8072-a0c209d00bc5/volumes"
Sep 16 19:33:45.318050 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:45.318014    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k"]
Sep 16 19:33:45.320906 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:45.318320    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k" podUID="ef314c3b-f862-4b89-8a16-633b13991399" containerName="kserve-container" containerID="cri-o://fb299558856136d10b17cefba6c4e0f1c8be4d873c65085e08211104c43a9668" gracePeriod=30
Sep 16 19:33:45.320906 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:45.318401    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k" podUID="ef314c3b-f862-4b89-8a16-633b13991399" containerName="kube-rbac-proxy" containerID="cri-o://4eac47705a0a6b6e6247dc1c5084824a35e6e37ce5cf06352ba810a2cd0cc5ba" gracePeriod=30
Sep 16 19:33:45.494519 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:45.494485    2557 generic.go:356] "Generic (PLEG): container finished" podID="ef314c3b-f862-4b89-8a16-633b13991399" containerID="4eac47705a0a6b6e6247dc1c5084824a35e6e37ce5cf06352ba810a2cd0cc5ba" exitCode=2
Sep 16 19:33:45.494667 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:45.494540    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k" event={"ID":"ef314c3b-f862-4b89-8a16-633b13991399","Type":"ContainerDied","Data":"4eac47705a0a6b6e6247dc1c5084824a35e6e37ce5cf06352ba810a2cd0cc5ba"}
Sep 16 19:33:47.036926 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:47.036891    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lz8tg/must-gather-wnj9p"]
Sep 16 19:33:47.037283 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:47.037200    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49e28776-61dd-4e5f-8072-a0c209d00bc5" containerName="kube-rbac-proxy"
Sep 16 19:33:47.037283 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:47.037211    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="49e28776-61dd-4e5f-8072-a0c209d00bc5" containerName="kube-rbac-proxy"
Sep 16 19:33:47.037283 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:47.037222    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e" containerName="kserve-container"
Sep 16 19:33:47.037283 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:47.037227    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e" containerName="kserve-container"
Sep 16 19:33:47.037283 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:47.037242    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e" containerName="kube-rbac-proxy"
Sep 16 19:33:47.037283 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:47.037247    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e" containerName="kube-rbac-proxy"
Sep 16 19:33:47.037283 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:47.037258    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49e28776-61dd-4e5f-8072-a0c209d00bc5" containerName="kserve-container"
Sep 16 19:33:47.037283 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:47.037264    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="49e28776-61dd-4e5f-8072-a0c209d00bc5" containerName="kserve-container"
Sep 16 19:33:47.037532 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:47.037309    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e" containerName="kserve-container"
Sep 16 19:33:47.037532 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:47.037318    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4457dd2-6c6b-42d4-b9f9-1cb5e2f35d0e" containerName="kube-rbac-proxy"
Sep 16 19:33:47.037532 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:47.037325    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="49e28776-61dd-4e5f-8072-a0c209d00bc5" containerName="kube-rbac-proxy"
Sep 16 19:33:47.037532 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:47.037332    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="49e28776-61dd-4e5f-8072-a0c209d00bc5" containerName="kserve-container"
Sep 16 19:33:47.040319 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:47.040303    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lz8tg/must-gather-wnj9p"
Sep 16 19:33:47.054256 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:33:47.054228    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-wnj9p\" is forbidden: User \"system:node:ip-10-0-130-26.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-lz8tg\": no relationship found between node 'ip-10-0-130-26.ec2.internal' and this object" podUID="3d0d7e7a-6226-4445-bb0e-71d30c3cb971" pod="openshift-must-gather-lz8tg/must-gather-wnj9p"
Sep 16 19:33:47.054384 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:33:47.054246    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-130-26.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-must-gather-lz8tg\": no relationship found between node 'ip-10-0-130-26.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-must-gather-lz8tg\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Sep 16 19:33:47.054384 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:33:47.054249    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-130-26.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-must-gather-lz8tg\": no relationship found between node 'ip-10-0-130-26.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-must-gather-lz8tg\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Sep 16 19:33:47.069759 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:47.069734    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq6x9\" (UniqueName: \"kubernetes.io/projected/3d0d7e7a-6226-4445-bb0e-71d30c3cb971-kube-api-access-wq6x9\") pod \"must-gather-wnj9p\" (UID: \"3d0d7e7a-6226-4445-bb0e-71d30c3cb971\") " pod="openshift-must-gather-lz8tg/must-gather-wnj9p"
Sep 16 19:33:47.069873 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:47.069817    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3d0d7e7a-6226-4445-bb0e-71d30c3cb971-must-gather-output\") pod \"must-gather-wnj9p\" (UID: \"3d0d7e7a-6226-4445-bb0e-71d30c3cb971\") " pod="openshift-must-gather-lz8tg/must-gather-wnj9p"
Sep 16 19:33:47.093303 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:47.093274    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lz8tg/must-gather-wnj9p"]
Sep 16 19:33:47.170748 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:47.170715    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq6x9\" (UniqueName: \"kubernetes.io/projected/3d0d7e7a-6226-4445-bb0e-71d30c3cb971-kube-api-access-wq6x9\") pod \"must-gather-wnj9p\" (UID: \"3d0d7e7a-6226-4445-bb0e-71d30c3cb971\") " pod="openshift-must-gather-lz8tg/must-gather-wnj9p"
Sep 16 19:33:47.170916 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:47.170765    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3d0d7e7a-6226-4445-bb0e-71d30c3cb971-must-gather-output\") pod \"must-gather-wnj9p\" (UID: \"3d0d7e7a-6226-4445-bb0e-71d30c3cb971\") " pod="openshift-must-gather-lz8tg/must-gather-wnj9p"
Sep 16 19:33:47.171062 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:47.171047    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3d0d7e7a-6226-4445-bb0e-71d30c3cb971-must-gather-output\") pod \"must-gather-wnj9p\" (UID: \"3d0d7e7a-6226-4445-bb0e-71d30c3cb971\") " pod="openshift-must-gather-lz8tg/must-gather-wnj9p"
Sep 16 19:33:48.078152 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:48.078119    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lz8tg\"/\"openshift-service-ca.crt\""
Sep 16 19:33:48.178473 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:33:48.178445    2557 projected.go:291] Couldn't get configMap openshift-must-gather-lz8tg/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition
Sep 16 19:33:48.178473 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:33:48.178480    2557 projected.go:196] Error preparing data for projected volume kube-api-access-wq6x9 for pod openshift-must-gather-lz8tg/must-gather-wnj9p: failed to sync configmap cache: timed out waiting for the condition
Sep 16 19:33:48.178651 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:33:48.178536    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3d0d7e7a-6226-4445-bb0e-71d30c3cb971-kube-api-access-wq6x9 podName:3d0d7e7a-6226-4445-bb0e-71d30c3cb971 nodeName:}" failed. No retries permitted until 2026-09-16 19:33:48.678520467 +0000 UTC m=+2942.755625305 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wq6x9" (UniqueName: "kubernetes.io/projected/3d0d7e7a-6226-4445-bb0e-71d30c3cb971-kube-api-access-wq6x9") pod "must-gather-wnj9p" (UID: "3d0d7e7a-6226-4445-bb0e-71d30c3cb971") : failed to sync configmap cache: timed out waiting for the condition
Sep 16 19:33:48.450148 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:48.450075    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lz8tg\"/\"kube-root-ca.crt\""
Sep 16 19:33:48.681359 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:48.681321    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq6x9\" (UniqueName: \"kubernetes.io/projected/3d0d7e7a-6226-4445-bb0e-71d30c3cb971-kube-api-access-wq6x9\") pod \"must-gather-wnj9p\" (UID: \"3d0d7e7a-6226-4445-bb0e-71d30c3cb971\") " pod="openshift-must-gather-lz8tg/must-gather-wnj9p"
Sep 16 19:33:48.683620 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:48.683599    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq6x9\" (UniqueName: \"kubernetes.io/projected/3d0d7e7a-6226-4445-bb0e-71d30c3cb971-kube-api-access-wq6x9\") pod \"must-gather-wnj9p\" (UID: \"3d0d7e7a-6226-4445-bb0e-71d30c3cb971\") " pod="openshift-must-gather-lz8tg/must-gather-wnj9p"
Sep 16 19:33:48.856807 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:48.856755    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lz8tg/must-gather-wnj9p"
Sep 16 19:33:49.126453 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:49.126428    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lz8tg/must-gather-wnj9p"]
Sep 16 19:33:49.128575 ip-10-0-130-26 kubenswrapper[2557]: W0916 19:33:49.128549    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d0d7e7a_6226_4445_bb0e_71d30c3cb971.slice/crio-a9aa13ffdf38e7241af731c30302ee818f0cfd8b4d37b520f9e48788a05de146 WatchSource:0}: Error finding container a9aa13ffdf38e7241af731c30302ee818f0cfd8b4d37b520f9e48788a05de146: Status 404 returned error can't find the container with id a9aa13ffdf38e7241af731c30302ee818f0cfd8b4d37b520f9e48788a05de146
Sep 16 19:33:49.130468 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:49.130446    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 19:33:49.442426 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:49.442334    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k" podUID="ef314c3b-f862-4b89-8a16-633b13991399" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.36:8643/healthz\": dial tcp 10.134.0.36:8643: connect: connection refused"
Sep 16 19:33:49.510121 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:49.510090    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lz8tg/must-gather-wnj9p" event={"ID":"3d0d7e7a-6226-4445-bb0e-71d30c3cb971","Type":"ContainerStarted","Data":"a9aa13ffdf38e7241af731c30302ee818f0cfd8b4d37b520f9e48788a05de146"}
Sep 16 19:33:53.532905 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:53.532856    2557 generic.go:356] "Generic (PLEG): container finished" podID="ef314c3b-f862-4b89-8a16-633b13991399" containerID="fb299558856136d10b17cefba6c4e0f1c8be4d873c65085e08211104c43a9668" exitCode=0
Sep 16 19:33:53.533429 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:53.532959    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k" event={"ID":"ef314c3b-f862-4b89-8a16-633b13991399","Type":"ContainerDied","Data":"fb299558856136d10b17cefba6c4e0f1c8be4d873c65085e08211104c43a9668"}
Sep 16 19:33:55.003169 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:55.003138    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k"
Sep 16 19:33:55.140681 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:55.140641    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ef314c3b-f862-4b89-8a16-633b13991399-kube-rbac-proxy-sar-config\") pod \"ef314c3b-f862-4b89-8a16-633b13991399\" (UID: \"ef314c3b-f862-4b89-8a16-633b13991399\") "
Sep 16 19:33:55.140681 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:55.140688    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ef314c3b-f862-4b89-8a16-633b13991399-proxy-tls\") pod \"ef314c3b-f862-4b89-8a16-633b13991399\" (UID: \"ef314c3b-f862-4b89-8a16-633b13991399\") "
Sep 16 19:33:55.140964 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:55.140763    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85gz2\" (UniqueName: \"kubernetes.io/projected/ef314c3b-f862-4b89-8a16-633b13991399-kube-api-access-85gz2\") pod \"ef314c3b-f862-4b89-8a16-633b13991399\" (UID: \"ef314c3b-f862-4b89-8a16-633b13991399\") "
Sep 16 19:33:55.141115 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:55.141082    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef314c3b-f862-4b89-8a16-633b13991399-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "ef314c3b-f862-4b89-8a16-633b13991399" (UID: "ef314c3b-f862-4b89-8a16-633b13991399"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:33:55.143174 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:55.143138    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef314c3b-f862-4b89-8a16-633b13991399-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ef314c3b-f862-4b89-8a16-633b13991399" (UID: "ef314c3b-f862-4b89-8a16-633b13991399"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:33:55.143293 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:55.143229    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef314c3b-f862-4b89-8a16-633b13991399-kube-api-access-85gz2" (OuterVolumeSpecName: "kube-api-access-85gz2") pod "ef314c3b-f862-4b89-8a16-633b13991399" (UID: "ef314c3b-f862-4b89-8a16-633b13991399"). InnerVolumeSpecName "kube-api-access-85gz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:33:55.241734 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:55.241669    2557 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ef314c3b-f862-4b89-8a16-633b13991399-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 19:33:55.241734 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:55.241698    2557 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ef314c3b-f862-4b89-8a16-633b13991399-proxy-tls\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 19:33:55.241734 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:55.241709    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85gz2\" (UniqueName: \"kubernetes.io/projected/ef314c3b-f862-4b89-8a16-633b13991399-kube-api-access-85gz2\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 19:33:55.442172 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:55.442118    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k" podUID="ef314c3b-f862-4b89-8a16-633b13991399" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.36:8643/healthz\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Sep 16 19:33:55.446436 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:55.446400    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k" podUID="ef314c3b-f862-4b89-8a16-633b13991399" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: i/o timeout"
Sep 16 19:33:55.544443 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:55.544401    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lz8tg/must-gather-wnj9p" event={"ID":"3d0d7e7a-6226-4445-bb0e-71d30c3cb971","Type":"ContainerStarted","Data":"d8706cdb8c333983b18ca43da771633df39435f9612f4f2cde560c7d40b2af83"}
Sep 16 19:33:55.544622 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:55.544458    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lz8tg/must-gather-wnj9p" event={"ID":"3d0d7e7a-6226-4445-bb0e-71d30c3cb971","Type":"ContainerStarted","Data":"1c856744debc2ffd4bc2669b1afbaa87e92b75093673c831319a538fe8a2860e"}
Sep 16 19:33:55.546132 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:55.546110    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k"
Sep 16 19:33:55.546364 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:55.546136    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k" event={"ID":"ef314c3b-f862-4b89-8a16-633b13991399","Type":"ContainerDied","Data":"4d6b4185757e635a2bd27f3e4ec685ac0fd4510e70391b4564508e3524909cc7"}
Sep 16 19:33:55.546364 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:55.546192    2557 scope.go:117] "RemoveContainer" containerID="4eac47705a0a6b6e6247dc1c5084824a35e6e37ce5cf06352ba810a2cd0cc5ba"
Sep 16 19:33:55.555285 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:55.555264    2557 scope.go:117] "RemoveContainer" containerID="fb299558856136d10b17cefba6c4e0f1c8be4d873c65085e08211104c43a9668"
Sep 16 19:33:55.566074 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:55.566035    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-lz8tg/must-gather-wnj9p" podStartSLOduration=3.427344018 podStartE2EDuration="9.566023105s" totalImagesPullingTime="6.138679087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:33:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:33:55.565747962 +0000 UTC m=+2949.642852821" watchObservedRunningTime="2026-09-16 19:33:55.566023105 +0000 UTC m=+2949.643127965"
Sep 16 19:33:55.582449 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:55.582418    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k"]
Sep 16 19:33:55.588339 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:55.588312    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ca22e-predictor-558b7cd765-n4t6k"]
Sep 16 19:33:56.472544 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:33:56.472507    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef314c3b-f862-4b89-8a16-633b13991399" path="/var/lib/kubelet/pods/ef314c3b-f862-4b89-8a16-633b13991399/volumes"
Sep 16 19:34:17.642601 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:17.642560    2557 generic.go:356] "Generic (PLEG): container finished" podID="3d0d7e7a-6226-4445-bb0e-71d30c3cb971" containerID="1c856744debc2ffd4bc2669b1afbaa87e92b75093673c831319a538fe8a2860e" exitCode=0
Sep 16 19:34:17.643028 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:17.642618    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lz8tg/must-gather-wnj9p" event={"ID":"3d0d7e7a-6226-4445-bb0e-71d30c3cb971","Type":"ContainerDied","Data":"1c856744debc2ffd4bc2669b1afbaa87e92b75093673c831319a538fe8a2860e"}
Sep 16 19:34:17.643028 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:17.642923    2557 scope.go:117] "RemoveContainer" containerID="1c856744debc2ffd4bc2669b1afbaa87e92b75093673c831319a538fe8a2860e"
Sep 16 19:34:17.690726 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:17.690696    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-lz8tg_must-gather-wnj9p_3d0d7e7a-6226-4445-bb0e-71d30c3cb971/gather/0.log"
Sep 16 19:34:21.113380 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:21.113349    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-c2j6m_b7d1f1fc-4f58-4499-a42b-1f9325efa292/global-pull-secret-syncer/0.log"
Sep 16 19:34:21.341739 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:21.341706    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-r9g58_4c1c55c5-0ed8-46eb-a61d-381273852b77/konnectivity-agent/0.log"
Sep 16 19:34:21.409119 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:21.409051    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-26.ec2.internal_26c1c5215cb05aa10e9d3bcd9724a09a/haproxy/0.log"
Sep 16 19:34:23.106750 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:23.106709    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-lz8tg/must-gather-wnj9p"]
Sep 16 19:34:23.107294 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:23.106955    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-lz8tg/must-gather-wnj9p" podUID="3d0d7e7a-6226-4445-bb0e-71d30c3cb971" containerName="copy" containerID="cri-o://d8706cdb8c333983b18ca43da771633df39435f9612f4f2cde560c7d40b2af83" gracePeriod=2
Sep 16 19:34:23.113096 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:23.113073    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-lz8tg/must-gather-wnj9p"]
Sep 16 19:34:23.338446 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:23.338426    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-lz8tg_must-gather-wnj9p_3d0d7e7a-6226-4445-bb0e-71d30c3cb971/copy/0.log"
Sep 16 19:34:23.341067 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:23.341046    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lz8tg/must-gather-wnj9p"
Sep 16 19:34:23.346171 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:34:23.346147    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-wnj9p\" is forbidden: User \"system:node:ip-10-0-130-26.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-lz8tg\": no relationship found between node 'ip-10-0-130-26.ec2.internal' and this object" podUID="3d0d7e7a-6226-4445-bb0e-71d30c3cb971" pod="openshift-must-gather-lz8tg/must-gather-wnj9p"
Sep 16 19:34:23.487281 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:23.487191    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3d0d7e7a-6226-4445-bb0e-71d30c3cb971-must-gather-output\") pod \"3d0d7e7a-6226-4445-bb0e-71d30c3cb971\" (UID: \"3d0d7e7a-6226-4445-bb0e-71d30c3cb971\") "
Sep 16 19:34:23.487458 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:23.487299    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wq6x9\" (UniqueName: \"kubernetes.io/projected/3d0d7e7a-6226-4445-bb0e-71d30c3cb971-kube-api-access-wq6x9\") pod \"3d0d7e7a-6226-4445-bb0e-71d30c3cb971\" (UID: \"3d0d7e7a-6226-4445-bb0e-71d30c3cb971\") "
Sep 16 19:34:23.488743 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:23.488715    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d0d7e7a-6226-4445-bb0e-71d30c3cb971-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "3d0d7e7a-6226-4445-bb0e-71d30c3cb971" (UID: "3d0d7e7a-6226-4445-bb0e-71d30c3cb971"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:34:23.489600 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:23.489578    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d0d7e7a-6226-4445-bb0e-71d30c3cb971-kube-api-access-wq6x9" (OuterVolumeSpecName: "kube-api-access-wq6x9") pod "3d0d7e7a-6226-4445-bb0e-71d30c3cb971" (UID: "3d0d7e7a-6226-4445-bb0e-71d30c3cb971"). InnerVolumeSpecName "kube-api-access-wq6x9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:34:23.588175 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:23.588143    2557 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3d0d7e7a-6226-4445-bb0e-71d30c3cb971-must-gather-output\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 19:34:23.588175 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:23.588171    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wq6x9\" (UniqueName: \"kubernetes.io/projected/3d0d7e7a-6226-4445-bb0e-71d30c3cb971-kube-api-access-wq6x9\") on node \"ip-10-0-130-26.ec2.internal\" DevicePath \"\""
Sep 16 19:34:23.666495 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:23.666466    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-lz8tg_must-gather-wnj9p_3d0d7e7a-6226-4445-bb0e-71d30c3cb971/copy/0.log"
Sep 16 19:34:23.666912 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:23.666887    2557 generic.go:356] "Generic (PLEG): container finished" podID="3d0d7e7a-6226-4445-bb0e-71d30c3cb971" containerID="d8706cdb8c333983b18ca43da771633df39435f9612f4f2cde560c7d40b2af83" exitCode=143
Sep 16 19:34:23.667025 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:23.666947    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lz8tg/must-gather-wnj9p"
Sep 16 19:34:23.667025 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:23.667001    2557 scope.go:117] "RemoveContainer" containerID="d8706cdb8c333983b18ca43da771633df39435f9612f4f2cde560c7d40b2af83"
Sep 16 19:34:23.669989 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:34:23.669959    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-wnj9p\" is forbidden: User \"system:node:ip-10-0-130-26.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-lz8tg\": no relationship found between node 'ip-10-0-130-26.ec2.internal' and this object" podUID="3d0d7e7a-6226-4445-bb0e-71d30c3cb971" pod="openshift-must-gather-lz8tg/must-gather-wnj9p"
Sep 16 19:34:23.676090 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:23.676076    2557 scope.go:117] "RemoveContainer" containerID="1c856744debc2ffd4bc2669b1afbaa87e92b75093673c831319a538fe8a2860e"
Sep 16 19:34:23.679029 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:34:23.679005    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-wnj9p\" is forbidden: User \"system:node:ip-10-0-130-26.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-lz8tg\": no relationship found between node 'ip-10-0-130-26.ec2.internal' and this object" podUID="3d0d7e7a-6226-4445-bb0e-71d30c3cb971" pod="openshift-must-gather-lz8tg/must-gather-wnj9p"
Sep 16 19:34:23.687624 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:23.687600    2557 scope.go:117] "RemoveContainer" containerID="d8706cdb8c333983b18ca43da771633df39435f9612f4f2cde560c7d40b2af83"
Sep 16 19:34:23.687924 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:34:23.687905    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8706cdb8c333983b18ca43da771633df39435f9612f4f2cde560c7d40b2af83\": container with ID starting with d8706cdb8c333983b18ca43da771633df39435f9612f4f2cde560c7d40b2af83 not found: ID does not exist" containerID="d8706cdb8c333983b18ca43da771633df39435f9612f4f2cde560c7d40b2af83"
Sep 16 19:34:23.687978 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:23.687934    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8706cdb8c333983b18ca43da771633df39435f9612f4f2cde560c7d40b2af83"} err="failed to get container status \"d8706cdb8c333983b18ca43da771633df39435f9612f4f2cde560c7d40b2af83\": rpc error: code = NotFound desc = could not find container \"d8706cdb8c333983b18ca43da771633df39435f9612f4f2cde560c7d40b2af83\": container with ID starting with d8706cdb8c333983b18ca43da771633df39435f9612f4f2cde560c7d40b2af83 not found: ID does not exist"
Sep 16 19:34:23.687978 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:23.687950    2557 scope.go:117] "RemoveContainer" containerID="1c856744debc2ffd4bc2669b1afbaa87e92b75093673c831319a538fe8a2860e"
Sep 16 19:34:23.688195 ip-10-0-130-26 kubenswrapper[2557]: E0916 19:34:23.688177    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c856744debc2ffd4bc2669b1afbaa87e92b75093673c831319a538fe8a2860e\": container with ID starting with 1c856744debc2ffd4bc2669b1afbaa87e92b75093673c831319a538fe8a2860e not found: ID does not exist" containerID="1c856744debc2ffd4bc2669b1afbaa87e92b75093673c831319a538fe8a2860e"
Sep 16 19:34:23.688248 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:23.688202    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c856744debc2ffd4bc2669b1afbaa87e92b75093673c831319a538fe8a2860e"} err="failed to get container status \"1c856744debc2ffd4bc2669b1afbaa87e92b75093673c831319a538fe8a2860e\": rpc error: code = NotFound desc = could not find container \"1c856744debc2ffd4bc2669b1afbaa87e92b75093673c831319a538fe8a2860e\": container with ID starting with 1c856744debc2ffd4bc2669b1afbaa87e92b75093673c831319a538fe8a2860e not found: ID does not exist"
Sep 16 19:34:24.465680 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:24.465644    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d0d7e7a-6226-4445-bb0e-71d30c3cb971" path="/var/lib/kubelet/pods/3d0d7e7a-6226-4445-bb0e-71d30c3cb971/volumes"
Sep 16 19:34:25.653984 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:25.653948    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-r9sr5_8c5c7bd7-cce3-4e15-8d25-07cea35438fc/kube-state-metrics/0.log"
Sep 16 19:34:25.831204 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:25.831170    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-r9sr5_8c5c7bd7-cce3-4e15-8d25-07cea35438fc/kube-rbac-proxy-main/0.log"
Sep 16 19:34:25.894191 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:25.894165    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-r9sr5_8c5c7bd7-cce3-4e15-8d25-07cea35438fc/kube-rbac-proxy-self/0.log"
Sep 16 19:34:25.944780 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:25.944743    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-68b5bf68d8-vwbqr_a388087b-93a5-410d-8fd8-d98a8ff1eb77/metrics-server/0.log"
Sep 16 19:34:25.983875 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:25.983843    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-nm7rr_a27c3da5-d71f-47bd-b483-f5f486da3a63/monitoring-plugin/0.log"
Sep 16 19:34:26.284887 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:26.284845    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xfmzw_c63581c7-f628-4dcd-9f39-a88418aff6e0/node-exporter/0.log"
Sep 16 19:34:26.310329 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:26.310308    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xfmzw_c63581c7-f628-4dcd-9f39-a88418aff6e0/kube-rbac-proxy/0.log"
Sep 16 19:34:26.332087 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:26.332065    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xfmzw_c63581c7-f628-4dcd-9f39-a88418aff6e0/init-textfile/0.log"
Sep 16 19:34:26.371265 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:26.371237    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-72c5r_2696852b-3bfd-4f06-8378-f88b8485e27a/kube-rbac-proxy-main/0.log"
Sep 16 19:34:26.398295 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:26.398271    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-72c5r_2696852b-3bfd-4f06-8378-f88b8485e27a/kube-rbac-proxy-self/0.log"
Sep 16 19:34:26.423299 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:26.423267    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-72c5r_2696852b-3bfd-4f06-8378-f88b8485e27a/openshift-state-metrics/0.log"
Sep 16 19:34:26.473088 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:26.473064    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_497bc5a6-4deb-4ca4-a954-469f358a1a01/prometheus/0.log"
Sep 16 19:34:26.494571 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:26.494548    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_497bc5a6-4deb-4ca4-a954-469f358a1a01/config-reloader/0.log"
Sep 16 19:34:26.526822 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:26.526750    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_497bc5a6-4deb-4ca4-a954-469f358a1a01/thanos-sidecar/0.log"
Sep 16 19:34:26.589936 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:26.589918    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_497bc5a6-4deb-4ca4-a954-469f358a1a01/kube-rbac-proxy-web/0.log"
Sep 16 19:34:26.649674 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:26.649648    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_497bc5a6-4deb-4ca4-a954-469f358a1a01/kube-rbac-proxy/0.log"
Sep 16 19:34:26.700365 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:26.700337    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_497bc5a6-4deb-4ca4-a954-469f358a1a01/kube-rbac-proxy-thanos/0.log"
Sep 16 19:34:26.773592 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:26.773561    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_497bc5a6-4deb-4ca4-a954-469f358a1a01/init-config-reloader/0.log"
Sep 16 19:34:29.202966 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.202924    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lpmdv/perf-node-gather-daemonset-f2vtm"]
Sep 16 19:34:29.203358 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.203229    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d0d7e7a-6226-4445-bb0e-71d30c3cb971" containerName="copy"
Sep 16 19:34:29.203358 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.203240    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d0d7e7a-6226-4445-bb0e-71d30c3cb971" containerName="copy"
Sep 16 19:34:29.203358 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.203260    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef314c3b-f862-4b89-8a16-633b13991399" containerName="kube-rbac-proxy"
Sep 16 19:34:29.203358 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.203266    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef314c3b-f862-4b89-8a16-633b13991399" containerName="kube-rbac-proxy"
Sep 16 19:34:29.203358 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.203277    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef314c3b-f862-4b89-8a16-633b13991399" containerName="kserve-container"
Sep 16 19:34:29.203358 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.203283    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef314c3b-f862-4b89-8a16-633b13991399" containerName="kserve-container"
Sep 16 19:34:29.203358 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.203289    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d0d7e7a-6226-4445-bb0e-71d30c3cb971" containerName="gather"
Sep 16 19:34:29.203358 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.203295    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d0d7e7a-6226-4445-bb0e-71d30c3cb971" containerName="gather"
Sep 16 19:34:29.203358 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.203339    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d0d7e7a-6226-4445-bb0e-71d30c3cb971" containerName="copy"
Sep 16 19:34:29.203358 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.203347    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef314c3b-f862-4b89-8a16-633b13991399" containerName="kserve-container"
Sep 16 19:34:29.203358 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.203353    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d0d7e7a-6226-4445-bb0e-71d30c3cb971" containerName="gather"
Sep 16 19:34:29.203358 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.203362    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef314c3b-f862-4b89-8a16-633b13991399" containerName="kube-rbac-proxy"
Sep 16 19:34:29.206334 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.206313    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lpmdv/perf-node-gather-daemonset-f2vtm"
Sep 16 19:34:29.212218 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.212185    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lpmdv\"/\"kube-root-ca.crt\""
Sep 16 19:34:29.212491 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.212468    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lpmdv\"/\"openshift-service-ca.crt\""
Sep 16 19:34:29.213623 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.213604    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-lpmdv\"/\"default-dockercfg-87trx\""
Sep 16 19:34:29.220858 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.220835    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lpmdv/perf-node-gather-daemonset-f2vtm"]
Sep 16 19:34:29.230386 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.230359    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1d6394f5-0d70-4777-88ff-579eb8440b25-sys\") pod \"perf-node-gather-daemonset-f2vtm\" (UID: \"1d6394f5-0d70-4777-88ff-579eb8440b25\") " pod="openshift-must-gather-lpmdv/perf-node-gather-daemonset-f2vtm"
Sep 16 19:34:29.230514 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.230395    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1d6394f5-0d70-4777-88ff-579eb8440b25-proc\") pod \"perf-node-gather-daemonset-f2vtm\" (UID: \"1d6394f5-0d70-4777-88ff-579eb8440b25\") " pod="openshift-must-gather-lpmdv/perf-node-gather-daemonset-f2vtm"
Sep 16 19:34:29.230514 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.230421    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1d6394f5-0d70-4777-88ff-579eb8440b25-podres\") pod \"perf-node-gather-daemonset-f2vtm\" (UID: \"1d6394f5-0d70-4777-88ff-579eb8440b25\") " pod="openshift-must-gather-lpmdv/perf-node-gather-daemonset-f2vtm"
Sep 16 19:34:29.230514 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.230490    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1d6394f5-0d70-4777-88ff-579eb8440b25-lib-modules\") pod \"perf-node-gather-daemonset-f2vtm\" (UID: \"1d6394f5-0d70-4777-88ff-579eb8440b25\") " pod="openshift-must-gather-lpmdv/perf-node-gather-daemonset-f2vtm"
Sep 16 19:34:29.230633 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.230517    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwmjk\" (UniqueName: \"kubernetes.io/projected/1d6394f5-0d70-4777-88ff-579eb8440b25-kube-api-access-qwmjk\") pod \"perf-node-gather-daemonset-f2vtm\" (UID: \"1d6394f5-0d70-4777-88ff-579eb8440b25\") " pod="openshift-must-gather-lpmdv/perf-node-gather-daemonset-f2vtm"
Sep 16 19:34:29.331111 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.331073    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1d6394f5-0d70-4777-88ff-579eb8440b25-sys\") pod \"perf-node-gather-daemonset-f2vtm\" (UID: \"1d6394f5-0d70-4777-88ff-579eb8440b25\") " pod="openshift-must-gather-lpmdv/perf-node-gather-daemonset-f2vtm"
Sep 16 19:34:29.331111 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.331118    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1d6394f5-0d70-4777-88ff-579eb8440b25-proc\") pod \"perf-node-gather-daemonset-f2vtm\" (UID: \"1d6394f5-0d70-4777-88ff-579eb8440b25\") " pod="openshift-must-gather-lpmdv/perf-node-gather-daemonset-f2vtm"
Sep 16 19:34:29.331347 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.331209    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1d6394f5-0d70-4777-88ff-579eb8440b25-sys\") pod \"perf-node-gather-daemonset-f2vtm\" (UID: \"1d6394f5-0d70-4777-88ff-579eb8440b25\") " pod="openshift-must-gather-lpmdv/perf-node-gather-daemonset-f2vtm"
Sep 16 19:34:29.331347 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.331236    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1d6394f5-0d70-4777-88ff-579eb8440b25-podres\") pod \"perf-node-gather-daemonset-f2vtm\" (UID: \"1d6394f5-0d70-4777-88ff-579eb8440b25\") " pod="openshift-must-gather-lpmdv/perf-node-gather-daemonset-f2vtm"
Sep 16 19:34:29.331347 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.331267    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1d6394f5-0d70-4777-88ff-579eb8440b25-lib-modules\") pod \"perf-node-gather-daemonset-f2vtm\" (UID: \"1d6394f5-0d70-4777-88ff-579eb8440b25\") " pod="openshift-must-gather-lpmdv/perf-node-gather-daemonset-f2vtm"
Sep 16 19:34:29.331347 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.331284    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwmjk\" (UniqueName: \"kubernetes.io/projected/1d6394f5-0d70-4777-88ff-579eb8440b25-kube-api-access-qwmjk\") pod \"perf-node-gather-daemonset-f2vtm\" (UID: \"1d6394f5-0d70-4777-88ff-579eb8440b25\") " pod="openshift-must-gather-lpmdv/perf-node-gather-daemonset-f2vtm"
Sep 16 19:34:29.331347 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.331264    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1d6394f5-0d70-4777-88ff-579eb8440b25-proc\") pod \"perf-node-gather-daemonset-f2vtm\" (UID: \"1d6394f5-0d70-4777-88ff-579eb8440b25\") " pod="openshift-must-gather-lpmdv/perf-node-gather-daemonset-f2vtm"
Sep 16 19:34:29.331573 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.331399    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1d6394f5-0d70-4777-88ff-579eb8440b25-lib-modules\") pod \"perf-node-gather-daemonset-f2vtm\" (UID: \"1d6394f5-0d70-4777-88ff-579eb8440b25\") " pod="openshift-must-gather-lpmdv/perf-node-gather-daemonset-f2vtm"
Sep 16 19:34:29.331573 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.331443    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1d6394f5-0d70-4777-88ff-579eb8440b25-podres\") pod \"perf-node-gather-daemonset-f2vtm\" (UID: \"1d6394f5-0d70-4777-88ff-579eb8440b25\") " pod="openshift-must-gather-lpmdv/perf-node-gather-daemonset-f2vtm"
Sep 16 19:34:29.341079 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.341057    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwmjk\" (UniqueName: \"kubernetes.io/projected/1d6394f5-0d70-4777-88ff-579eb8440b25-kube-api-access-qwmjk\") pod \"perf-node-gather-daemonset-f2vtm\" (UID: \"1d6394f5-0d70-4777-88ff-579eb8440b25\") " pod="openshift-must-gather-lpmdv/perf-node-gather-daemonset-f2vtm"
Sep 16 19:34:29.516360 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.516255    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lpmdv/perf-node-gather-daemonset-f2vtm"
Sep 16 19:34:29.798576 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.798551    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lpmdv/perf-node-gather-daemonset-f2vtm"]
Sep 16 19:34:29.801165 ip-10-0-130-26 kubenswrapper[2557]: W0916 19:34:29.801130    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1d6394f5_0d70_4777_88ff_579eb8440b25.slice/crio-48457641ed09dd64fbf723a9bd09a28687845f46a4c203ffab008bf4b6ddffcb WatchSource:0}: Error finding container 48457641ed09dd64fbf723a9bd09a28687845f46a4c203ffab008bf4b6ddffcb: Status 404 returned error can't find the container with id 48457641ed09dd64fbf723a9bd09a28687845f46a4c203ffab008bf4b6ddffcb
Sep 16 19:34:29.983998 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:29.983962    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-7gwzm_6eaae832-99a9-45d5-b648-d212603a44cb/volume-data-source-validator/0.log"
Sep 16 19:34:30.697297 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:30.697253    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lpmdv/perf-node-gather-daemonset-f2vtm" event={"ID":"1d6394f5-0d70-4777-88ff-579eb8440b25","Type":"ContainerStarted","Data":"754b4d202b8e144c136b748a84f1b3be1535df7e060990cd5ab7b92212afd70f"}
Sep 16 19:34:30.697694 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:30.697317    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lpmdv/perf-node-gather-daemonset-f2vtm" event={"ID":"1d6394f5-0d70-4777-88ff-579eb8440b25","Type":"ContainerStarted","Data":"48457641ed09dd64fbf723a9bd09a28687845f46a4c203ffab008bf4b6ddffcb"}
Sep 16 19:34:30.697694 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:30.697371    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-lpmdv/perf-node-gather-daemonset-f2vtm"
Sep 16 19:34:30.716440 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:30.716392    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-lpmdv/perf-node-gather-daemonset-f2vtm" podStartSLOduration=1.7163780640000001 podStartE2EDuration="1.716378064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 19:34:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:34:30.715895216 +0000 UTC m=+2984.793000077" watchObservedRunningTime="2026-09-16 19:34:30.716378064 +0000 UTC m=+2984.793482924"
Sep 16 19:34:30.982472 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:30.982391    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sdw9s_8ac962d8-4a04-40f2-a02c-ad61a26d1fbd/dns/0.log"
Sep 16 19:34:31.013878 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:31.013851    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sdw9s_8ac962d8-4a04-40f2-a02c-ad61a26d1fbd/kube-rbac-proxy/0.log"
Sep 16 19:34:31.124972 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:31.124939    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4rzjs_55642348-e5c2-42ed-af38-d0631bd164aa/dns-node-resolver/0.log"
Sep 16 19:34:31.656723 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:31.656693    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-q6g22_5c6f6f8b-e6fc-4a16-b084-7e851b3c3cc6/node-ca/0.log"
Sep 16 19:34:32.474277 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:32.474246    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6cd8c6bfcd-mctrp_c70456fc-7f1b-4b8c-bd77-fb3523016153/router/0.log"
Sep 16 19:34:32.829554 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:32.829516    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-58tb5_2c43af6d-a516-4fc1-8f99-9a99fad657e8/serve-healthcheck-canary/0.log"
Sep 16 19:34:33.429089 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:33.429059    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-ftvj4_fb6525d5-7015-41e7-a307-c282e1f6bd79/insights-operator/0.log"
Sep 16 19:34:33.431213 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:33.431190    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-ftvj4_fb6525d5-7015-41e7-a307-c282e1f6bd79/insights-operator/1.log"
Sep 16 19:34:33.555802 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:33.555754    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m8kzv_b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53/kube-rbac-proxy/0.log"
Sep 16 19:34:33.577035 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:33.577008    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m8kzv_b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53/exporter/0.log"
Sep 16 19:34:33.625548 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:33.625521    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m8kzv_b237c1e5-ba74-4f7c-b6d9-9c4f85d7bb53/extractor/0.log"
Sep 16 19:34:36.416400 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:36.416367    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-7b966d7984-pks9w_4ec3e5f7-fc05-463f-8623-1f294e553cbd/manager/0.log"
Sep 16 19:34:36.460429 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:36.460389    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-87rph_f37977cb-a549-4543-9061-f7e25bef7bbb/s3-init/0.log"
Sep 16 19:34:36.477306 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:36.477279    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-87rph_f37977cb-a549-4543-9061-f7e25bef7bbb/download-hf-model/0.log"
Sep 16 19:34:36.710756 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:36.710687    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-lpmdv/perf-node-gather-daemonset-f2vtm"
Sep 16 19:34:42.319862 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:42.319829    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-6g9b9_87795909-0ae4-4163-aef4-bb1e143885da/kube-multus/0.log"
Sep 16 19:34:42.687574 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:42.687491    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-shx66_837f7ec9-d8aa-45b3-bebc-2eb825742375/kube-multus-additional-cni-plugins/0.log"
Sep 16 19:34:42.712739 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:42.712705    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-shx66_837f7ec9-d8aa-45b3-bebc-2eb825742375/egress-router-binary-copy/0.log"
Sep 16 19:34:42.734320 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:42.734290    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-shx66_837f7ec9-d8aa-45b3-bebc-2eb825742375/cni-plugins/0.log"
Sep 16 19:34:42.757089 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:42.757052    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-shx66_837f7ec9-d8aa-45b3-bebc-2eb825742375/bond-cni-plugin/0.log"
Sep 16 19:34:42.780207 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:42.780173    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-shx66_837f7ec9-d8aa-45b3-bebc-2eb825742375/routeoverride-cni/0.log"
Sep 16 19:34:42.802627 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:42.802592    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-shx66_837f7ec9-d8aa-45b3-bebc-2eb825742375/whereabouts-cni-bincopy/0.log"
Sep 16 19:34:42.827895 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:42.827867    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-shx66_837f7ec9-d8aa-45b3-bebc-2eb825742375/whereabouts-cni/0.log"
Sep 16 19:34:42.919360 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:42.919324    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4wnsv_c19f514e-c3a8-41d2-9de1-ad2d86fce986/network-metrics-daemon/0.log"
Sep 16 19:34:42.944362 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:42.944280    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4wnsv_c19f514e-c3a8-41d2-9de1-ad2d86fce986/kube-rbac-proxy/0.log"
Sep 16 19:34:43.853281 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:43.853242    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mgl4t_c85031e5-5e03-4f66-b61c-636789b6e44f/ovn-controller/0.log"
Sep 16 19:34:43.876396 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:43.876359    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mgl4t_c85031e5-5e03-4f66-b61c-636789b6e44f/ovn-acl-logging/0.log"
Sep 16 19:34:43.904289 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:43.904261    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mgl4t_c85031e5-5e03-4f66-b61c-636789b6e44f/kube-rbac-proxy-node/0.log"
Sep 16 19:34:43.934707 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:43.934682    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mgl4t_c85031e5-5e03-4f66-b61c-636789b6e44f/kube-rbac-proxy-ovn-metrics/0.log"
Sep 16 19:34:43.956432 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:43.956389    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mgl4t_c85031e5-5e03-4f66-b61c-636789b6e44f/northd/0.log"
Sep 16 19:34:43.988747 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:43.988723    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mgl4t_c85031e5-5e03-4f66-b61c-636789b6e44f/nbdb/0.log"
Sep 16 19:34:44.013633 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:44.013602    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mgl4t_c85031e5-5e03-4f66-b61c-636789b6e44f/sbdb/0.log"
Sep 16 19:34:44.069830 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:44.069751    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mgl4t_c85031e5-5e03-4f66-b61c-636789b6e44f/ovnkube-controller/0.log"
Sep 16 19:34:45.439239 ip-10-0-130-26 kubenswrapper[2557]: I0916 19:34:45.439162    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-gtj9s_4aff5bc4-d936-4bcc-9f5b-cb0f55fedacf/network-check-target-container/0.log"