Jul 09 09:34:01.705173 ip-10-0-137-199 systemd[1]: Starting Kubernetes Kubelet...
Jul 09 09:34:02.149497 ip-10-0-137-199 kubenswrapper[2549]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 09:34:02.149497 ip-10-0-137-199 kubenswrapper[2549]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 09 09:34:02.149497 ip-10-0-137-199 kubenswrapper[2549]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 09:34:02.149497 ip-10-0-137-199 kubenswrapper[2549]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 09 09:34:02.149497 ip-10-0-137-199 kubenswrapper[2549]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 09:34:02.150476 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.150328    2549 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 09 09:34:02.153921 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.153907    2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 09:34:02.153921 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.153920    2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 09:34:02.153921 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.153924    2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 09:34:02.154013 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.153927    2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 09:34:02.154013 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.153932    2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 09:34:02.154013 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.153935    2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 09:34:02.154013 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.153938    2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 09:34:02.154013 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.153941    2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 09:34:02.154013 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.153944    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 09:34:02.154013 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.153947    2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 09:34:02.154013 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.153950    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 09:34:02.154013 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.153953    2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 09:34:02.154013 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.153956    2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 09:34:02.154013 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.153958    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 09:34:02.154013 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.153961    2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 09:34:02.154013 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.153964    2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 09:34:02.154013 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.153966    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 09:34:02.154013 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.153969    2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 09:34:02.154013 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.153971    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 09:34:02.154013 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.153974    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 09:34:02.154013 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.153977    2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 09:34:02.154013 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.153983    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 09:34:02.154013 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.153986    2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 09:34:02.154470 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.153989    2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 09:34:02.154470 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.153991    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 09:34:02.154470 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.153994    2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 09:34:02.154470 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.153997    2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 09:34:02.154470 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.153999    2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 09:34:02.154470 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154002    2549 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 09:34:02.154470 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154004    2549 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 09:34:02.154470 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154007    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 09:34:02.154470 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154010    2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 09:34:02.154470 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154014    2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 09:34:02.154470 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154018    2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 09:34:02.154470 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154021    2549 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 09:34:02.154470 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154024    2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 09:34:02.154470 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154027    2549 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 09:34:02.154470 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154030    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 09:34:02.154470 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154033    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 09:34:02.154470 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154035    2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 09:34:02.154470 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154038    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 09:34:02.154470 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154040    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 09:34:02.154470 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154043    2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 09:34:02.154983 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154045    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 09:34:02.154983 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154048    2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 09:34:02.154983 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154050    2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 09:34:02.154983 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154053    2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 09:34:02.154983 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154056    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 09:34:02.154983 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154059    2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 09:34:02.154983 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154061    2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 09:34:02.154983 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154064    2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 09:34:02.154983 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154067    2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 09:34:02.154983 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154069    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 09:34:02.154983 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154072    2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 09:34:02.154983 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154074    2549 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 09:34:02.154983 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154076    2549 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 09:34:02.154983 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154079    2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 09:34:02.154983 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154082    2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 09:34:02.154983 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154086    2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 09:34:02.154983 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154090    2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 09:34:02.154983 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154093    2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 09:34:02.154983 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154096    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 09:34:02.154983 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154099    2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 09:34:02.155484 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154101    2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 09:34:02.155484 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154103    2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 09:34:02.155484 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154106    2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 09:34:02.155484 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154108    2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 09:34:02.155484 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154111    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 09:34:02.155484 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154113    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 09:34:02.155484 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154116    2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 09:34:02.155484 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154120    2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 09:34:02.155484 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154123    2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 09:34:02.155484 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154126    2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 09:34:02.155484 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154129    2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 09:34:02.155484 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154131    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 09:34:02.155484 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154134    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 09:34:02.155484 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154136    2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 09:34:02.155484 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154139    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 09:34:02.155484 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154142    2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 09:34:02.155484 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154145    2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 09:34:02.155484 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154150    2549 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 09:34:02.155484 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154153    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 09:34:02.155946 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154156    2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 09:34:02.155946 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154159    2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 09:34:02.155946 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154161    2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 09:34:02.155946 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154164    2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 09:34:02.155946 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154166    2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 09:34:02.155946 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154169    2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 09:34:02.155946 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154172    2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 09:34:02.155946 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154175    2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 09:34:02.155946 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154178    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 09:34:02.155946 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154180    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 09:34:02.155946 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154183    2549 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 09:34:02.155946 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154185    2549 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 09:34:02.155946 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154188    2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 09:34:02.155946 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154191    2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 09:34:02.155946 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154193    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 09:34:02.155946 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154196    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 09:34:02.155946 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154198    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 09:34:02.155946 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154950    2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 09:34:02.155946 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154957    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 09:34:02.156385 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154960    2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 09:34:02.156385 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154963    2549 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 09:34:02.156385 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154966    2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 09:34:02.156385 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154969    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 09:34:02.156385 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154972    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 09:34:02.156385 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154975    2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 09:34:02.156385 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154977    2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 09:34:02.156385 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154980    2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 09:34:02.156385 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154982    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 09:34:02.156385 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154985    2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 09:34:02.156385 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154988    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 09:34:02.156385 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154991    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 09:34:02.156385 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154994    2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 09:34:02.156385 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.154997    2549 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 09:34:02.156385 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155000    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 09:34:02.156385 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155003    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 09:34:02.156385 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155005    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 09:34:02.156385 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155008    2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 09:34:02.156385 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155011    2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 09:34:02.156385 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155014    2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 09:34:02.156920 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155016    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 09:34:02.156920 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155019    2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 09:34:02.156920 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155023    2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 09:34:02.156920 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155027    2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 09:34:02.156920 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155030    2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 09:34:02.156920 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155033    2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 09:34:02.156920 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155035    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 09:34:02.156920 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155038    2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 09:34:02.156920 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155041    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 09:34:02.156920 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155044    2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 09:34:02.156920 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155046    2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 09:34:02.156920 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155048    2549 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 09:34:02.156920 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155051    2549 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 09:34:02.156920 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155053    2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 09:34:02.156920 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155056    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 09:34:02.156920 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155058    2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 09:34:02.156920 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155061    2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 09:34:02.156920 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155063    2549 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 09:34:02.156920 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155065    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 09:34:02.156920 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155068    2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 09:34:02.157418 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155070    2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 09:34:02.157418 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155074    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 09:34:02.157418 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155077    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 09:34:02.157418 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155079    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 09:34:02.157418 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155082    2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 09:34:02.157418 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155085    2549 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 09:34:02.157418 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155095    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 09:34:02.157418 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155098    2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 09:34:02.157418 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155101    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 09:34:02.157418 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155103    2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 09:34:02.157418 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155106    2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 09:34:02.157418 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155109    2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 09:34:02.157418 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155112    2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 09:34:02.157418 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155115    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 09:34:02.157418 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155117    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 09:34:02.157418 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155121    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 09:34:02.157418 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155124    2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 09:34:02.157418 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155126    2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 09:34:02.157418 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155129    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 09:34:02.157418 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155131    2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 09:34:02.157924 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155134    2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 09:34:02.157924 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155136    2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 09:34:02.157924 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155139    2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 09:34:02.157924 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155142    2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 09:34:02.157924 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155145    2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 09:34:02.157924 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155149    2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 09:34:02.157924 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155152    2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 09:34:02.157924 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155155    2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 09:34:02.157924 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155157    2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 09:34:02.157924 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155160    2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 09:34:02.157924 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155163    2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 09:34:02.157924 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155166    2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 09:34:02.157924 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155169    2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 09:34:02.157924 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155171    2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 09:34:02.157924 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155174    2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 09:34:02.157924 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155177    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 09:34:02.157924 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155180    2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 09:34:02.157924 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155184    2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 09:34:02.157924 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155189    2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 09:34:02.158373 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155192    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 09:34:02.158373 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155195    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 09:34:02.158373 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155198    2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 09:34:02.158373 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155201    2549 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 09:34:02.158373 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155204    2549 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 09:34:02.158373 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155207    2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 09:34:02.158373 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155210    2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 09:34:02.158373 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155213    2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 09:34:02.158373 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155215    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 09:34:02.158373 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155218    2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 09:34:02.158373 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155220    2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 09:34:02.158373 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155223    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 09:34:02.158373 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155225    2549 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 09:34:02.158373 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155228    2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 09:34:02.158373 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155230    2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 09:34:02.158373 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155233    2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 09:34:02.158373 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155235    2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 09:34:02.158373 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155238    2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 09:34:02.158373 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155608    2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 09:34:02.158373 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155613    2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 09:34:02.158838 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155616    2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 09:34:02.158838 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155619    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 09:34:02.158838 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155621    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 09:34:02.158838 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155624    2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 09:34:02.158838 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155626    2549 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 09:34:02.158838 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155629    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 09:34:02.158838 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155632    2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 09:34:02.158838 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155634    2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 09:34:02.158838 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155637    2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 09:34:02.158838 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155640    2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 09:34:02.158838 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155642    2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 09:34:02.158838 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155646    2549 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 09:34:02.158838 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155648    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 09:34:02.158838 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155651    2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 09:34:02.158838 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155653    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 09:34:02.158838 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155656    2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 09:34:02.158838 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155658    2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 09:34:02.158838 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155661    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 09:34:02.158838 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155664    2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 09:34:02.158838 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155666    2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 09:34:02.159355 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155670    2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 09:34:02.159355 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155672    2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 09:34:02.159355 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155675    2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 09:34:02.159355 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155677    2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 09:34:02.159355 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155680    2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 09:34:02.159355 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155683    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 09:34:02.159355 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155685    2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 09:34:02.159355 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155688    2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 09:34:02.159355 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155690    2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 09:34:02.159355 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155693    2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 09:34:02.159355 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155695    2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 09:34:02.159355 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155698    2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 09:34:02.159355 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155700    2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 09:34:02.159355 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155703    2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 09:34:02.159355 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155705    2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 09:34:02.159355 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155709    2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 09:34:02.159355 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155713    2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 09:34:02.159355 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155717    2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 09:34:02.159355 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155720    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 09:34:02.159355 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155724    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 09:34:02.159820 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155727    2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 09:34:02.159820 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155731    2549 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 09:34:02.159820 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155735    2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 09:34:02.159820 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155739    2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 09:34:02.159820 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155743    2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 09:34:02.159820 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155746    2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 09:34:02.159820 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155749    2549 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 09:34:02.159820 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155751    2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 09:34:02.159820 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155755    2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 09:34:02.159820 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155757    2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 09:34:02.159820 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155760    2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 09:34:02.159820 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155762    2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 09:34:02.159820 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155767    2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 09:34:02.159820 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155770    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 09:34:02.159820 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155772    2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 09:34:02.159820 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155775    2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 09:34:02.159820 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155778    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 09:34:02.159820 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155780    2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 09:34:02.159820 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155783    2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 09:34:02.159820 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155785    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 09:34:02.160330 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155788    2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 09:34:02.160330 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155791    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 09:34:02.160330 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155796    2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 09:34:02.160330 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155799    2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 09:34:02.160330 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155803    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 09:34:02.160330 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155806    2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 09:34:02.160330 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155808    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 09:34:02.160330 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155812    2549 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 09:34:02.160330 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155815    2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 09:34:02.160330 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155818    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 09:34:02.160330 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155821    2549 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 09:34:02.160330 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155824    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 09:34:02.160330 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155826    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 09:34:02.160330 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155829    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 09:34:02.160330 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155832    2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 09:34:02.160330 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155834    2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 09:34:02.160330 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155837    2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 09:34:02.160330 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155840    2549 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 09:34:02.160758 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155843    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 09:34:02.160758 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155845    2549 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 09:34:02.160758 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155848    2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 09:34:02.160758 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155851    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 09:34:02.160758 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155854    2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 09:34:02.160758 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155857    2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 09:34:02.160758 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155861    2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 09:34:02.160758 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155878    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 09:34:02.160758 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155881    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 09:34:02.160758 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155883    2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 09:34:02.160758 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155886    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 09:34:02.160758 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155888    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 09:34:02.160758 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155890    2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 09:34:02.160758 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155893    2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 09:34:02.160758 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155896    2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 09:34:02.160758 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155900    2549 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 09:34:02.160758 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155903    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 09:34:02.160758 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155905    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 09:34:02.160758 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.155908    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 09:34:02.160758 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.155979    2549 flags.go:64] FLAG: --address="0.0.0.0"
Jul 09 09:34:02.161253 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.155987    2549 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 09 09:34:02.161253 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.155994    2549 flags.go:64] FLAG: --anonymous-auth="true"
Jul 09 09:34:02.161253 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.155998    2549 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 09 09:34:02.161253 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156002    2549 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 09 09:34:02.161253 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156006    2549 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 09 09:34:02.161253 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156012    2549 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 09 09:34:02.161253 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156018    2549 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 09 09:34:02.161253 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156023    2549 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 09 09:34:02.161253 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156028    2549 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 09 09:34:02.161253 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156031    2549 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 09 09:34:02.161253 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156034    2549 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 09 09:34:02.161253 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156038    2549 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 09 09:34:02.161253 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156042    2549 flags.go:64] FLAG: --cgroup-root=""
Jul 09 09:34:02.161253 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156044    2549 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 09 09:34:02.161253 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156049    2549 flags.go:64] FLAG: --client-ca-file=""
Jul 09 09:34:02.161253 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156052    2549 flags.go:64] FLAG: --cloud-provider="external"
Jul 09 09:34:02.161253 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156055    2549 flags.go:64] FLAG: --cluster-dns="[]"
Jul 09 09:34:02.161253 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156059    2549 flags.go:64] FLAG: --cluster-domain=""
Jul 09 09:34:02.161253 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156063    2549 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 09 09:34:02.161253 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156066    2549 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 09 09:34:02.161253 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156069    2549 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 09 09:34:02.161253 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156072    2549 flags.go:64] FLAG: --container-log-max-files="5"
Jul 09 09:34:02.161253 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156076    2549 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 09 09:34:02.161253 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156079    2549 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 09 09:34:02.161814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156082    2549 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 09 09:34:02.161814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156086    2549 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 09 09:34:02.161814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156089    2549 flags.go:64] FLAG: --contention-profiling="false"
Jul 09 09:34:02.161814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156092    2549 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 09 09:34:02.161814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156095    2549 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 09 09:34:02.161814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156098    2549 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 09 09:34:02.161814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156101    2549 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 09 09:34:02.161814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156105    2549 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 09 09:34:02.161814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156108    2549 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 09 09:34:02.161814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156111    2549 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 09 09:34:02.161814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156114    2549 flags.go:64] FLAG: --enable-load-reader="false"
Jul 09 09:34:02.161814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156117    2549 flags.go:64] FLAG: --enable-server="true"
Jul 09 09:34:02.161814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156119    2549 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 09 09:34:02.161814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156130    2549 flags.go:64] FLAG: --event-burst="100"
Jul 09 09:34:02.161814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156133    2549 flags.go:64] FLAG: --event-qps="50"
Jul 09 09:34:02.161814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156136    2549 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 09 09:34:02.161814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156139    2549 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 09 09:34:02.161814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156142    2549 flags.go:64] FLAG: --eviction-hard=""
Jul 09 09:34:02.161814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156150    2549 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 09 09:34:02.161814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156153    2549 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 09 09:34:02.161814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156156    2549 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 09 09:34:02.161814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156159    2549 flags.go:64] FLAG: --eviction-soft=""
Jul 09 09:34:02.161814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156163    2549 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 09 09:34:02.161814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156167    2549 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 09 09:34:02.161814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156169    2549 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 09 09:34:02.162483 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156172    2549 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 09 09:34:02.162483 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156177    2549 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 09 09:34:02.162483 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156180    2549 flags.go:64] FLAG: --fail-swap-on="true"
Jul 09 09:34:02.162483 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156183    2549 flags.go:64] FLAG: --feature-gates=""
Jul 09 09:34:02.162483 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156187    2549 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 09 09:34:02.162483 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156190    2549 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 09 09:34:02.162483 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156193    2549 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 09 09:34:02.162483 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156196    2549 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 09 09:34:02.162483 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156199    2549 flags.go:64] FLAG: --healthz-port="10248"
Jul 09 09:34:02.162483 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156202    2549 flags.go:64] FLAG: --help="false"
Jul 09 09:34:02.162483 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156204    2549 flags.go:64] FLAG: --hostname-override="ip-10-0-137-199.ec2.internal"
Jul 09 09:34:02.162483 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156207    2549 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 09 09:34:02.162483 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156210    2549 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 09 09:34:02.162483 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156213    2549 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 09 09:34:02.162483 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156217    2549 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 09 09:34:02.162483 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156220    2549 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 09 09:34:02.162483 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156223    2549 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 09 09:34:02.162483 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156226    2549 flags.go:64] FLAG: --image-service-endpoint=""
Jul 09 09:34:02.162483 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156228    2549 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 09 09:34:02.162483 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156231    2549 flags.go:64] FLAG: --kube-api-burst="100"
Jul 09 09:34:02.162483 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156234    2549 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 09 09:34:02.162483 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156237    2549 flags.go:64] FLAG: --kube-api-qps="50"
Jul 09 09:34:02.162483 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156240    2549 flags.go:64] FLAG: --kube-reserved=""
Jul 09 09:34:02.162483 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156243    2549 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 09 09:34:02.163083 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156246    2549 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 09 09:34:02.163083 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156249    2549 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 09 09:34:02.163083 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156251    2549 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 09 09:34:02.163083 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156254    2549 flags.go:64] FLAG: --lock-file=""
Jul 09 09:34:02.163083 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156257    2549 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 09 09:34:02.163083 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156263    2549 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 09 09:34:02.163083 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156266    2549 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 09 09:34:02.163083 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156271    2549 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 09 09:34:02.163083 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156274    2549 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 09 09:34:02.163083 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156279    2549 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 09 09:34:02.163083 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156281    2549 flags.go:64] FLAG: --logging-format="text"
Jul 09 09:34:02.163083 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156284    2549 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 09 09:34:02.163083 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156287    2549 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 09 09:34:02.163083 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156290    2549 flags.go:64] FLAG: --manifest-url=""
Jul 09 09:34:02.163083 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156293    2549 flags.go:64] FLAG: --manifest-url-header=""
Jul 09 09:34:02.163083 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156297    2549 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 09 09:34:02.163083 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156300    2549 flags.go:64] FLAG: --max-open-files="1000000"
Jul 09 09:34:02.163083 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156304    2549 flags.go:64] FLAG: --max-pods="110"
Jul 09 09:34:02.163083 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156307    2549 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 09 09:34:02.163083 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156310    2549 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 09 09:34:02.163083 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156313    2549 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 09 09:34:02.163083 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156316    2549 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 09 09:34:02.163083 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156319    2549 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 09 09:34:02.163083 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156322    2549 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 09 09:34:02.163083 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156324    2549 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 09 09:34:02.163696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156331    2549 flags.go:64] FLAG: --node-status-max-images="50"
Jul 09 09:34:02.163696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156334    2549 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 09 09:34:02.163696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156337    2549 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 09 09:34:02.163696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156340    2549 flags.go:64] FLAG: --pod-cidr=""
Jul 09 09:34:02.163696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156342    2549 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 09 09:34:02.163696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156348    2549 flags.go:64] FLAG: --pod-manifest-path=""
Jul 09 09:34:02.163696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156351    2549 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 09 09:34:02.163696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156354    2549 flags.go:64] FLAG: --pods-per-core="0"
Jul 09 09:34:02.163696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156357    2549 flags.go:64] FLAG: --port="10250"
Jul 09 09:34:02.163696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156360    2549 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 09 09:34:02.163696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156363    2549 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01ec99ada21b2d39a"
Jul 09 09:34:02.163696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156366    2549 flags.go:64] FLAG: --qos-reserved=""
Jul 09 09:34:02.163696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156369    2549 flags.go:64] FLAG: --read-only-port="10255"
Jul 09 09:34:02.163696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156374    2549 flags.go:64] FLAG: --register-node="true"
Jul 09 09:34:02.163696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156377    2549 flags.go:64] FLAG: --register-with-taints=""
Jul 09 09:34:02.163696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156381    2549 flags.go:64] FLAG: --registry-burst="10"
Jul 09 09:34:02.163696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156385    2549 flags.go:64] FLAG: --registry-qps="5"
Jul 09 09:34:02.163696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156388    2549 flags.go:64] FLAG: --reserved-cpus=""
Jul 09 09:34:02.163696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156390    2549 flags.go:64] FLAG: --reserved-memory=""
Jul 09 09:34:02.163696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156394    2549 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 09 09:34:02.163696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156397    2549 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 09 09:34:02.163696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156400    2549 flags.go:64] FLAG: --rotate-certificates="false"
Jul 09 09:34:02.163696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156403    2549 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 09 09:34:02.163696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156406    2549 flags.go:64] FLAG: --runonce="false"
Jul 09 09:34:02.163696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156409    2549 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 09 09:34:02.164313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156412    2549 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 09 09:34:02.164313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156415    2549 flags.go:64] FLAG: --seccomp-default="false"
Jul 09 09:34:02.164313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156418    2549 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 09 09:34:02.164313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156421    2549 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 09 09:34:02.164313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156424    2549 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 09 09:34:02.164313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156427    2549 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 09 09:34:02.164313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156430    2549 flags.go:64] FLAG: --storage-driver-password="root"
Jul 09 09:34:02.164313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156432    2549 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 09 09:34:02.164313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156435    2549 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 09 09:34:02.164313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156438    2549 flags.go:64] FLAG: --storage-driver-user="root"
Jul 09 09:34:02.164313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156441    2549 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 09 09:34:02.164313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156444    2549 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 09 09:34:02.164313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156447    2549 flags.go:64] FLAG: --system-cgroups=""
Jul 09 09:34:02.164313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156450    2549 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 09 09:34:02.164313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156455    2549 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 09 09:34:02.164313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156458    2549 flags.go:64] FLAG: --tls-cert-file=""
Jul 09 09:34:02.164313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156461    2549 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 09 09:34:02.164313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156465    2549 flags.go:64] FLAG: --tls-min-version=""
Jul 09 09:34:02.164313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156467    2549 flags.go:64] FLAG: --tls-private-key-file=""
Jul 09 09:34:02.164313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156470    2549 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 09 09:34:02.164313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156474    2549 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 09 09:34:02.164313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156477    2549 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 09 09:34:02.164313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156480    2549 flags.go:64] FLAG: --v="2"
Jul 09 09:34:02.164313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156485    2549 flags.go:64] FLAG: --version="false"
Jul 09 09:34:02.164313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156489    2549 flags.go:64] FLAG: --vmodule=""
Jul 09 09:34:02.164910 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156492    2549 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 09 09:34:02.164910 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156495    2549 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 09 09:34:02.164910 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156614    2549 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 09:34:02.164910 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156618    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 09:34:02.164910 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156620    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 09:34:02.164910 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156623    2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 09:34:02.164910 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156626    2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 09:34:02.164910 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156629    2549 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 09:34:02.164910 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156631    2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 09:34:02.164910 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156634    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 09:34:02.164910 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156636    2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 09:34:02.164910 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156639    2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 09:34:02.164910 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156642    2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 09:34:02.164910 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156644    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 09:34:02.164910 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156647    2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 09:34:02.164910 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156649    2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 09:34:02.164910 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156652    2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 09:34:02.164910 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156656    2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 09:34:02.164910 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156659    2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 09:34:02.165364 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156662    2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 09:34:02.165364 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156665    2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 09:34:02.165364 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156668    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 09:34:02.165364 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156670    2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 09:34:02.165364 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156673    2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 09:34:02.165364 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156676    2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 09:34:02.165364 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156678    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 09:34:02.165364 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156681    2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 09:34:02.165364 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156686    2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 09:34:02.165364 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156689    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 09:34:02.165364 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156697    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 09:34:02.165364 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156700    2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 09:34:02.165364 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156702    2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 09:34:02.165364 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156705    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 09:34:02.165364 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156707    2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 09:34:02.165364 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156710    2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 09:34:02.165364 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156713    2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 09:34:02.165364 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156715    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 09:34:02.165364 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156718    2549 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 09:34:02.165364 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156720    2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 09:34:02.165836 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156723    2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 09:34:02.165836 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156725    2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 09:34:02.165836 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156728    2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 09:34:02.165836 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156730    2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 09:34:02.165836 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156733    2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 09:34:02.165836 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156736    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 09:34:02.165836 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156738    2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 09:34:02.165836 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156742    2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 09:34:02.165836 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156745    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 09:34:02.165836 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156748    2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 09:34:02.165836 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156751    2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 09:34:02.165836 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156754    2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 09:34:02.165836 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156757    2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 09:34:02.165836 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156759    2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 09:34:02.165836 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156762    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 09:34:02.165836 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156765    2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 09:34:02.165836 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156768    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 09:34:02.165836 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156770    2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 09:34:02.165836 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156772    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 09:34:02.166375 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156775    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 09:34:02.166375 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156779    2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 09:34:02.166375 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156782    2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 09:34:02.166375 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156784    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 09:34:02.166375 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156787    2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 09:34:02.166375 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156790    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 09:34:02.166375 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156792    2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 09:34:02.166375 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156795    2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 09:34:02.166375 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156797    2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 09:34:02.166375 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156800    2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 09:34:02.166375 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156802    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 09:34:02.166375 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156805    2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 09:34:02.166375 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156808    2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 09:34:02.166375 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156810    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 09:34:02.166375 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156813    2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 09:34:02.166375 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156815    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 09:34:02.166375 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156818    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 09:34:02.166375 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156820    2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 09:34:02.166375 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156823    2549 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 09:34:02.166375 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156826    2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 09:34:02.166856 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156828    2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 09:34:02.166856 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156830    2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 09:34:02.166856 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156834    2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 09:34:02.166856 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156838    2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 09:34:02.166856 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156841    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 09:34:02.166856 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156844    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 09:34:02.166856 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156846    2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 09:34:02.166856 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156849    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 09:34:02.166856 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156852    2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 09:34:02.166856 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156855    2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 09:34:02.166856 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156858    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 09:34:02.166856 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156874    2549 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 09:34:02.166856 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156877    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 09:34:02.166856 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156881    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 09:34:02.166856 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156884    2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 09:34:02.166856 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156887    2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 09:34:02.166856 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156889    2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 09:34:02.166856 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156892    2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 09:34:02.166856 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156895    2549 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 09:34:02.166856 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156898    2549 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 09:34:02.167393 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156900    2549 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 09:34:02.167393 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156903    2549 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 09:34:02.167393 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.156906    2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 09:34:02.167393 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.156912    2549 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 09:34:02.167393 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.163542    2549 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 09 09:34:02.167393 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.163557    2549 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 09 09:34:02.167393 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.163574    2549 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 09 09:34:02.167393 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.163580    2549 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 09 09:34:02.167393 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163635    2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 09:34:02.167393 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163640    2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 09:34:02.167393 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163644    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 09:34:02.167393 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163647    2549 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 09:34:02.167393 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163650    2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 09:34:02.167393 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163653    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 09:34:02.167393 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163657    2549 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 09:34:02.167764 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163660    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 09:34:02.167764 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163662    2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 09:34:02.167764 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163665    2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 09:34:02.167764 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163667    2549 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 09:34:02.167764 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163670    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 09:34:02.167764 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163672    2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 09:34:02.167764 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163675    2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 09:34:02.167764 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163678    2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 09:34:02.167764 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163680    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 09:34:02.167764 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163683    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 09:34:02.167764 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163686    2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 09:34:02.167764 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163688    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 09:34:02.167764 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163691    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 09:34:02.167764 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163693    2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 09:34:02.167764 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163696    2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 09:34:02.167764 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163699    2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 09:34:02.167764 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163702    2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 09:34:02.167764 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163705    2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 09:34:02.167764 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163707    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 09:34:02.168246 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163710    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 09:34:02.168246 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163712    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 09:34:02.168246 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163715    2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 09:34:02.168246 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163721    2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 09:34:02.168246 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163724    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 09:34:02.168246 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163727    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 09:34:02.168246 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163730    2549 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 09:34:02.168246 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163732    2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 09:34:02.168246 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163735    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 09:34:02.168246 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163738    2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 09:34:02.168246 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163741    2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 09:34:02.168246 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163744    2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 09:34:02.168246 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163746    2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 09:34:02.168246 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163749    2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 09:34:02.168246 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163751    2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 09:34:02.168246 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163754    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 09:34:02.168246 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163757    2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 09:34:02.168246 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163759    2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 09:34:02.168246 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163762    2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 09:34:02.168246 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163764    2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 09:34:02.168728 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163767    2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 09:34:02.168728 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163769    2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 09:34:02.168728 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163772    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 09:34:02.168728 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163775    2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 09:34:02.168728 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163779    2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 09:34:02.168728 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163784    2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 09:34:02.168728 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163788    2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 09:34:02.168728 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163792    2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 09:34:02.168728 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163796    2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 09:34:02.168728 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163799    2549 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 09:34:02.168728 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163802    2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 09:34:02.168728 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163805    2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 09:34:02.168728 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163807    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 09:34:02.168728 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163810    2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 09:34:02.168728 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163813    2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 09:34:02.168728 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163817    2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 09:34:02.168728 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163820    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 09:34:02.168728 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163823    2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 09:34:02.168728 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163826    2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 09:34:02.169197 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163829    2549 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 09:34:02.169197 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163831    2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 09:34:02.169197 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163834    2549 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 09:34:02.169197 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163837    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 09:34:02.169197 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163839    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 09:34:02.169197 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163842    2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 09:34:02.169197 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163844    2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 09:34:02.169197 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163847    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 09:34:02.169197 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163850    2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 09:34:02.169197 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163852    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 09:34:02.169197 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163855    2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 09:34:02.169197 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163857    2549 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 09:34:02.169197 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163860    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 09:34:02.169197 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163878    2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 09:34:02.169197 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163881    2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 09:34:02.169197 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163884    2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 09:34:02.169197 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163887    2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 09:34:02.169197 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163890    2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 09:34:02.169197 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163892    2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 09:34:02.169197 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163895    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 09:34:02.169677 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163898    2549 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 09:34:02.169677 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163900    2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 09:34:02.169677 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163903    2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 09:34:02.169677 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163905    2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 09:34:02.169677 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163908    2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 09:34:02.169677 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163910    2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 09:34:02.169677 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163913    2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 09:34:02.169677 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163916    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 09:34:02.169677 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163919    2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 09:34:02.169677 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163922    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 09:34:02.169677 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163926    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 09:34:02.169677 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163929    2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 09:34:02.169677 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163931    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 09:34:02.169677 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.163934    2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 09:34:02.169677 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.163939    2549 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 09:34:02.170053 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164058    2549 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 09:34:02.170053 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164064    2549 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 09:34:02.170053 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164067    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 09:34:02.170053 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164070    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 09:34:02.170053 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164073    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 09:34:02.170053 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164077    2549 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 09:34:02.170053 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164080    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 09:34:02.170053 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164082    2549 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 09:34:02.170053 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164085    2549 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 09:34:02.170053 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164088    2549 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 09:34:02.170053 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164090    2549 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 09:34:02.170053 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164093    2549 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 09:34:02.170053 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164097    2549 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 09:34:02.170053 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164101    2549 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 09:34:02.170053 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164103    2549 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 09:34:02.170053 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164106    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 09:34:02.170053 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164109    2549 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 09:34:02.170053 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164112    2549 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 09:34:02.170053 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164115    2549 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 09:34:02.170502 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164118    2549 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 09:34:02.170502 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164120    2549 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 09:34:02.170502 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164123    2549 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 09:34:02.170502 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164126    2549 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 09:34:02.170502 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164128    2549 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 09:34:02.170502 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164131    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 09:34:02.170502 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164134    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 09:34:02.170502 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164137    2549 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 09:34:02.170502 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164139    2549 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 09:34:02.170502 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164143    2549 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 09:34:02.170502 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164145    2549 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 09:34:02.170502 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164148    2549 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 09:34:02.170502 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164150    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 09:34:02.170502 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164153    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 09:34:02.170502 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164155    2549 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 09:34:02.170502 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164158    2549 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 09:34:02.170502 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164160    2549 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 09:34:02.170502 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164164    2549 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 09:34:02.170502 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164167    2549 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 09:34:02.171063 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164170    2549 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 09:34:02.171063 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164173    2549 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 09:34:02.171063 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164176    2549 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 09:34:02.171063 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164179    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 09:34:02.171063 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164181    2549 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 09:34:02.171063 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164184    2549 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 09:34:02.171063 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164187    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 09:34:02.171063 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164190    2549 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 09:34:02.171063 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164193    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 09:34:02.171063 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164195    2549 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 09:34:02.171063 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164197    2549 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 09:34:02.171063 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164200    2549 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 09:34:02.171063 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164203    2549 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 09:34:02.171063 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164205    2549 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 09:34:02.171063 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164209    2549 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 09:34:02.171063 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164213    2549 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 09:34:02.171063 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164215    2549 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 09:34:02.171063 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164218    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 09:34:02.171063 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164221    2549 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 09:34:02.171543 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164223    2549 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 09:34:02.171543 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164226    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 09:34:02.171543 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164229    2549 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 09:34:02.171543 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164231    2549 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 09:34:02.171543 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164234    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 09:34:02.171543 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164237    2549 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 09:34:02.171543 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164240    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 09:34:02.171543 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164243    2549 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 09:34:02.171543 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164245    2549 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 09:34:02.171543 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164248    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 09:34:02.171543 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164250    2549 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 09:34:02.171543 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164253    2549 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 09:34:02.171543 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164255    2549 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 09:34:02.171543 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164258    2549 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 09:34:02.171543 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164260    2549 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 09:34:02.171543 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164262    2549 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 09:34:02.171543 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164265    2549 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 09:34:02.171543 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164267    2549 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 09:34:02.171543 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164270    2549 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 09:34:02.171543 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164272    2549 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 09:34:02.172039 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164275    2549 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 09:34:02.172039 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164277    2549 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 09:34:02.172039 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164280    2549 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 09:34:02.172039 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164282    2549 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 09:34:02.172039 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164284    2549 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 09:34:02.172039 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164287    2549 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 09:34:02.172039 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164289    2549 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 09:34:02.172039 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164292    2549 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 09:34:02.172039 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164294    2549 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 09:34:02.172039 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164297    2549 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 09:34:02.172039 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164299    2549 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 09:34:02.172039 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164302    2549 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 09:34:02.172039 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164305    2549 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 09:34:02.172039 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164308    2549 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 09:34:02.172039 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164310    2549 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 09:34:02.172039 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164314    2549 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 09:34:02.172039 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164316    2549 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 09:34:02.172039 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164320    2549 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 09:34:02.172039 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164323    2549 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 09:34:02.172039 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164325    2549 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 09:34:02.172507 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164328    2549 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 09:34:02.172507 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:02.164330    2549 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 09:34:02.172507 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.164335    2549 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 09:34:02.172507 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.164437    2549 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 09 09:34:02.172507 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.166836    2549 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 09 09:34:02.172507 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.169230    2549 server.go:1019] "Starting client certificate rotation"
Jul 09 09:34:02.172507 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.169325    2549 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 09:34:02.172507 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.169363    2549 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 09:34:02.173287 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.173273    2549 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 09:34:02.174430 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.174413    2549 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 09:34:02.187893 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.187874    2549 log.go:25] "Validated CRI v1 runtime API"
Jul 09 09:34:02.189922 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.189908    2549 log.go:25] "Validated CRI v1 image API"
Jul 09 09:34:02.191942 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.191918    2549 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 09 09:34:02.195291 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.195259    2549 fs.go:135] Filesystem UUIDs: map[1c8f5725-f7b6-4f06-9288-c574c2d13dc1:/dev/nvme0n1p3 445f72d0-569a-4dec-be21-c0b481115540:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 09 09:34:02.195357 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.195292    2549 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 09 09:34:02.200021 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.200005    2549 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 09:34:02.201675 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.201559    2549 manager.go:216] Machine: {Timestamp:2026-07-09 09:34:02.199970661 +0000 UTC m=+0.371129208 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099172 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25c2ec3349fcb4f655923855e182bf SystemUUID:ec25c2ec-3349-fcb4-f655-923855e182bf BootID:83f9901d-da9e-4509-b7b8-cb74d38644b5 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d6:8b:9a:bb:27 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d6:8b:9a:bb:27 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:62:1f:09:2c:38:7e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 09 09:34:02.201675 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.201671    2549 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 09 09:34:02.201801 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.201789    2549 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 09 09:34:02.204147 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.204126    2549 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 09 09:34:02.204279 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.204150    2549 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-199.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 09 09:34:02.204783 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.204773    2549 topology_manager.go:138] "Creating topology manager with none policy"
Jul 09 09:34:02.204812 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.204786    2549 container_manager_linux.go:309] "Creating device plugin manager"
Jul 09 09:34:02.204812 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.204793    2549 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 09:34:02.206205 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.206194    2549 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 09:34:02.207739 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.207727    2549 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 09 09:34:02.210998 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.210986    2549 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 09:34:02.211129 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.211119    2549 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 09 09:34:02.211159 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.211150    2549 kubelet.go:486] "Attempting to sync node with API server"
Jul 09 09:34:02.211193 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.211159    2549 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 09 09:34:02.211193 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.211180    2549 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 09 09:34:02.211193 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.211189    2549 kubelet.go:398] "Adding apiserver pod source"
Jul 09 09:34:02.211273 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.211197    2549 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 09 09:34:02.211547 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.211536    2549 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 09:34:02.211596 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.211554    2549 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 09:34:02.214183 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.214167    2549 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 09 09:34:02.217102 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.217087    2549 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 09 09:34:02.217181 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.217116    2549 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 09 09:34:02.218734 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.218722    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 09 09:34:02.218785 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.218738    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 09 09:34:02.218785 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.218745    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 09 09:34:02.218785 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.218751    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 09 09:34:02.218785 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.218765    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 09 09:34:02.218785 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.218771    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 09 09:34:02.218785 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.218779    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 09 09:34:02.218785 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.218785    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 09 09:34:02.218988 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.218792    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 09 09:34:02.218988 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.218797    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 09 09:34:02.218988 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.218809    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 09 09:34:02.218988 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.218815    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 09 09:34:02.220576 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.220564    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 09 09:34:02.220576 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.220574    2549 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 09 09:34:02.222149 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:02.222129    2549 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 09 09:34:02.222243 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:02.222226    2549 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-199.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 09 09:34:02.222373 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.222358    2549 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-199.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 09:34:02.223637 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.223624    2549 server.go:1268] "Started kubelet"
Jul 09 09:34:02.223709 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.223682    2549 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 09 09:34:02.224353 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.224238    2549 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 09 09:34:02.224481 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.224463    2549 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 09 09:34:02.224606 ip-10-0-137-199 systemd[1]: Started Kubernetes Kubelet.
Jul 09 09:34:02.225698 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.225671    2549 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 09 09:34:02.231956 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:02.231124    2549 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 09 09:34:02.231956 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.231689    2549 server.go:310] "Adding debug handlers to kubelet server"
Jul 09 09:34:02.234011 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.233992    2549 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5dk7t"
Jul 09 09:34:02.235938 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.235920    2549 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 09 09:34:02.235938 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.235936    2549 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 09 09:34:02.236065 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:02.236019    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-199.ec2.internal\" not found"
Jul 09 09:34:02.236502 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.236481    2549 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 09 09:34:02.236616 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.236602    2549 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 09 09:34:02.236679 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.236618    2549 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 09 09:34:02.236750 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.236725    2549 reconstruct.go:97] "Volume reconstruction finished"
Jul 09 09:34:02.236860 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.236742    2549 reconciler.go:29] "Reconciler: start to sync state"
Jul 09 09:34:02.237306 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.237290    2549 factory.go:55] Registering systemd factory
Jul 09 09:34:02.237378 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.237332    2549 factory.go:223] Registration of the systemd container factory successfully
Jul 09 09:34:02.237573 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.237548    2549 factory.go:153] Registering CRI-O factory
Jul 09 09:34:02.237573 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.237568    2549 factory.go:223] Registration of the crio container factory successfully
Jul 09 09:34:02.237705 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.237624    2549 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 09 09:34:02.237705 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.237651    2549 factory.go:103] Registering Raw factory
Jul 09 09:34:02.237705 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.237681    2549 manager.go:1196] Started watching for new ooms in manager
Jul 09 09:34:02.238170 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.238157    2549 manager.go:349] Starting recovery of all containers
Jul 09 09:34:02.239382 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:02.239352    2549 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 09 09:34:02.239474 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:02.239375    2549 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-199.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 09 09:34:02.240457 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:02.239454    2549 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-199.ec2.internal.18c0955469cdc040  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-199.ec2.internal,UID:ip-10-0-137-199.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-199.ec2.internal,},FirstTimestamp:2026-07-09 09:34:02.22359968 +0000 UTC m=+0.394758227,LastTimestamp:2026-07-09 09:34:02.22359968 +0000 UTC m=+0.394758227,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-199.ec2.internal,}"
Jul 09 09:34:02.242750 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.242722    2549 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5dk7t"
Jul 09 09:34:02.248042 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.248025    2549 manager.go:354] Recovery completed
Jul 09 09:34:02.252111 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.252099    2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 09:34:02.253827 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.253809    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-199.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 09:34:02.253932 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.253844    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-199.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 09:34:02.254299 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.254287    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-199.ec2.internal" event="NodeHasSufficientPID"
Jul 09 09:34:02.254757 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.254745    2549 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 09 09:34:02.254757 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.254757    2549 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 09 09:34:02.254916 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.254773    2549 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 09:34:02.254916 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.254821    2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 09:34:02.255463 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.255449    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-199.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 09:34:02.255497 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.255474    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-199.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 09:34:02.255497 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.255485    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-199.ec2.internal" event="NodeHasSufficientPID"
Jul 09 09:34:02.256089 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.256076    2549 policy_none.go:49] "None policy: Start"
Jul 09 09:34:02.256153 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.256094    2549 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 09 09:34:02.256153 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.256106    2549 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 09 09:34:02.257005 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.256993    2549 policy_none.go:47] "Start"
Jul 09 09:34:02.291545 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.291438    2549 manager.go:337] "Starting Device Plugin manager"
Jul 09 09:34:02.291651 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:02.291573    2549 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 09 09:34:02.291651 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.291583    2549 server.go:87] "Starting device plugin registration server"
Jul 09 09:34:02.291779 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.291769    2549 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 09 09:34:02.291811 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.291780    2549 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 09 09:34:02.291913 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.291895    2549 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 09 09:34:02.292010 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.291975    2549 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 09 09:34:02.292010 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.291984    2549 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 09 09:34:02.292442 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:02.292422    2549 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 09 09:34:02.292502 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:02.292478    2549 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-199.ec2.internal\" not found"
Jul 09 09:34:02.377893 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.377855    2549 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 09 09:34:02.379140 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.379123    2549 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 09 09:34:02.379212 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.379168    2549 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 09 09:34:02.379212 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.379196    2549 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 09 09:34:02.379296 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:02.379241    2549 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 09 09:34:02.383118 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.383100    2549 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 09:34:02.392285 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.392271    2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 09:34:02.393065 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.393047    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-199.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 09:34:02.393140 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.393081    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-199.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 09:34:02.393140 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.393095    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-199.ec2.internal" event="NodeHasSufficientPID"
Jul 09 09:34:02.393140 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.393120    2549 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-199.ec2.internal"
Jul 09 09:34:02.403221 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.403183    2549 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-199.ec2.internal"
Jul 09 09:34:02.403221 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:02.403208    2549 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-199.ec2.internal\": node \"ip-10-0-137-199.ec2.internal\" not found"
Jul 09 09:34:02.442760 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:02.442745    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-199.ec2.internal\" not found"
Jul 09 09:34:02.479572 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.479555    2549 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-199.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-199.ec2.internal"]
Jul 09 09:34:02.479629 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.479606    2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 09:34:02.480337 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.480323    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-199.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 09:34:02.480398 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.480347    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-199.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 09:34:02.480398 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.480356    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-199.ec2.internal" event="NodeHasSufficientPID"
Jul 09 09:34:02.481544 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.481534    2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 09:34:02.481679 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.481667    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-199.ec2.internal"
Jul 09 09:34:02.481723 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.481694    2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 09:34:02.482127 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.482112    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-199.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 09:34:02.482185 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.482137    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-199.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 09:34:02.482185 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.482146    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-199.ec2.internal" event="NodeHasSufficientPID"
Jul 09 09:34:02.482696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.482682    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-199.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 09:34:02.482765 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.482717    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-199.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 09:34:02.482765 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.482731    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-199.ec2.internal" event="NodeHasSufficientPID"
Jul 09 09:34:02.483122 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.483111    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-199.ec2.internal"
Jul 09 09:34:02.483174 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.483133    2549 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 09:34:02.483694 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.483679    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-199.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 09:34:02.483755 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.483706    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-199.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 09:34:02.483755 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.483718    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-199.ec2.internal" event="NodeHasSufficientPID"
Jul 09 09:34:02.518384 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:02.518371    2549 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-199.ec2.internal\" not found" node="ip-10-0-137-199.ec2.internal"
Jul 09 09:34:02.522352 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:02.522338    2549 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-199.ec2.internal\" not found" node="ip-10-0-137-199.ec2.internal"
Jul 09 09:34:02.537731 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.537709    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f7edc44c9cdff1100ab1bbace1365704-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-199.ec2.internal\" (UID: \"f7edc44c9cdff1100ab1bbace1365704\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-199.ec2.internal"
Jul 09 09:34:02.537825 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.537737    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7edc44c9cdff1100ab1bbace1365704-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-199.ec2.internal\" (UID: \"f7edc44c9cdff1100ab1bbace1365704\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-199.ec2.internal"
Jul 09 09:34:02.537825 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.537754    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d0bea77c1d44c6a069aa75a3ce77a6d6-config\") pod \"kube-apiserver-proxy-ip-10-0-137-199.ec2.internal\" (UID: \"d0bea77c1d44c6a069aa75a3ce77a6d6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-199.ec2.internal"
Jul 09 09:34:02.543766 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:02.543753    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-199.ec2.internal\" not found"
Jul 09 09:34:02.638563 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.638545    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f7edc44c9cdff1100ab1bbace1365704-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-199.ec2.internal\" (UID: \"f7edc44c9cdff1100ab1bbace1365704\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-199.ec2.internal"
Jul 09 09:34:02.638625 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.638566    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7edc44c9cdff1100ab1bbace1365704-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-199.ec2.internal\" (UID: \"f7edc44c9cdff1100ab1bbace1365704\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-199.ec2.internal"
Jul 09 09:34:02.638625 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.638581    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d0bea77c1d44c6a069aa75a3ce77a6d6-config\") pod \"kube-apiserver-proxy-ip-10-0-137-199.ec2.internal\" (UID: \"d0bea77c1d44c6a069aa75a3ce77a6d6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-199.ec2.internal"
Jul 09 09:34:02.638685 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.638625    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d0bea77c1d44c6a069aa75a3ce77a6d6-config\") pod \"kube-apiserver-proxy-ip-10-0-137-199.ec2.internal\" (UID: \"d0bea77c1d44c6a069aa75a3ce77a6d6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-199.ec2.internal"
Jul 09 09:34:02.638685 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.638628    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f7edc44c9cdff1100ab1bbace1365704-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-199.ec2.internal\" (UID: \"f7edc44c9cdff1100ab1bbace1365704\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-199.ec2.internal"
Jul 09 09:34:02.638685 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.638643    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7edc44c9cdff1100ab1bbace1365704-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-199.ec2.internal\" (UID: \"f7edc44c9cdff1100ab1bbace1365704\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-199.ec2.internal"
Jul 09 09:34:02.644643 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:02.644628    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-199.ec2.internal\" not found"
Jul 09 09:34:02.745027 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:02.744967    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-199.ec2.internal\" not found"
Jul 09 09:34:02.821305 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.821281    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-199.ec2.internal"
Jul 09 09:34:02.825900 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:02.825886    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-199.ec2.internal"
Jul 09 09:34:02.845923 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:02.845905    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-199.ec2.internal\" not found"
Jul 09 09:34:02.946302 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:02.946264    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-199.ec2.internal\" not found"
Jul 09 09:34:03.046729 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:03.046667    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-199.ec2.internal\" not found"
Jul 09 09:34:03.121737 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:03.121707    2549 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 09:34:03.147318 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:03.147293    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-199.ec2.internal\" not found"
Jul 09 09:34:03.167373 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:03.167355    2549 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 09 09:34:03.167730 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:03.167472    2549 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 09 09:34:03.167730 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:03.167473    2549 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 09 09:34:03.244873 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:03.244831    2549 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 09:29:02 +0000 UTC" deadline="2028-02-06 19:17:58.30989791 +0000 UTC"
Jul 09 09:34:03.244873 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:03.244855    2549 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13857h43m55.065045724s"
Jul 09 09:34:03.247960 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:03.247945    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-199.ec2.internal\" not found"
Jul 09 09:34:03.312948 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:03.312912    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0bea77c1d44c6a069aa75a3ce77a6d6.slice/crio-260b41d51398faca6399d86542b42b31cca702d16af5b0d98129c7c60c4dd49e WatchSource:0}: Error finding container 260b41d51398faca6399d86542b42b31cca702d16af5b0d98129c7c60c4dd49e: Status 404 returned error can't find the container with id 260b41d51398faca6399d86542b42b31cca702d16af5b0d98129c7c60c4dd49e
Jul 09 09:34:03.313388 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:03.313362    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7edc44c9cdff1100ab1bbace1365704.slice/crio-c7ea0ca07c75900c506b9efe551584ab38618b15c41075f301293300d78a1904 WatchSource:0}: Error finding container c7ea0ca07c75900c506b9efe551584ab38618b15c41075f301293300d78a1904: Status 404 returned error can't find the container with id c7ea0ca07c75900c506b9efe551584ab38618b15c41075f301293300d78a1904
Jul 09 09:34:03.316606 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:03.316594    2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 09:34:03.348330 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:03.348309    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-199.ec2.internal\" not found"
Jul 09 09:34:03.382233 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:03.382193    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-199.ec2.internal" event={"ID":"f7edc44c9cdff1100ab1bbace1365704","Type":"ContainerStarted","Data":"c7ea0ca07c75900c506b9efe551584ab38618b15c41075f301293300d78a1904"}
Jul 09 09:34:03.383079 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:03.383063    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-199.ec2.internal" event={"ID":"d0bea77c1d44c6a069aa75a3ce77a6d6","Type":"ContainerStarted","Data":"260b41d51398faca6399d86542b42b31cca702d16af5b0d98129c7c60c4dd49e"}
Jul 09 09:34:03.448546 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:03.448513    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-199.ec2.internal\" not found"
Jul 09 09:34:03.549042 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:03.549021    2549 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-199.ec2.internal\" not found"
Jul 09 09:34:03.642171 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:03.642112    2549 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 09:34:03.736530 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:03.736489    2549 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-199.ec2.internal"
Jul 09 09:34:03.758601 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:03.758579    2549 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 09:34:03.759571 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:03.759552    2549 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-199.ec2.internal"
Jul 09 09:34:03.772699 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:03.772672    2549 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 09:34:03.776441 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:03.776421    2549 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 09:34:04.211664 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.211624    2549 apiserver.go:52] "Watching apiserver"
Jul 09 09:34:04.223158 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.223085    2549 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 09 09:34:04.224420 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.224389    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-8xp94","openshift-multus/multus-additional-cni-plugins-9h8vf","openshift-multus/network-metrics-daemon-rqzsx","openshift-network-diagnostics/network-check-target-ljjkv","openshift-network-operator/iptables-alerter-xqkc6","kube-system/konnectivity-agent-c4jnr","openshift-cluster-node-tuning-operator/tuned-g7dzq","openshift-image-registry/node-ca-nz6rq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-199.ec2.internal","openshift-multus/multus-mfcn7","openshift-ovn-kubernetes/ovnkube-node-j2nr7","kube-system/global-pull-secret-syncer-ccqx8","kube-system/kube-apiserver-proxy-ip-10-0-137-199.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ffq5d"]
Jul 09 09:34:04.227218 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.227196    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.229464 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.229442    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.230206 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.229928    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 09 09:34:04.230206 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.230024    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 09 09:34:04.230206 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.230036    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7wrvv\""
Jul 09 09:34:04.231856 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.231685    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:04.231856 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.231741    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 09 09:34:04.231856 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.231765    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 09 09:34:04.231856 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:04.231794    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqzsx" podUID="72ab801d-5d43-4fbc-a26a-60fab96be57b"
Jul 09 09:34:04.232180 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.232075    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 09 09:34:04.232180 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.232164    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 09 09:34:04.232290 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.232202    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 09 09:34:04.233280 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.233165    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 09 09:34:04.233280 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.233249    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wpfgt\""
Jul 09 09:34:04.234103 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.234082    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:04.234194 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:04.234148    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljjkv" podUID="2ab646db-3779-4dbf-a300-a7b166d929ab"
Jul 09 09:34:04.236508 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.236435    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xqkc6"
Jul 09 09:34:04.236508 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.236459    2549 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 09 09:34:04.238483 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.238455    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c4jnr"
Jul 09 09:34:04.239531 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.239512    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 09 09:34:04.239802 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.239781    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 09 09:34:04.239913 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.239858    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 09 09:34:04.240611 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.240593    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zjhw4\""
Jul 09 09:34:04.241109 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.241090    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ffq5d"
Jul 09 09:34:04.241452 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.241435    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 09 09:34:04.241633 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.241590    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 09 09:34:04.241772 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.241754    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-tt6jd\""
Jul 09 09:34:04.243388 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.243373    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 09 09:34:04.243602 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.243583    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 09 09:34:04.244364 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.243986    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 09 09:34:04.244364 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.244320    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jc9k2\""
Jul 09 09:34:04.244473 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.244371    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nz6rq"
Jul 09 09:34:04.245975 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.245946    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9h8vf"
Jul 09 09:34:04.246479 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.246437    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-etc-sysconfig\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.246479 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.246469    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-run-openvswitch\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.246620 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.246496    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.246620 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.246522    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/35942e40-0ffc-472e-bc29-be3077dc61d5-ovn-node-metrics-cert\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.246620 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.246550    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44vjp\" (UniqueName: \"kubernetes.io/projected/7e60f669-e68e-4467-ad31-c37b3c9883b9-kube-api-access-44vjp\") pod \"iptables-alerter-xqkc6\" (UID: \"7e60f669-e68e-4467-ad31-c37b3c9883b9\") " pod="openshift-network-operator/iptables-alerter-xqkc6"
Jul 09 09:34:04.246620 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.246581    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-host\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.246620 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.246603    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/edcfc3e5-6a61-430a-af17-c26e68349a3e-tmp\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.246861 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.246625    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pp9ph\" (UniqueName: \"kubernetes.io/projected/edcfc3e5-6a61-430a-af17-c26e68349a3e-kube-api-access-pp9ph\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.246861 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.246650    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7e60f669-e68e-4467-ad31-c37b3c9883b9-iptables-alerter-script\") pod \"iptables-alerter-xqkc6\" (UID: \"7e60f669-e68e-4467-ad31-c37b3c9883b9\") " pod="openshift-network-operator/iptables-alerter-xqkc6"
Jul 09 09:34:04.246861 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.246673    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f4defe16-3199-4cea-8b4e-88375572edbf-agent-certs\") pod \"konnectivity-agent-c4jnr\" (UID: \"f4defe16-3199-4cea-8b4e-88375572edbf\") " pod="kube-system/konnectivity-agent-c4jnr"
Jul 09 09:34:04.246861 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.246697    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-var-lib-openvswitch\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.246861 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.246721    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/35942e40-0ffc-472e-bc29-be3077dc61d5-ovnkube-config\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.246861 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.246744    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-etc-kubernetes\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.246861 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.246795    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-etc-sysctl-conf\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.246861 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.246827    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 09 09:34:04.246861 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.246832    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/edcfc3e5-6a61-430a-af17-c26e68349a3e-etc-tuned\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.247353 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.246894    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-host-slash\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.247353 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.246912    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 09 09:34:04.247353 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.246925    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/35942e40-0ffc-472e-bc29-be3077dc61d5-env-overrides\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.247353 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.246956    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8kgd\" (UniqueName: \"kubernetes.io/projected/35942e40-0ffc-472e-bc29-be3077dc61d5-kube-api-access-w8kgd\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.247353 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.246984    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7e60f669-e68e-4467-ad31-c37b3c9883b9-host-slash\") pod \"iptables-alerter-xqkc6\" (UID: \"7e60f669-e68e-4467-ad31-c37b3c9883b9\") " pod="openshift-network-operator/iptables-alerter-xqkc6"
Jul 09 09:34:04.247353 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.247011    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-run\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.247353 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.247044    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-lib-modules\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.247353 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.247107    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 09 09:34:04.247353 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.247239    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-b2lfc\""
Jul 09 09:34:04.247759 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.247618    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm94z\" (UniqueName: \"kubernetes.io/projected/72ab801d-5d43-4fbc-a26a-60fab96be57b-kube-api-access-vm94z\") pod \"network-metrics-daemon-rqzsx\" (UID: \"72ab801d-5d43-4fbc-a26a-60fab96be57b\") " pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:04.247759 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.247653    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6299\" (UniqueName: \"kubernetes.io/projected/2ab646db-3779-4dbf-a300-a7b166d929ab-kube-api-access-t6299\") pod \"network-check-target-ljjkv\" (UID: \"2ab646db-3779-4dbf-a300-a7b166d929ab\") " pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:04.247759 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.247683    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f4defe16-3199-4cea-8b4e-88375572edbf-konnectivity-ca\") pod \"konnectivity-agent-c4jnr\" (UID: \"f4defe16-3199-4cea-8b4e-88375572edbf\") " pod="kube-system/konnectivity-agent-c4jnr"
Jul 09 09:34:04.247759 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.247710    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-run-systemd\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.247966 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.247739    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-etc-openvswitch\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.247966 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.247798    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-node-log\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.247966 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.247847    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-etc-sysctl-d\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.247966 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.247896    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-etc-systemd\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.247966 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.247922    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-systemd-units\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.248180 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.247969    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-run-ovn\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.248180 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.248006    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-log-socket\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.248180 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.248033    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-host-run-ovn-kubernetes\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.248180 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.248056    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-host-cni-netd\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.248180 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.248077    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/35942e40-0ffc-472e-bc29-be3077dc61d5-ovnkube-script-lib\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.248180 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.248103    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-etc-modprobe-d\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.248180 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.248131    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-sys\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.248180 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.248157    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-var-lib-kubelet\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.248507 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.248198    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-host-kubelet\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.248507 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.248220    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-host-run-netns\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.248507 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.248242    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-host-cni-bin\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.248507 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.248265    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72ab801d-5d43-4fbc-a26a-60fab96be57b-metrics-certs\") pod \"network-metrics-daemon-rqzsx\" (UID: \"72ab801d-5d43-4fbc-a26a-60fab96be57b\") " pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:04.248507 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.248354    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.249172 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.249151    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 09 09:34:04.249274 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.249252    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 09 09:34:04.249368 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.249334    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 09 09:34:04.249524 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.249506    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-npq2n\""
Jul 09 09:34:04.249600 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.249539    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 09 09:34:04.249906 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.249844    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 09 09:34:04.250843 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.250327    2549 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 09:34:04.251518 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.251493    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4xh6b\""
Jul 09 09:34:04.251758 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.251736    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 09 09:34:04.253579 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.253562    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8xp94"
Jul 09 09:34:04.255858 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.255838    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 09 09:34:04.255960 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.255887    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xmpks\""
Jul 09 09:34:04.256023 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.255964    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:04.256076 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:04.256024    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccqx8" podUID="282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee"
Jul 09 09:34:04.256306 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.256289    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 09 09:34:04.269530 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.269512    2549 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dmw5l"
Jul 09 09:34:04.278586 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.278567    2549 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dmw5l"
Jul 09 09:34:04.289297 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.289280    2549 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 09:34:04.338135 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.338107    2549 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 09 09:34:04.348802 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.348777    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eae6a875-8c48-46ae-bac2-cbf1b0cfc34f-cni-binary-copy\") pod \"multus-additional-cni-plugins-9h8vf\" (UID: \"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f\") " pod="openshift-multus/multus-additional-cni-plugins-9h8vf"
Jul 09 09:34:04.348974 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.348806    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-host-run-multus-certs\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.348974 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.348835    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-var-lib-kubelet\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.348974 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.348859    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-host-run-netns\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.348974 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.348906    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-host-cni-bin\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.348974 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.348932    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72ab801d-5d43-4fbc-a26a-60fab96be57b-metrics-certs\") pod \"network-metrics-daemon-rqzsx\" (UID: \"72ab801d-5d43-4fbc-a26a-60fab96be57b\") " pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:04.348974 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.348951    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-host-run-netns\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.348974 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.348956    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44vjp\" (UniqueName: \"kubernetes.io/projected/7e60f669-e68e-4467-ad31-c37b3c9883b9-kube-api-access-44vjp\") pod \"iptables-alerter-xqkc6\" (UID: \"7e60f669-e68e-4467-ad31-c37b3c9883b9\") " pod="openshift-network-operator/iptables-alerter-xqkc6"
Jul 09 09:34:04.349345 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349035    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2e407356-17fe-4a9f-81da-417042050209-serviceca\") pod \"node-ca-nz6rq\" (UID: \"2e407356-17fe-4a9f-81da-417042050209\") " pod="openshift-image-registry/node-ca-nz6rq"
Jul 09 09:34:04.349345 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349058    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-kubelet-config\") pod \"global-pull-secret-syncer-ccqx8\" (UID: \"282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee\") " pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:04.349345 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:04.349075    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 09:34:04.349345 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349080    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/885b74bd-5140-4025-9485-046e3eacfc6c-sys-fs\") pod \"aws-ebs-csi-driver-node-ffq5d\" (UID: \"885b74bd-5140-4025-9485-046e3eacfc6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ffq5d"
Jul 09 09:34:04.349345 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:04.349174    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72ab801d-5d43-4fbc-a26a-60fab96be57b-metrics-certs podName:72ab801d-5d43-4fbc-a26a-60fab96be57b nodeName:}" failed. No retries permitted until 2026-07-09 09:34:04.8491311 +0000 UTC m=+3.020289648 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72ab801d-5d43-4fbc-a26a-60fab96be57b-metrics-certs") pod "network-metrics-daemon-rqzsx" (UID: "72ab801d-5d43-4fbc-a26a-60fab96be57b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 09:34:04.349345 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349217    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-etc-sysconfig\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.349345 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349249    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.349345 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349260    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-var-lib-kubelet\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.349345 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349276    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/35942e40-0ffc-472e-bc29-be3077dc61d5-ovn-node-metrics-cert\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.349345 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349302    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-etc-sysconfig\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.349345 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349310    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.349345 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349327    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-host-cni-bin\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.349954 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349366    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/885b74bd-5140-4025-9485-046e3eacfc6c-socket-dir\") pod \"aws-ebs-csi-driver-node-ffq5d\" (UID: \"885b74bd-5140-4025-9485-046e3eacfc6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ffq5d"
Jul 09 09:34:04.349954 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349395    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/79826bec-2b69-42e5-87f1-94c078b40ac4-multus-daemon-config\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.349954 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349421    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2fjf\" (UniqueName: \"kubernetes.io/projected/79826bec-2b69-42e5-87f1-94c078b40ac4-kube-api-access-t2fjf\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.349954 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349449    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-host\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.349954 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349501    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-host\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.349954 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349517    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/edcfc3e5-6a61-430a-af17-c26e68349a3e-tmp\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.349954 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349560    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7e60f669-e68e-4467-ad31-c37b3c9883b9-iptables-alerter-script\") pod \"iptables-alerter-xqkc6\" (UID: \"7e60f669-e68e-4467-ad31-c37b3c9883b9\") " pod="openshift-network-operator/iptables-alerter-xqkc6"
Jul 09 09:34:04.349954 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349592    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-original-pull-secret\") pod \"global-pull-secret-syncer-ccqx8\" (UID: \"282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee\") " pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:04.349954 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349619    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/885b74bd-5140-4025-9485-046e3eacfc6c-registration-dir\") pod \"aws-ebs-csi-driver-node-ffq5d\" (UID: \"885b74bd-5140-4025-9485-046e3eacfc6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ffq5d"
Jul 09 09:34:04.349954 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349636    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-host-run-k8s-cni-cncf-io\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.349954 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349645    2549 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 09 09:34:04.349954 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349677    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f4defe16-3199-4cea-8b4e-88375572edbf-agent-certs\") pod \"konnectivity-agent-c4jnr\" (UID: \"f4defe16-3199-4cea-8b4e-88375572edbf\") " pod="kube-system/konnectivity-agent-c4jnr"
Jul 09 09:34:04.349954 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349747    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-var-lib-openvswitch\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.349954 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349774    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/35942e40-0ffc-472e-bc29-be3077dc61d5-ovnkube-config\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.349954 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349846    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-var-lib-openvswitch\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.349954 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349892    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-dbus\") pod \"global-pull-secret-syncer-ccqx8\" (UID: \"282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee\") " pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:04.349954 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349918    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eae6a875-8c48-46ae-bac2-cbf1b0cfc34f-os-release\") pod \"multus-additional-cni-plugins-9h8vf\" (UID: \"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f\") " pod="openshift-multus/multus-additional-cni-plugins-9h8vf"
Jul 09 09:34:04.350607 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349941    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eae6a875-8c48-46ae-bac2-cbf1b0cfc34f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9h8vf\" (UID: \"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f\") " pod="openshift-multus/multus-additional-cni-plugins-9h8vf"
Jul 09 09:34:04.350607 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349966    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kc24f\" (UniqueName: \"kubernetes.io/projected/eae6a875-8c48-46ae-bac2-cbf1b0cfc34f-kube-api-access-kc24f\") pod \"multus-additional-cni-plugins-9h8vf\" (UID: \"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f\") " pod="openshift-multus/multus-additional-cni-plugins-9h8vf"
Jul 09 09:34:04.350607 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.349992    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/885b74bd-5140-4025-9485-046e3eacfc6c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ffq5d\" (UID: \"885b74bd-5140-4025-9485-046e3eacfc6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ffq5d"
Jul 09 09:34:04.350607 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350017    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-etc-sysctl-conf\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.350607 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350042    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-host-slash\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.350607 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350064    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/35942e40-0ffc-472e-bc29-be3077dc61d5-env-overrides\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.350607 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350088    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2e407356-17fe-4a9f-81da-417042050209-host\") pod \"node-ca-nz6rq\" (UID: \"2e407356-17fe-4a9f-81da-417042050209\") " pod="openshift-image-registry/node-ca-nz6rq"
Jul 09 09:34:04.350607 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350112    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg6cw\" (UniqueName: \"kubernetes.io/projected/2e407356-17fe-4a9f-81da-417042050209-kube-api-access-cg6cw\") pod \"node-ca-nz6rq\" (UID: \"2e407356-17fe-4a9f-81da-417042050209\") " pod="openshift-image-registry/node-ca-nz6rq"
Jul 09 09:34:04.350607 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350128    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7e60f669-e68e-4467-ad31-c37b3c9883b9-iptables-alerter-script\") pod \"iptables-alerter-xqkc6\" (UID: \"7e60f669-e68e-4467-ad31-c37b3c9883b9\") " pod="openshift-network-operator/iptables-alerter-xqkc6"
Jul 09 09:34:04.350607 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350138    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83ccd32a-ea2e-4ad9-95c7-f79c33043432-tmp-dir\") pod \"node-resolver-8xp94\" (UID: \"83ccd32a-ea2e-4ad9-95c7-f79c33043432\") " pod="openshift-dns/node-resolver-8xp94"
Jul 09 09:34:04.350607 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350165    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-lib-modules\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.350607 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350189    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/885b74bd-5140-4025-9485-046e3eacfc6c-etc-selinux\") pod \"aws-ebs-csi-driver-node-ffq5d\" (UID: \"885b74bd-5140-4025-9485-046e3eacfc6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ffq5d"
Jul 09 09:34:04.350607 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350223    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-cnibin\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.350607 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350248    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/79826bec-2b69-42e5-87f1-94c078b40ac4-cni-binary-copy\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.350607 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350274    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-multus-socket-dir-parent\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.350607 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350287    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-etc-sysctl-conf\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.350607 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350300    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-run-systemd\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.351393 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350326    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/35942e40-0ffc-472e-bc29-be3077dc61d5-ovnkube-config\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.351393 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350332    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eae6a875-8c48-46ae-bac2-cbf1b0cfc34f-system-cni-dir\") pod \"multus-additional-cni-plugins-9h8vf\" (UID: \"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f\") " pod="openshift-multus/multus-additional-cni-plugins-9h8vf"
Jul 09 09:34:04.351393 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350377    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-run-systemd\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.351393 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350409    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-etc-sysctl-d\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.351393 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350415    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-host-slash\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.351393 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350437    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-run-ovn\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.351393 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350472    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-lib-modules\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.351393 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350501    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-host-run-ovn-kubernetes\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.351393 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350545    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/885b74bd-5140-4025-9485-046e3eacfc6c-device-dir\") pod \"aws-ebs-csi-driver-node-ffq5d\" (UID: \"885b74bd-5140-4025-9485-046e3eacfc6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ffq5d"
Jul 09 09:34:04.351393 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350572    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-multus-cni-dir\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.351393 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350574    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-host-run-ovn-kubernetes\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.351393 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350596    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-etc-sysctl-d\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.351393 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350609    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-multus-conf-dir\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.351393 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350634    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-run-ovn\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.351393 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350672    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-etc-kubernetes\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.351393 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350710    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-etc-modprobe-d\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.351393 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350749    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-sys\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.352161 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350777    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-host-kubelet\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.352161 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350823    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/35942e40-0ffc-472e-bc29-be3077dc61d5-env-overrides\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.352161 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350829    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-system-cni-dir\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.352161 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350821    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-etc-modprobe-d\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.352161 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350854    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-host-kubelet\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.352161 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350857    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-os-release\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.352161 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350909    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-sys\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.352161 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350942    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-run-openvswitch\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.352161 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.350975    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-host-var-lib-cni-bin\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.352161 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.351002    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-run-openvswitch\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.352161 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.351052    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-host-var-lib-kubelet\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.352161 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.351078    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-hostroot\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.352161 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.351107    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pp9ph\" (UniqueName: \"kubernetes.io/projected/edcfc3e5-6a61-430a-af17-c26e68349a3e-kube-api-access-pp9ph\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.352161 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.351131    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eae6a875-8c48-46ae-bac2-cbf1b0cfc34f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9h8vf\" (UID: \"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f\") " pod="openshift-multus/multus-additional-cni-plugins-9h8vf"
Jul 09 09:34:04.352161 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.351257    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-etc-kubernetes\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.352161 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.351284    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/edcfc3e5-6a61-430a-af17-c26e68349a3e-etc-tuned\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.352161 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.351310    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8kgd\" (UniqueName: \"kubernetes.io/projected/35942e40-0ffc-472e-bc29-be3077dc61d5-kube-api-access-w8kgd\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.352953 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.351324    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-etc-kubernetes\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.352953 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.351361    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7e60f669-e68e-4467-ad31-c37b3c9883b9-host-slash\") pod \"iptables-alerter-xqkc6\" (UID: \"7e60f669-e68e-4467-ad31-c37b3c9883b9\") " pod="openshift-network-operator/iptables-alerter-xqkc6"
Jul 09 09:34:04.352953 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.351383    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/83ccd32a-ea2e-4ad9-95c7-f79c33043432-hosts-file\") pod \"node-resolver-8xp94\" (UID: \"83ccd32a-ea2e-4ad9-95c7-f79c33043432\") " pod="openshift-dns/node-resolver-8xp94"
Jul 09 09:34:04.352953 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.351414    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbb98\" (UniqueName: \"kubernetes.io/projected/885b74bd-5140-4025-9485-046e3eacfc6c-kube-api-access-rbb98\") pod \"aws-ebs-csi-driver-node-ffq5d\" (UID: \"885b74bd-5140-4025-9485-046e3eacfc6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ffq5d"
Jul 09 09:34:04.352953 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.351435    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eae6a875-8c48-46ae-bac2-cbf1b0cfc34f-cnibin\") pod \"multus-additional-cni-plugins-9h8vf\" (UID: \"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f\") " pod="openshift-multus/multus-additional-cni-plugins-9h8vf"
Jul 09 09:34:04.352953 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.351455    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eae6a875-8c48-46ae-bac2-cbf1b0cfc34f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9h8vf\" (UID: \"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f\") " pod="openshift-multus/multus-additional-cni-plugins-9h8vf"
Jul 09 09:34:04.352953 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.351475    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-run\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.352953 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.351498    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vm94z\" (UniqueName: \"kubernetes.io/projected/72ab801d-5d43-4fbc-a26a-60fab96be57b-kube-api-access-vm94z\") pod \"network-metrics-daemon-rqzsx\" (UID: \"72ab801d-5d43-4fbc-a26a-60fab96be57b\") " pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:04.352953 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.351545    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6299\" (UniqueName: \"kubernetes.io/projected/2ab646db-3779-4dbf-a300-a7b166d929ab-kube-api-access-t6299\") pod \"network-check-target-ljjkv\" (UID: \"2ab646db-3779-4dbf-a300-a7b166d929ab\") " pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:04.352953 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.351580    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4qj8\" (UniqueName: \"kubernetes.io/projected/83ccd32a-ea2e-4ad9-95c7-f79c33043432-kube-api-access-j4qj8\") pod \"node-resolver-8xp94\" (UID: \"83ccd32a-ea2e-4ad9-95c7-f79c33043432\") " pod="openshift-dns/node-resolver-8xp94"
Jul 09 09:34:04.352953 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.351606    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-host-var-lib-cni-multus\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.352953 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.351632    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f4defe16-3199-4cea-8b4e-88375572edbf-konnectivity-ca\") pod \"konnectivity-agent-c4jnr\" (UID: \"f4defe16-3199-4cea-8b4e-88375572edbf\") " pod="kube-system/konnectivity-agent-c4jnr"
Jul 09 09:34:04.352953 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.351656    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-etc-openvswitch\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.352953 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.351679    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-node-log\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.352953 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.351703    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-host-run-netns\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.352953 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.351729    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-etc-systemd\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.352953 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.351787    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-systemd-units\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.353755 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.351813    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-log-socket\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.353755 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.351839    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-host-cni-netd\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.353755 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.351893    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/35942e40-0ffc-472e-bc29-be3077dc61d5-ovnkube-script-lib\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.353755 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.352005    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7e60f669-e68e-4467-ad31-c37b3c9883b9-host-slash\") pod \"iptables-alerter-xqkc6\" (UID: \"7e60f669-e68e-4467-ad31-c37b3c9883b9\") " pod="openshift-network-operator/iptables-alerter-xqkc6"
Jul 09 09:34:04.353755 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.352071    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-etc-systemd\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.353755 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.352161    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/edcfc3e5-6a61-430a-af17-c26e68349a3e-run\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.353755 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.352370    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-etc-openvswitch\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.353755 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.352400    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/35942e40-0ffc-472e-bc29-be3077dc61d5-ovnkube-script-lib\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.353755 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.352427    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-node-log\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.353755 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.352451    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-systemd-units\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.353755 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.352464    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-log-socket\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.353755 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.352489    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/35942e40-0ffc-472e-bc29-be3077dc61d5-host-cni-netd\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.353755 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.352745    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f4defe16-3199-4cea-8b4e-88375572edbf-konnectivity-ca\") pod \"konnectivity-agent-c4jnr\" (UID: \"f4defe16-3199-4cea-8b4e-88375572edbf\") " pod="kube-system/konnectivity-agent-c4jnr"
Jul 09 09:34:04.353755 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.353586    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/35942e40-0ffc-472e-bc29-be3077dc61d5-ovn-node-metrics-cert\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.354423 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.354238    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f4defe16-3199-4cea-8b4e-88375572edbf-agent-certs\") pod \"konnectivity-agent-c4jnr\" (UID: \"f4defe16-3199-4cea-8b4e-88375572edbf\") " pod="kube-system/konnectivity-agent-c4jnr"
Jul 09 09:34:04.354635 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.354614    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/edcfc3e5-6a61-430a-af17-c26e68349a3e-tmp\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.355028 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.354891    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/edcfc3e5-6a61-430a-af17-c26e68349a3e-etc-tuned\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.357831 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.357806    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44vjp\" (UniqueName: \"kubernetes.io/projected/7e60f669-e68e-4467-ad31-c37b3c9883b9-kube-api-access-44vjp\") pod \"iptables-alerter-xqkc6\" (UID: \"7e60f669-e68e-4467-ad31-c37b3c9883b9\") " pod="openshift-network-operator/iptables-alerter-xqkc6"
Jul 09 09:34:04.358087 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:04.358069    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 09:34:04.358195 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:04.358095    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 09:34:04.358195 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:04.358108    2549 projected.go:196] Error preparing data for projected volume kube-api-access-t6299 for pod openshift-network-diagnostics/network-check-target-ljjkv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 09:34:04.358195 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:04.358166    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ab646db-3779-4dbf-a300-a7b166d929ab-kube-api-access-t6299 podName:2ab646db-3779-4dbf-a300-a7b166d929ab nodeName:}" failed. No retries permitted until 2026-07-09 09:34:04.858149158 +0000 UTC m=+3.029307692 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-t6299" (UniqueName: "kubernetes.io/projected/2ab646db-3779-4dbf-a300-a7b166d929ab-kube-api-access-t6299") pod "network-check-target-ljjkv" (UID: "2ab646db-3779-4dbf-a300-a7b166d929ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 09:34:04.360301 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.360279    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pp9ph\" (UniqueName: \"kubernetes.io/projected/edcfc3e5-6a61-430a-af17-c26e68349a3e-kube-api-access-pp9ph\") pod \"tuned-g7dzq\" (UID: \"edcfc3e5-6a61-430a-af17-c26e68349a3e\") " pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.360686 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.360669    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8kgd\" (UniqueName: \"kubernetes.io/projected/35942e40-0ffc-472e-bc29-be3077dc61d5-kube-api-access-w8kgd\") pod \"ovnkube-node-j2nr7\" (UID: \"35942e40-0ffc-472e-bc29-be3077dc61d5\") " pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.361977 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.361956    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm94z\" (UniqueName: \"kubernetes.io/projected/72ab801d-5d43-4fbc-a26a-60fab96be57b-kube-api-access-vm94z\") pod \"network-metrics-daemon-rqzsx\" (UID: \"72ab801d-5d43-4fbc-a26a-60fab96be57b\") " pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:04.452477 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.452419    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/885b74bd-5140-4025-9485-046e3eacfc6c-device-dir\") pod \"aws-ebs-csi-driver-node-ffq5d\" (UID: \"885b74bd-5140-4025-9485-046e3eacfc6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ffq5d"
Jul 09 09:34:04.452477 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.452466    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-multus-cni-dir\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.452745 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.452490    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-multus-conf-dir\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.452745 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.452513    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-etc-kubernetes\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.452745 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.452539    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-system-cni-dir\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.452745 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.452561    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-os-release\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.452745 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.452585    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-host-var-lib-cni-bin\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.452745 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.452609    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-host-var-lib-kubelet\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.452745 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.452633    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-hostroot\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.452745 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.452657    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eae6a875-8c48-46ae-bac2-cbf1b0cfc34f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9h8vf\" (UID: \"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f\") " pod="openshift-multus/multus-additional-cni-plugins-9h8vf"
Jul 09 09:34:04.452745 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.452687    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/83ccd32a-ea2e-4ad9-95c7-f79c33043432-hosts-file\") pod \"node-resolver-8xp94\" (UID: \"83ccd32a-ea2e-4ad9-95c7-f79c33043432\") " pod="openshift-dns/node-resolver-8xp94"
Jul 09 09:34:04.452745 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.452713    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbb98\" (UniqueName: \"kubernetes.io/projected/885b74bd-5140-4025-9485-046e3eacfc6c-kube-api-access-rbb98\") pod \"aws-ebs-csi-driver-node-ffq5d\" (UID: \"885b74bd-5140-4025-9485-046e3eacfc6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ffq5d"
Jul 09 09:34:04.452745 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.452738    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eae6a875-8c48-46ae-bac2-cbf1b0cfc34f-cnibin\") pod \"multus-additional-cni-plugins-9h8vf\" (UID: \"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f\") " pod="openshift-multus/multus-additional-cni-plugins-9h8vf"
Jul 09 09:34:04.453206 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.452762    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eae6a875-8c48-46ae-bac2-cbf1b0cfc34f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9h8vf\" (UID: \"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f\") " pod="openshift-multus/multus-additional-cni-plugins-9h8vf"
Jul 09 09:34:04.453206 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.452804    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4qj8\" (UniqueName: \"kubernetes.io/projected/83ccd32a-ea2e-4ad9-95c7-f79c33043432-kube-api-access-j4qj8\") pod \"node-resolver-8xp94\" (UID: \"83ccd32a-ea2e-4ad9-95c7-f79c33043432\") " pod="openshift-dns/node-resolver-8xp94"
Jul 09 09:34:04.453206 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.452826    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-host-var-lib-cni-multus\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.453206 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.452856    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-host-run-netns\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.453206 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.452905    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eae6a875-8c48-46ae-bac2-cbf1b0cfc34f-cni-binary-copy\") pod \"multus-additional-cni-plugins-9h8vf\" (UID: \"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f\") " pod="openshift-multus/multus-additional-cni-plugins-9h8vf"
Jul 09 09:34:04.453206 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.452929    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-host-run-multus-certs\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.453206 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.452961    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-hostroot\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.453206 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.452979    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2e407356-17fe-4a9f-81da-417042050209-serviceca\") pod \"node-ca-nz6rq\" (UID: \"2e407356-17fe-4a9f-81da-417042050209\") " pod="openshift-image-registry/node-ca-nz6rq"
Jul 09 09:34:04.453206 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453037    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/885b74bd-5140-4025-9485-046e3eacfc6c-device-dir\") pod \"aws-ebs-csi-driver-node-ffq5d\" (UID: \"885b74bd-5140-4025-9485-046e3eacfc6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ffq5d"
Jul 09 09:34:04.453206 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453073    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-kubelet-config\") pod \"global-pull-secret-syncer-ccqx8\" (UID: \"282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee\") " pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:04.453206 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453127    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-kubelet-config\") pod \"global-pull-secret-syncer-ccqx8\" (UID: \"282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee\") " pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:04.453206 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453149    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-host-run-netns\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.453878 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453260    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-multus-cni-dir\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.453878 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453298    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-multus-conf-dir\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.453878 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453335    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-etc-kubernetes\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.453878 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453382    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-system-cni-dir\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.453878 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453416    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-host-var-lib-cni-multus\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.453878 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453439    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-os-release\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.453878 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453477    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-host-var-lib-cni-bin\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.453878 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453483    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2e407356-17fe-4a9f-81da-417042050209-serviceca\") pod \"node-ca-nz6rq\" (UID: \"2e407356-17fe-4a9f-81da-417042050209\") " pod="openshift-image-registry/node-ca-nz6rq"
Jul 09 09:34:04.453878 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453512    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-host-var-lib-kubelet\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.453878 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453534    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-host-run-multus-certs\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.453878 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453557    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/83ccd32a-ea2e-4ad9-95c7-f79c33043432-hosts-file\") pod \"node-resolver-8xp94\" (UID: \"83ccd32a-ea2e-4ad9-95c7-f79c33043432\") " pod="openshift-dns/node-resolver-8xp94"
Jul 09 09:34:04.453878 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453647    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eae6a875-8c48-46ae-bac2-cbf1b0cfc34f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9h8vf\" (UID: \"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f\") " pod="openshift-multus/multus-additional-cni-plugins-9h8vf"
Jul 09 09:34:04.453878 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453688    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/885b74bd-5140-4025-9485-046e3eacfc6c-sys-fs\") pod \"aws-ebs-csi-driver-node-ffq5d\" (UID: \"885b74bd-5140-4025-9485-046e3eacfc6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ffq5d"
Jul 09 09:34:04.453878 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453718    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/885b74bd-5140-4025-9485-046e3eacfc6c-socket-dir\") pod \"aws-ebs-csi-driver-node-ffq5d\" (UID: \"885b74bd-5140-4025-9485-046e3eacfc6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ffq5d"
Jul 09 09:34:04.453878 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453724    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eae6a875-8c48-46ae-bac2-cbf1b0cfc34f-cnibin\") pod \"multus-additional-cni-plugins-9h8vf\" (UID: \"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f\") " pod="openshift-multus/multus-additional-cni-plugins-9h8vf"
Jul 09 09:34:04.453878 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453747    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/79826bec-2b69-42e5-87f1-94c078b40ac4-multus-daemon-config\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.453878 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453773    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2fjf\" (UniqueName: \"kubernetes.io/projected/79826bec-2b69-42e5-87f1-94c078b40ac4-kube-api-access-t2fjf\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.453878 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453778    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/885b74bd-5140-4025-9485-046e3eacfc6c-sys-fs\") pod \"aws-ebs-csi-driver-node-ffq5d\" (UID: \"885b74bd-5140-4025-9485-046e3eacfc6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ffq5d"
Jul 09 09:34:04.454728 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453799    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-original-pull-secret\") pod \"global-pull-secret-syncer-ccqx8\" (UID: \"282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee\") " pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:04.454728 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453823    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/885b74bd-5140-4025-9485-046e3eacfc6c-registration-dir\") pod \"aws-ebs-csi-driver-node-ffq5d\" (UID: \"885b74bd-5140-4025-9485-046e3eacfc6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ffq5d"
Jul 09 09:34:04.454728 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453848    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-host-run-k8s-cni-cncf-io\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.454728 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453895    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-dbus\") pod \"global-pull-secret-syncer-ccqx8\" (UID: \"282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee\") " pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:04.454728 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453911    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eae6a875-8c48-46ae-bac2-cbf1b0cfc34f-cni-binary-copy\") pod \"multus-additional-cni-plugins-9h8vf\" (UID: \"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f\") " pod="openshift-multus/multus-additional-cni-plugins-9h8vf"
Jul 09 09:34:04.454728 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453920    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eae6a875-8c48-46ae-bac2-cbf1b0cfc34f-os-release\") pod \"multus-additional-cni-plugins-9h8vf\" (UID: \"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f\") " pod="openshift-multus/multus-additional-cni-plugins-9h8vf"
Jul 09 09:34:04.454728 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453944    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eae6a875-8c48-46ae-bac2-cbf1b0cfc34f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9h8vf\" (UID: \"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f\") " pod="openshift-multus/multus-additional-cni-plugins-9h8vf"
Jul 09 09:34:04.454728 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453970    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kc24f\" (UniqueName: \"kubernetes.io/projected/eae6a875-8c48-46ae-bac2-cbf1b0cfc34f-kube-api-access-kc24f\") pod \"multus-additional-cni-plugins-9h8vf\" (UID: \"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f\") " pod="openshift-multus/multus-additional-cni-plugins-9h8vf"
Jul 09 09:34:04.454728 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453978    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-host-run-k8s-cni-cncf-io\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.454728 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453995    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/885b74bd-5140-4025-9485-046e3eacfc6c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ffq5d\" (UID: \"885b74bd-5140-4025-9485-046e3eacfc6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ffq5d"
Jul 09 09:34:04.454728 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:04.454006    2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 09:34:04.454728 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.454036    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2e407356-17fe-4a9f-81da-417042050209-host\") pod \"node-ca-nz6rq\" (UID: \"2e407356-17fe-4a9f-81da-417042050209\") " pod="openshift-image-registry/node-ca-nz6rq"
Jul 09 09:34:04.454728 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.454047    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eae6a875-8c48-46ae-bac2-cbf1b0cfc34f-os-release\") pod \"multus-additional-cni-plugins-9h8vf\" (UID: \"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f\") " pod="openshift-multus/multus-additional-cni-plugins-9h8vf"
Jul 09 09:34:04.454728 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.454049    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/885b74bd-5140-4025-9485-046e3eacfc6c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ffq5d\" (UID: \"885b74bd-5140-4025-9485-046e3eacfc6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ffq5d"
Jul 09 09:34:04.454728 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.453987    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/885b74bd-5140-4025-9485-046e3eacfc6c-registration-dir\") pod \"aws-ebs-csi-driver-node-ffq5d\" (UID: \"885b74bd-5140-4025-9485-046e3eacfc6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ffq5d"
Jul 09 09:34:04.454728 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.454064    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-dbus\") pod \"global-pull-secret-syncer-ccqx8\" (UID: \"282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee\") " pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:04.454728 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:04.454076    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-original-pull-secret podName:282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee nodeName:}" failed. No retries permitted until 2026-07-09 09:34:04.954054782 +0000 UTC m=+3.125213331 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-original-pull-secret") pod "global-pull-secret-syncer-ccqx8" (UID: "282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee") : object "kube-system"/"original-pull-secret" not registered
Jul 09 09:34:04.455449 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.454090    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2e407356-17fe-4a9f-81da-417042050209-host\") pod \"node-ca-nz6rq\" (UID: \"2e407356-17fe-4a9f-81da-417042050209\") " pod="openshift-image-registry/node-ca-nz6rq"
Jul 09 09:34:04.455449 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.454100    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg6cw\" (UniqueName: \"kubernetes.io/projected/2e407356-17fe-4a9f-81da-417042050209-kube-api-access-cg6cw\") pod \"node-ca-nz6rq\" (UID: \"2e407356-17fe-4a9f-81da-417042050209\") " pod="openshift-image-registry/node-ca-nz6rq"
Jul 09 09:34:04.455449 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.454132    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83ccd32a-ea2e-4ad9-95c7-f79c33043432-tmp-dir\") pod \"node-resolver-8xp94\" (UID: \"83ccd32a-ea2e-4ad9-95c7-f79c33043432\") " pod="openshift-dns/node-resolver-8xp94"
Jul 09 09:34:04.455449 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.454150    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/885b74bd-5140-4025-9485-046e3eacfc6c-socket-dir\") pod \"aws-ebs-csi-driver-node-ffq5d\" (UID: \"885b74bd-5140-4025-9485-046e3eacfc6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ffq5d"
Jul 09 09:34:04.455449 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.454163    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/885b74bd-5140-4025-9485-046e3eacfc6c-etc-selinux\") pod \"aws-ebs-csi-driver-node-ffq5d\" (UID: \"885b74bd-5140-4025-9485-046e3eacfc6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ffq5d"
Jul 09 09:34:04.455449 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.454009    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eae6a875-8c48-46ae-bac2-cbf1b0cfc34f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9h8vf\" (UID: \"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f\") " pod="openshift-multus/multus-additional-cni-plugins-9h8vf"
Jul 09 09:34:04.455449 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.454187    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-cnibin\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.455449 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.454210    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/79826bec-2b69-42e5-87f1-94c078b40ac4-cni-binary-copy\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.455449 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.454226    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eae6a875-8c48-46ae-bac2-cbf1b0cfc34f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9h8vf\" (UID: \"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f\") " pod="openshift-multus/multus-additional-cni-plugins-9h8vf"
Jul 09 09:34:04.455449 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.454238    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-multus-socket-dir-parent\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.455449 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.454267    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eae6a875-8c48-46ae-bac2-cbf1b0cfc34f-system-cni-dir\") pod \"multus-additional-cni-plugins-9h8vf\" (UID: \"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f\") " pod="openshift-multus/multus-additional-cni-plugins-9h8vf"
Jul 09 09:34:04.455449 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.454282    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-cnibin\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.455449 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.454319    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/79826bec-2b69-42e5-87f1-94c078b40ac4-multus-daemon-config\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.455449 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.454334    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/885b74bd-5140-4025-9485-046e3eacfc6c-etc-selinux\") pod \"aws-ebs-csi-driver-node-ffq5d\" (UID: \"885b74bd-5140-4025-9485-046e3eacfc6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ffq5d"
Jul 09 09:34:04.455449 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.454337    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eae6a875-8c48-46ae-bac2-cbf1b0cfc34f-system-cni-dir\") pod \"multus-additional-cni-plugins-9h8vf\" (UID: \"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f\") " pod="openshift-multus/multus-additional-cni-plugins-9h8vf"
Jul 09 09:34:04.455449 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.454337    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/79826bec-2b69-42e5-87f1-94c078b40ac4-multus-socket-dir-parent\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.455449 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.454437    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83ccd32a-ea2e-4ad9-95c7-f79c33043432-tmp-dir\") pod \"node-resolver-8xp94\" (UID: \"83ccd32a-ea2e-4ad9-95c7-f79c33043432\") " pod="openshift-dns/node-resolver-8xp94"
Jul 09 09:34:04.455449 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.454670    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/79826bec-2b69-42e5-87f1-94c078b40ac4-cni-binary-copy\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.465703 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.465622    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg6cw\" (UniqueName: \"kubernetes.io/projected/2e407356-17fe-4a9f-81da-417042050209-kube-api-access-cg6cw\") pod \"node-ca-nz6rq\" (UID: \"2e407356-17fe-4a9f-81da-417042050209\") " pod="openshift-image-registry/node-ca-nz6rq"
Jul 09 09:34:04.465810 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.465740    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2fjf\" (UniqueName: \"kubernetes.io/projected/79826bec-2b69-42e5-87f1-94c078b40ac4-kube-api-access-t2fjf\") pod \"multus-mfcn7\" (UID: \"79826bec-2b69-42e5-87f1-94c078b40ac4\") " pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.466370 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.466346    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc24f\" (UniqueName: \"kubernetes.io/projected/eae6a875-8c48-46ae-bac2-cbf1b0cfc34f-kube-api-access-kc24f\") pod \"multus-additional-cni-plugins-9h8vf\" (UID: \"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f\") " pod="openshift-multus/multus-additional-cni-plugins-9h8vf"
Jul 09 09:34:04.466554 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.466522    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4qj8\" (UniqueName: \"kubernetes.io/projected/83ccd32a-ea2e-4ad9-95c7-f79c33043432-kube-api-access-j4qj8\") pod \"node-resolver-8xp94\" (UID: \"83ccd32a-ea2e-4ad9-95c7-f79c33043432\") " pod="openshift-dns/node-resolver-8xp94"
Jul 09 09:34:04.467387 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.467355    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbb98\" (UniqueName: \"kubernetes.io/projected/885b74bd-5140-4025-9485-046e3eacfc6c-kube-api-access-rbb98\") pod \"aws-ebs-csi-driver-node-ffq5d\" (UID: \"885b74bd-5140-4025-9485-046e3eacfc6c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ffq5d"
Jul 09 09:34:04.540278 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.540245    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-g7dzq"
Jul 09 09:34:04.548576 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.548552    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:04.559835 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.559813    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xqkc6"
Jul 09 09:34:04.566899 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.566879    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c4jnr"
Jul 09 09:34:04.573588 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.573568    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ffq5d"
Jul 09 09:34:04.581255 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.581234    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nz6rq"
Jul 09 09:34:04.587972 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.587955    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9h8vf"
Jul 09 09:34:04.595563 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.595547    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mfcn7"
Jul 09 09:34:04.602334 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.602316    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8xp94"
Jul 09 09:34:04.856131 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.856059    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72ab801d-5d43-4fbc-a26a-60fab96be57b-metrics-certs\") pod \"network-metrics-daemon-rqzsx\" (UID: \"72ab801d-5d43-4fbc-a26a-60fab96be57b\") " pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:04.856272 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:04.856222    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 09:34:04.856313 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:04.856284    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72ab801d-5d43-4fbc-a26a-60fab96be57b-metrics-certs podName:72ab801d-5d43-4fbc-a26a-60fab96be57b nodeName:}" failed. No retries permitted until 2026-07-09 09:34:05.856269564 +0000 UTC m=+4.027428114 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72ab801d-5d43-4fbc-a26a-60fab96be57b-metrics-certs") pod "network-metrics-daemon-rqzsx" (UID: "72ab801d-5d43-4fbc-a26a-60fab96be57b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 09:34:04.956688 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.956655    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6299\" (UniqueName: \"kubernetes.io/projected/2ab646db-3779-4dbf-a300-a7b166d929ab-kube-api-access-t6299\") pod \"network-check-target-ljjkv\" (UID: \"2ab646db-3779-4dbf-a300-a7b166d929ab\") " pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:04.956898 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:04.956719    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-original-pull-secret\") pod \"global-pull-secret-syncer-ccqx8\" (UID: \"282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee\") " pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:04.956898 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:04.956784    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 09:34:04.956898 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:04.956807    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 09:34:04.956898 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:04.956818    2549 projected.go:196] Error preparing data for projected volume kube-api-access-t6299 for pod openshift-network-diagnostics/network-check-target-ljjkv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 09:34:04.956898 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:04.956818    2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 09:34:04.956898 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:04.956883    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-original-pull-secret podName:282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee nodeName:}" failed. No retries permitted until 2026-07-09 09:34:05.956858119 +0000 UTC m=+4.128016667 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-original-pull-secret") pod "global-pull-secret-syncer-ccqx8" (UID: "282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee") : object "kube-system"/"original-pull-secret" not registered
Jul 09 09:34:04.956898 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:04.956899    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ab646db-3779-4dbf-a300-a7b166d929ab-kube-api-access-t6299 podName:2ab646db-3779-4dbf-a300-a7b166d929ab nodeName:}" failed. No retries permitted until 2026-07-09 09:34:05.956892715 +0000 UTC m=+4.128051248 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-t6299" (UniqueName: "kubernetes.io/projected/2ab646db-3779-4dbf-a300-a7b166d929ab-kube-api-access-t6299") pod "network-check-target-ljjkv" (UID: "2ab646db-3779-4dbf-a300-a7b166d929ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 09:34:05.202251 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:05.202217    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedcfc3e5_6a61_430a_af17_c26e68349a3e.slice/crio-b23fbedba2ceafc74947997fe1dfd6299f9cbcb0f5bb616fd328621351498308 WatchSource:0}: Error finding container b23fbedba2ceafc74947997fe1dfd6299f9cbcb0f5bb616fd328621351498308: Status 404 returned error can't find the container with id b23fbedba2ceafc74947997fe1dfd6299f9cbcb0f5bb616fd328621351498308
Jul 09 09:34:05.205689 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:05.205655    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod885b74bd_5140_4025_9485_046e3eacfc6c.slice/crio-55f2d8464f3c5f3f98bb32ccc4f06ec8e93c1a83cd4ffdcc84ae806024a6c104 WatchSource:0}: Error finding container 55f2d8464f3c5f3f98bb32ccc4f06ec8e93c1a83cd4ffdcc84ae806024a6c104: Status 404 returned error can't find the container with id 55f2d8464f3c5f3f98bb32ccc4f06ec8e93c1a83cd4ffdcc84ae806024a6c104
Jul 09 09:34:05.207703 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:05.207685    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79826bec_2b69_42e5_87f1_94c078b40ac4.slice/crio-1637a481318048d9b0fdfe2102d82fc937c35489eb32aa634453932e1fb36924 WatchSource:0}: Error finding container 1637a481318048d9b0fdfe2102d82fc937c35489eb32aa634453932e1fb36924: Status 404 returned error can't find the container with id 1637a481318048d9b0fdfe2102d82fc937c35489eb32aa634453932e1fb36924
Jul 09 09:34:05.208788 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:05.208766    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35942e40_0ffc_472e_bc29_be3077dc61d5.slice/crio-126943074fdc11f59c89dfb076971670b0fb1b9d89da5a480c48961aced987d1 WatchSource:0}: Error finding container 126943074fdc11f59c89dfb076971670b0fb1b9d89da5a480c48961aced987d1: Status 404 returned error can't find the container with id 126943074fdc11f59c89dfb076971670b0fb1b9d89da5a480c48961aced987d1
Jul 09 09:34:05.209745 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:05.209638    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeae6a875_8c48_46ae_bac2_cbf1b0cfc34f.slice/crio-1a5b1ed5def76c15caba482ba4731cf54a7b82c87d632cc42e250a856d080b7b WatchSource:0}: Error finding container 1a5b1ed5def76c15caba482ba4731cf54a7b82c87d632cc42e250a856d080b7b: Status 404 returned error can't find the container with id 1a5b1ed5def76c15caba482ba4731cf54a7b82c87d632cc42e250a856d080b7b
Jul 09 09:34:05.210195 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:05.210091    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83ccd32a_ea2e_4ad9_95c7_f79c33043432.slice/crio-b6af12a959f37aedf0fbe629d5f256ab8af833b11ba7284d873b0964c1b3f6b4 WatchSource:0}: Error finding container b6af12a959f37aedf0fbe629d5f256ab8af833b11ba7284d873b0964c1b3f6b4: Status 404 returned error can't find the container with id b6af12a959f37aedf0fbe629d5f256ab8af833b11ba7284d873b0964c1b3f6b4
Jul 09 09:34:05.211141 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:05.211110    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e60f669_e68e_4467_ad31_c37b3c9883b9.slice/crio-6f22b0e1ceff2cc36914eddd56647a20f7c58526517986741e8f0c3f508f077e WatchSource:0}: Error finding container 6f22b0e1ceff2cc36914eddd56647a20f7c58526517986741e8f0c3f508f077e: Status 404 returned error can't find the container with id 6f22b0e1ceff2cc36914eddd56647a20f7c58526517986741e8f0c3f508f077e
Jul 09 09:34:05.213059 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:05.212360    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4defe16_3199_4cea_8b4e_88375572edbf.slice/crio-812ca963203e603cec18d44060dc2c712c10dfe6d0a064c2738383d4724ebeba WatchSource:0}: Error finding container 812ca963203e603cec18d44060dc2c712c10dfe6d0a064c2738383d4724ebeba: Status 404 returned error can't find the container with id 812ca963203e603cec18d44060dc2c712c10dfe6d0a064c2738383d4724ebeba
Jul 09 09:34:05.214136 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:05.214036    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e407356_17fe_4a9f_81da_417042050209.slice/crio-2758e48fb7e0e57923270e147b305121fb27b7bc2b0d8690d7eb89278c14c28a WatchSource:0}: Error finding container 2758e48fb7e0e57923270e147b305121fb27b7bc2b0d8690d7eb89278c14c28a: Status 404 returned error can't find the container with id 2758e48fb7e0e57923270e147b305121fb27b7bc2b0d8690d7eb89278c14c28a
Jul 09 09:34:05.281723 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:05.281674    2549 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 09:29:04 +0000 UTC" deadline="2027-12-20 07:22:40.249655292 +0000 UTC"
Jul 09 09:34:05.281723 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:05.281719    2549 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12693h48m34.967940869s"
Jul 09 09:34:05.380339 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:05.380308    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:05.380495 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:05.380426    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljjkv" podUID="2ab646db-3779-4dbf-a300-a7b166d929ab"
Jul 09 09:34:05.380495 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:05.380479    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:05.380608 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:05.380590    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccqx8" podUID="282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee"
Jul 09 09:34:05.386584 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:05.386558    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nz6rq" event={"ID":"2e407356-17fe-4a9f-81da-417042050209","Type":"ContainerStarted","Data":"2758e48fb7e0e57923270e147b305121fb27b7bc2b0d8690d7eb89278c14c28a"}
Jul 09 09:34:05.388199 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:05.388174    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c4jnr" event={"ID":"f4defe16-3199-4cea-8b4e-88375572edbf","Type":"ContainerStarted","Data":"812ca963203e603cec18d44060dc2c712c10dfe6d0a064c2738383d4724ebeba"}
Jul 09 09:34:05.391277 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:05.391231    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8xp94" event={"ID":"83ccd32a-ea2e-4ad9-95c7-f79c33043432","Type":"ContainerStarted","Data":"b6af12a959f37aedf0fbe629d5f256ab8af833b11ba7284d873b0964c1b3f6b4"}
Jul 09 09:34:05.392487 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:05.392465    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7" event={"ID":"35942e40-0ffc-472e-bc29-be3077dc61d5","Type":"ContainerStarted","Data":"126943074fdc11f59c89dfb076971670b0fb1b9d89da5a480c48961aced987d1"}
Jul 09 09:34:05.393686 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:05.393664    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-g7dzq" event={"ID":"edcfc3e5-6a61-430a-af17-c26e68349a3e","Type":"ContainerStarted","Data":"b23fbedba2ceafc74947997fe1dfd6299f9cbcb0f5bb616fd328621351498308"}
Jul 09 09:34:05.395683 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:05.395662    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-199.ec2.internal" event={"ID":"d0bea77c1d44c6a069aa75a3ce77a6d6","Type":"ContainerStarted","Data":"56de87f1d1825c42ea7e54e075d1ef9f796fbca4ec932b8a8f922a7cd348dd4e"}
Jul 09 09:34:05.396816 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:05.396795    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xqkc6" event={"ID":"7e60f669-e68e-4467-ad31-c37b3c9883b9","Type":"ContainerStarted","Data":"6f22b0e1ceff2cc36914eddd56647a20f7c58526517986741e8f0c3f508f077e"}
Jul 09 09:34:05.397985 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:05.397961    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9h8vf" event={"ID":"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f","Type":"ContainerStarted","Data":"1a5b1ed5def76c15caba482ba4731cf54a7b82c87d632cc42e250a856d080b7b"}
Jul 09 09:34:05.398971 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:05.398950    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mfcn7" event={"ID":"79826bec-2b69-42e5-87f1-94c078b40ac4","Type":"ContainerStarted","Data":"1637a481318048d9b0fdfe2102d82fc937c35489eb32aa634453932e1fb36924"}
Jul 09 09:34:05.400186 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:05.400168    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ffq5d" event={"ID":"885b74bd-5140-4025-9485-046e3eacfc6c","Type":"ContainerStarted","Data":"55f2d8464f3c5f3f98bb32ccc4f06ec8e93c1a83cd4ffdcc84ae806024a6c104"}
Jul 09 09:34:05.411454 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:05.411415    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-199.ec2.internal" podStartSLOduration=2.411403708 podStartE2EDuration="2.411403708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:34:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:34:05.411083482 +0000 UTC m=+3.582242040" watchObservedRunningTime="2026-07-09 09:34:05.411403708 +0000 UTC m=+3.582562264"
Jul 09 09:34:05.863426 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:05.862701    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72ab801d-5d43-4fbc-a26a-60fab96be57b-metrics-certs\") pod \"network-metrics-daemon-rqzsx\" (UID: \"72ab801d-5d43-4fbc-a26a-60fab96be57b\") " pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:05.863426 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:05.862923    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 09:34:05.863426 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:05.862990    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72ab801d-5d43-4fbc-a26a-60fab96be57b-metrics-certs podName:72ab801d-5d43-4fbc-a26a-60fab96be57b nodeName:}" failed. No retries permitted until 2026-07-09 09:34:07.862971032 +0000 UTC m=+6.034129584 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72ab801d-5d43-4fbc-a26a-60fab96be57b-metrics-certs") pod "network-metrics-daemon-rqzsx" (UID: "72ab801d-5d43-4fbc-a26a-60fab96be57b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 09:34:05.964005 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:05.963963    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-original-pull-secret\") pod \"global-pull-secret-syncer-ccqx8\" (UID: \"282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee\") " pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:05.964193 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:05.964043    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6299\" (UniqueName: \"kubernetes.io/projected/2ab646db-3779-4dbf-a300-a7b166d929ab-kube-api-access-t6299\") pod \"network-check-target-ljjkv\" (UID: \"2ab646db-3779-4dbf-a300-a7b166d929ab\") " pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:05.964193 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:05.964180    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 09:34:05.964306 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:05.964203    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 09:34:05.964306 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:05.964216    2549 projected.go:196] Error preparing data for projected volume kube-api-access-t6299 for pod openshift-network-diagnostics/network-check-target-ljjkv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 09:34:05.964306 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:05.964276    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ab646db-3779-4dbf-a300-a7b166d929ab-kube-api-access-t6299 podName:2ab646db-3779-4dbf-a300-a7b166d929ab nodeName:}" failed. No retries permitted until 2026-07-09 09:34:07.964256833 +0000 UTC m=+6.135415381 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-t6299" (UniqueName: "kubernetes.io/projected/2ab646db-3779-4dbf-a300-a7b166d929ab-kube-api-access-t6299") pod "network-check-target-ljjkv" (UID: "2ab646db-3779-4dbf-a300-a7b166d929ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 09:34:05.964725 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:05.964680    2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 09:34:05.964829 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:05.964740    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-original-pull-secret podName:282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee nodeName:}" failed. No retries permitted until 2026-07-09 09:34:07.964716516 +0000 UTC m=+6.135875069 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-original-pull-secret") pod "global-pull-secret-syncer-ccqx8" (UID: "282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee") : object "kube-system"/"original-pull-secret" not registered
Jul 09 09:34:06.208371 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:06.208082    2549 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 09:34:06.282830 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:06.282700    2549 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 09:29:04 +0000 UTC" deadline="2028-02-09 15:24:58.366009042 +0000 UTC"
Jul 09 09:34:06.282830 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:06.282760    2549 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13925h50m52.083253102s"
Jul 09 09:34:06.382985 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:06.382954    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:06.383151 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:06.383098    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqzsx" podUID="72ab801d-5d43-4fbc-a26a-60fab96be57b"
Jul 09 09:34:06.411846 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:06.411811    2549 generic.go:356] "Generic (PLEG): container finished" podID="f7edc44c9cdff1100ab1bbace1365704" containerID="1df7f8ed2bd3cfd6b7167b837e855138f12e5139c50959ebec63360963d3c9b3" exitCode=0
Jul 09 09:34:06.413101 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:06.412801    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-199.ec2.internal" event={"ID":"f7edc44c9cdff1100ab1bbace1365704","Type":"ContainerDied","Data":"1df7f8ed2bd3cfd6b7167b837e855138f12e5139c50959ebec63360963d3c9b3"}
Jul 09 09:34:07.379860 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:07.379830    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:07.380319 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:07.379974    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljjkv" podUID="2ab646db-3779-4dbf-a300-a7b166d929ab"
Jul 09 09:34:07.380376 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:07.380365    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:07.380477 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:07.380455    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccqx8" podUID="282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee"
Jul 09 09:34:07.424477 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:07.423691    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-199.ec2.internal" event={"ID":"f7edc44c9cdff1100ab1bbace1365704","Type":"ContainerStarted","Data":"7e9d693b06bdfd71c6092a67643bf133cea65215cb4ed2a771a324e474e9f6ad"}
Jul 09 09:34:07.883582 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:07.882960    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72ab801d-5d43-4fbc-a26a-60fab96be57b-metrics-certs\") pod \"network-metrics-daemon-rqzsx\" (UID: \"72ab801d-5d43-4fbc-a26a-60fab96be57b\") " pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:07.883582 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:07.883180    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 09:34:07.883582 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:07.883249    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72ab801d-5d43-4fbc-a26a-60fab96be57b-metrics-certs podName:72ab801d-5d43-4fbc-a26a-60fab96be57b nodeName:}" failed. No retries permitted until 2026-07-09 09:34:11.883229351 +0000 UTC m=+10.054387906 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72ab801d-5d43-4fbc-a26a-60fab96be57b-metrics-certs") pod "network-metrics-daemon-rqzsx" (UID: "72ab801d-5d43-4fbc-a26a-60fab96be57b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 09:34:07.984195 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:07.984142    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6299\" (UniqueName: \"kubernetes.io/projected/2ab646db-3779-4dbf-a300-a7b166d929ab-kube-api-access-t6299\") pod \"network-check-target-ljjkv\" (UID: \"2ab646db-3779-4dbf-a300-a7b166d929ab\") " pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:07.984376 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:07.984280    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-original-pull-secret\") pod \"global-pull-secret-syncer-ccqx8\" (UID: \"282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee\") " pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:07.984471 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:07.984452    2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 09:34:07.984541 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:07.984523    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-original-pull-secret podName:282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee nodeName:}" failed. No retries permitted until 2026-07-09 09:34:11.984503907 +0000 UTC m=+10.155662459 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-original-pull-secret") pod "global-pull-secret-syncer-ccqx8" (UID: "282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee") : object "kube-system"/"original-pull-secret" not registered
Jul 09 09:34:07.984940 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:07.984921    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 09:34:07.985034 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:07.984945    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 09:34:07.985034 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:07.984963    2549 projected.go:196] Error preparing data for projected volume kube-api-access-t6299 for pod openshift-network-diagnostics/network-check-target-ljjkv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 09:34:07.985034 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:07.985030    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ab646db-3779-4dbf-a300-a7b166d929ab-kube-api-access-t6299 podName:2ab646db-3779-4dbf-a300-a7b166d929ab nodeName:}" failed. No retries permitted until 2026-07-09 09:34:11.985012617 +0000 UTC m=+10.156171210 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-t6299" (UniqueName: "kubernetes.io/projected/2ab646db-3779-4dbf-a300-a7b166d929ab-kube-api-access-t6299") pod "network-check-target-ljjkv" (UID: "2ab646db-3779-4dbf-a300-a7b166d929ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 09:34:08.380339 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:08.380257    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:08.380776 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:08.380413    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqzsx" podUID="72ab801d-5d43-4fbc-a26a-60fab96be57b"
Jul 09 09:34:09.379814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:09.379764    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:09.379814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:09.379806    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:09.380063 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:09.379909    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljjkv" podUID="2ab646db-3779-4dbf-a300-a7b166d929ab"
Jul 09 09:34:09.380063 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:09.380026    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccqx8" podUID="282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee"
Jul 09 09:34:10.380101 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:10.380016    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:10.380512 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:10.380159    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqzsx" podUID="72ab801d-5d43-4fbc-a26a-60fab96be57b"
Jul 09 09:34:11.380024 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:11.379984    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:11.380187 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:11.380112    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccqx8" podUID="282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee"
Jul 09 09:34:11.380187 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:11.379979    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:11.380609 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:11.380223    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljjkv" podUID="2ab646db-3779-4dbf-a300-a7b166d929ab"
Jul 09 09:34:11.917184 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:11.917144    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72ab801d-5d43-4fbc-a26a-60fab96be57b-metrics-certs\") pod \"network-metrics-daemon-rqzsx\" (UID: \"72ab801d-5d43-4fbc-a26a-60fab96be57b\") " pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:11.917365 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:11.917328    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 09:34:11.917426 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:11.917400    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72ab801d-5d43-4fbc-a26a-60fab96be57b-metrics-certs podName:72ab801d-5d43-4fbc-a26a-60fab96be57b nodeName:}" failed. No retries permitted until 2026-07-09 09:34:19.917377948 +0000 UTC m=+18.088536488 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72ab801d-5d43-4fbc-a26a-60fab96be57b-metrics-certs") pod "network-metrics-daemon-rqzsx" (UID: "72ab801d-5d43-4fbc-a26a-60fab96be57b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 09:34:12.017956 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:12.017918    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-original-pull-secret\") pod \"global-pull-secret-syncer-ccqx8\" (UID: \"282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee\") " pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:12.018121 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:12.017982    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6299\" (UniqueName: \"kubernetes.io/projected/2ab646db-3779-4dbf-a300-a7b166d929ab-kube-api-access-t6299\") pod \"network-check-target-ljjkv\" (UID: \"2ab646db-3779-4dbf-a300-a7b166d929ab\") " pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:12.018121 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:12.018068    2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 09:34:12.018121 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:12.018094    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 09:34:12.018121 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:12.018106    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 09:34:12.018121 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:12.018115    2549 projected.go:196] Error preparing data for projected volume kube-api-access-t6299 for pod openshift-network-diagnostics/network-check-target-ljjkv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 09:34:12.018284 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:12.018135    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-original-pull-secret podName:282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee nodeName:}" failed. No retries permitted until 2026-07-09 09:34:20.018119608 +0000 UTC m=+18.189278166 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-original-pull-secret") pod "global-pull-secret-syncer-ccqx8" (UID: "282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee") : object "kube-system"/"original-pull-secret" not registered
Jul 09 09:34:12.018284 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:12.018153    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ab646db-3779-4dbf-a300-a7b166d929ab-kube-api-access-t6299 podName:2ab646db-3779-4dbf-a300-a7b166d929ab nodeName:}" failed. No retries permitted until 2026-07-09 09:34:20.018141737 +0000 UTC m=+18.189300297 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-t6299" (UniqueName: "kubernetes.io/projected/2ab646db-3779-4dbf-a300-a7b166d929ab-kube-api-access-t6299") pod "network-check-target-ljjkv" (UID: "2ab646db-3779-4dbf-a300-a7b166d929ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 09:34:12.381105 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:12.381023    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:12.381537 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:12.381157    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqzsx" podUID="72ab801d-5d43-4fbc-a26a-60fab96be57b"
Jul 09 09:34:13.379916 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:13.379722    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:13.379916 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:13.379884    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccqx8" podUID="282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee"
Jul 09 09:34:13.380131 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:13.379722    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:13.380131 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:13.380001    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljjkv" podUID="2ab646db-3779-4dbf-a300-a7b166d929ab"
Jul 09 09:34:14.379981 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:14.379950    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:14.380410 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:14.380089    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqzsx" podUID="72ab801d-5d43-4fbc-a26a-60fab96be57b"
Jul 09 09:34:15.380142 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:15.380109    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:15.380142 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:15.380148    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:15.380671 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:15.380233    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccqx8" podUID="282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee"
Jul 09 09:34:15.380671 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:15.380347    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljjkv" podUID="2ab646db-3779-4dbf-a300-a7b166d929ab"
Jul 09 09:34:16.380413 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:16.380355    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:16.380823 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:16.380499    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqzsx" podUID="72ab801d-5d43-4fbc-a26a-60fab96be57b"
Jul 09 09:34:17.380055 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:17.380021    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:17.380267 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:17.380123    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljjkv" podUID="2ab646db-3779-4dbf-a300-a7b166d929ab"
Jul 09 09:34:17.380327 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:17.380307    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:17.380434 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:17.380402    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccqx8" podUID="282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee"
Jul 09 09:34:18.380431 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:18.380390    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:18.380888 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:18.380538    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqzsx" podUID="72ab801d-5d43-4fbc-a26a-60fab96be57b"
Jul 09 09:34:19.380030 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:19.379993    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:19.380230 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:19.380104    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljjkv" podUID="2ab646db-3779-4dbf-a300-a7b166d929ab"
Jul 09 09:34:19.380230 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:19.380163    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:19.380324 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:19.380278    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccqx8" podUID="282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee"
Jul 09 09:34:19.987334 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:19.987304    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72ab801d-5d43-4fbc-a26a-60fab96be57b-metrics-certs\") pod \"network-metrics-daemon-rqzsx\" (UID: \"72ab801d-5d43-4fbc-a26a-60fab96be57b\") " pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:19.987746 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:19.987441    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 09:34:19.987746 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:19.987498    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72ab801d-5d43-4fbc-a26a-60fab96be57b-metrics-certs podName:72ab801d-5d43-4fbc-a26a-60fab96be57b nodeName:}" failed. No retries permitted until 2026-07-09 09:34:35.987482693 +0000 UTC m=+34.158641240 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72ab801d-5d43-4fbc-a26a-60fab96be57b-metrics-certs") pod "network-metrics-daemon-rqzsx" (UID: "72ab801d-5d43-4fbc-a26a-60fab96be57b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 09:34:20.087625 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:20.087597    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6299\" (UniqueName: \"kubernetes.io/projected/2ab646db-3779-4dbf-a300-a7b166d929ab-kube-api-access-t6299\") pod \"network-check-target-ljjkv\" (UID: \"2ab646db-3779-4dbf-a300-a7b166d929ab\") " pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:20.087772 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:20.087644    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-original-pull-secret\") pod \"global-pull-secret-syncer-ccqx8\" (UID: \"282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee\") " pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:20.087772 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:20.087753    2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 09:34:20.087836 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:20.087774    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 09:34:20.087836 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:20.087799    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 09:34:20.087836 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:20.087808    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-original-pull-secret podName:282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee nodeName:}" failed. No retries permitted until 2026-07-09 09:34:36.087792036 +0000 UTC m=+34.258950587 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-original-pull-secret") pod "global-pull-secret-syncer-ccqx8" (UID: "282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee") : object "kube-system"/"original-pull-secret" not registered
Jul 09 09:34:20.087836 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:20.087813    2549 projected.go:196] Error preparing data for projected volume kube-api-access-t6299 for pod openshift-network-diagnostics/network-check-target-ljjkv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 09:34:20.088012 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:20.087885    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ab646db-3779-4dbf-a300-a7b166d929ab-kube-api-access-t6299 podName:2ab646db-3779-4dbf-a300-a7b166d929ab nodeName:}" failed. No retries permitted until 2026-07-09 09:34:36.087849129 +0000 UTC m=+34.259007679 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-t6299" (UniqueName: "kubernetes.io/projected/2ab646db-3779-4dbf-a300-a7b166d929ab-kube-api-access-t6299") pod "network-check-target-ljjkv" (UID: "2ab646db-3779-4dbf-a300-a7b166d929ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 09:34:20.380400 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:20.380326    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:20.380564 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:20.380443    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqzsx" podUID="72ab801d-5d43-4fbc-a26a-60fab96be57b"
Jul 09 09:34:21.380126 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:21.380084    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:21.380535 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:21.380231    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccqx8" podUID="282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee"
Jul 09 09:34:21.380535 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:21.380268    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:21.380535 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:21.380384    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljjkv" podUID="2ab646db-3779-4dbf-a300-a7b166d929ab"
Jul 09 09:34:22.380444 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:22.380407    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:22.380910 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:22.380515    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqzsx" podUID="72ab801d-5d43-4fbc-a26a-60fab96be57b"
Jul 09 09:34:23.379477 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:23.379442    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:23.379657 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:23.379447    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:23.379657 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:23.379577    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljjkv" podUID="2ab646db-3779-4dbf-a300-a7b166d929ab"
Jul 09 09:34:23.379777 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:23.379681    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccqx8" podUID="282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee"
Jul 09 09:34:24.380060 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:24.379664    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:24.380549 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:24.380150    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqzsx" podUID="72ab801d-5d43-4fbc-a26a-60fab96be57b"
Jul 09 09:34:24.459825 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:24.459797    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7" event={"ID":"35942e40-0ffc-472e-bc29-be3077dc61d5","Type":"ContainerStarted","Data":"ab0b955a74efbc6a62c9abd7b5fefb48e84615ca0ee83b43f38502581c0fea43"}
Jul 09 09:34:24.459959 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:24.459942    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7" event={"ID":"35942e40-0ffc-472e-bc29-be3077dc61d5","Type":"ContainerStarted","Data":"bfa3ab23a9be4dee2d787472c88a276180944f37ee8fa1e7bd0c7cd36cb172b5"}
Jul 09 09:34:24.461450 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:24.461294    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-g7dzq" event={"ID":"edcfc3e5-6a61-430a-af17-c26e68349a3e","Type":"ContainerStarted","Data":"edaf329d0cfda796449db828502c28258c689c606b9e93f0cf2ee6229679cb17"}
Jul 09 09:34:24.462986 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:24.462897    2549 generic.go:356] "Generic (PLEG): container finished" podID="eae6a875-8c48-46ae-bac2-cbf1b0cfc34f" containerID="3e9972ece3ce526cccf569279902d89abb042063220a70c5f75b241c474a8c89" exitCode=0
Jul 09 09:34:24.463083 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:24.463022    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9h8vf" event={"ID":"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f","Type":"ContainerDied","Data":"3e9972ece3ce526cccf569279902d89abb042063220a70c5f75b241c474a8c89"}
Jul 09 09:34:24.466424 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:24.466339    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ffq5d" event={"ID":"885b74bd-5140-4025-9485-046e3eacfc6c","Type":"ContainerStarted","Data":"db7da70e159e394fe60dd0b8d23bd92e7d2c63c8979a95daee0934eec4b6076a"}
Jul 09 09:34:24.468150 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:24.468097    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nz6rq" event={"ID":"2e407356-17fe-4a9f-81da-417042050209","Type":"ContainerStarted","Data":"ec21c2413d439db348a45e7d10184476ff545033cefc80e92d1b0f28c4cdcf32"}
Jul 09 09:34:24.469636 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:24.469583    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c4jnr" event={"ID":"f4defe16-3199-4cea-8b4e-88375572edbf","Type":"ContainerStarted","Data":"cfcd805d40f73694d997cb57f22b297d60ffdbc1d99393092fc4780594ddb463"}
Jul 09 09:34:24.470849 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:24.470802    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8xp94" event={"ID":"83ccd32a-ea2e-4ad9-95c7-f79c33043432","Type":"ContainerStarted","Data":"5c52ede3f0a7130fca25498e62030ed70c8a851ea9b99b05d3fbe671140341cc"}
Jul 09 09:34:24.486488 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:24.486434    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-g7dzq" podStartSLOduration=3.57179139 podStartE2EDuration="22.486416134s" totalImagesPullingTime="18.914624744s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:34:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:34:24.483329743 +0000 UTC m=+22.654488300" watchObservedRunningTime="2026-07-09 09:34:24.486416134 +0000 UTC m=+22.657574691"
Jul 09 09:34:24.486662 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:24.486633    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-199.ec2.internal" podStartSLOduration=21.486621948 podStartE2EDuration="21.486621948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:34:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:34:07.441130365 +0000 UTC m=+5.612288925" watchObservedRunningTime="2026-07-09 09:34:24.486621948 +0000 UTC m=+22.657780508"
Jul 09 09:34:24.500141 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:24.500087    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nz6rq" podStartSLOduration=3.599288972 podStartE2EDuration="22.500067805s" totalImagesPullingTime="18.900778833s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:34:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:34:24.499650568 +0000 UTC m=+22.670809125" watchObservedRunningTime="2026-07-09 09:34:24.500067805 +0000 UTC m=+22.671226362"
Jul 09 09:34:24.554003 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:24.553950    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-c4jnr" podStartSLOduration=7.609369814 podStartE2EDuration="22.553929567s" totalImagesPullingTime="14.944559753s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:34:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:34:24.536221095 +0000 UTC m=+22.707379647" watchObservedRunningTime="2026-07-09 09:34:24.553929567 +0000 UTC m=+22.725088124"
Jul 09 09:34:24.554272 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:24.554239    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mfcn7" podStartSLOduration=3.383962603 podStartE2EDuration="22.554227309s" totalImagesPullingTime="19.170264706s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:34:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:34:24.553457735 +0000 UTC m=+22.724616291" watchObservedRunningTime="2026-07-09 09:34:24.554227309 +0000 UTC m=+22.725385866"
Jul 09 09:34:24.570704 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:24.570666    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8xp94" podStartSLOduration=3.667827658 podStartE2EDuration="22.570651137s" totalImagesPullingTime="18.902823479s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:34:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:34:24.570415449 +0000 UTC m=+22.741574007" watchObservedRunningTime="2026-07-09 09:34:24.570651137 +0000 UTC m=+22.741809692"
Jul 09 09:34:25.380164 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:25.380126    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:25.381010 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:25.380352    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:25.381010 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:25.380338    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljjkv" podUID="2ab646db-3779-4dbf-a300-a7b166d929ab"
Jul 09 09:34:25.381010 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:25.380443    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccqx8" podUID="282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee"
Jul 09 09:34:25.475355 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:25.475317    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7" event={"ID":"35942e40-0ffc-472e-bc29-be3077dc61d5","Type":"ContainerStarted","Data":"2621432f1e12159367fb16f15f65975863011b43c0c9b01d24deac91531147f0"}
Jul 09 09:34:25.475506 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:25.475390    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7" event={"ID":"35942e40-0ffc-472e-bc29-be3077dc61d5","Type":"ContainerStarted","Data":"e95bc98290fe11ba0edec7feef2078f765b945642bcc439e694ba675f06701ea"}
Jul 09 09:34:25.475506 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:25.475422    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7" event={"ID":"35942e40-0ffc-472e-bc29-be3077dc61d5","Type":"ContainerStarted","Data":"8806892240ebbaa7765157eca1fa36db88d35fe995e6f0c4f0b15dba0268cf1a"}
Jul 09 09:34:25.475506 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:25.475451    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7" event={"ID":"35942e40-0ffc-472e-bc29-be3077dc61d5","Type":"ContainerStarted","Data":"1b557515883365353beb614596c156b2b353f4876527864571afd465dc44e032"}
Jul 09 09:34:25.476640 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:25.476613    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xqkc6" event={"ID":"7e60f669-e68e-4467-ad31-c37b3c9883b9","Type":"ContainerStarted","Data":"3fb03cb8a1c202b9a0d623cad2315cf33f783170c59e0bca0afbe4694f230e34"}
Jul 09 09:34:25.478159 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:25.478136    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mfcn7" event={"ID":"79826bec-2b69-42e5-87f1-94c078b40ac4","Type":"ContainerStarted","Data":"4f3c4f2687925ab9f3b96fe8b92908e55931ca2a57035cfe99c78721feb81b25"}
Jul 09 09:34:25.677431 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:25.677410    2549 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 09 09:34:26.304314 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:26.304199    2549 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T09:34:25.677427781Z","UUID":"a6df7356-8f90-457b-a3a3-cc87a0b71744","Handler":null,"Name":"","Endpoint":""}
Jul 09 09:34:26.306083 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:26.306025    2549 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 09 09:34:26.306216 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:26.306123    2549 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 09 09:34:26.380278 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:26.380247    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:26.380672 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:26.380373    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqzsx" podUID="72ab801d-5d43-4fbc-a26a-60fab96be57b"
Jul 09 09:34:26.482791 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:26.482710    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ffq5d" event={"ID":"885b74bd-5140-4025-9485-046e3eacfc6c","Type":"ContainerStarted","Data":"34044ef35c54a45ab199ec6bd65a20028acd802ca10e76217ea7331c039127b8"}
Jul 09 09:34:26.554184 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:26.554153    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-c4jnr"
Jul 09 09:34:26.555030 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:26.554966    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-c4jnr"
Jul 09 09:34:26.570334 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:26.570289    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xqkc6" podStartSLOduration=5.685056605 podStartE2EDuration="24.57027223s" totalImagesPullingTime="18.885215625s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:34:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:34:25.496072424 +0000 UTC m=+23.667230980" watchObservedRunningTime="2026-07-09 09:34:26.57027223 +0000 UTC m=+24.741430841"
Jul 09 09:34:27.380086 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:27.379847    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:27.380294 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:27.379892    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:27.380294 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:27.380130    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccqx8" podUID="282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee"
Jul 09 09:34:27.380294 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:27.380178    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljjkv" podUID="2ab646db-3779-4dbf-a300-a7b166d929ab"
Jul 09 09:34:27.488774 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:27.488734    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7" event={"ID":"35942e40-0ffc-472e-bc29-be3077dc61d5","Type":"ContainerStarted","Data":"ee98fd9c01fcbed902c2f2c92b1865de7f73e02e45bbb01297f25d57e8ed8fa0"}
Jul 09 09:34:27.491099 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:27.491068    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ffq5d" event={"ID":"885b74bd-5140-4025-9485-046e3eacfc6c","Type":"ContainerStarted","Data":"03441922ba9aa5c0df4e070319f37ef0e3d04eb50012a740471333612f8697ee"}
Jul 09 09:34:27.491412 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:27.491355    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-c4jnr"
Jul 09 09:34:27.491948 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:27.491925    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-c4jnr"
Jul 09 09:34:27.512221 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:27.512170    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ffq5d" podStartSLOduration=4.09454269 podStartE2EDuration="25.512157457s" totalImagesPullingTime="21.417614767s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:34:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:34:27.512011881 +0000 UTC m=+25.683170436" watchObservedRunningTime="2026-07-09 09:34:27.512157457 +0000 UTC m=+25.683316012"
Jul 09 09:34:28.380475 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:28.380438    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:28.380998 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:28.380588    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqzsx" podUID="72ab801d-5d43-4fbc-a26a-60fab96be57b"
Jul 09 09:34:29.380090 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:29.379915    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:29.380229 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:29.379949    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:29.380229 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:29.380162    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljjkv" podUID="2ab646db-3779-4dbf-a300-a7b166d929ab"
Jul 09 09:34:29.380328 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:29.380240    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccqx8" podUID="282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee"
Jul 09 09:34:29.497738 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:29.497650    2549 generic.go:356] "Generic (PLEG): container finished" podID="eae6a875-8c48-46ae-bac2-cbf1b0cfc34f" containerID="fb3c2c07a25232fdece91b89371a87ecc983c92bd3d000f8d6a17a284c3c9a12" exitCode=0
Jul 09 09:34:29.497738 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:29.497717    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9h8vf" event={"ID":"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f","Type":"ContainerDied","Data":"fb3c2c07a25232fdece91b89371a87ecc983c92bd3d000f8d6a17a284c3c9a12"}
Jul 09 09:34:29.501596 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:29.501564    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7" event={"ID":"35942e40-0ffc-472e-bc29-be3077dc61d5","Type":"ContainerStarted","Data":"f3fce3de30df14be5f91491eb927aa91e1a139350580afda42affb2153c5fa62"}
Jul 09 09:34:29.501853 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:29.501827    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:29.501961 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:29.501911    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:29.516929 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:29.516910    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:29.517055 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:29.517042    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:29.549950 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:29.549914    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7" podStartSLOduration=8.614622249 podStartE2EDuration="27.549902341s" totalImagesPullingTime="18.935280092s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:34:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:34:29.548463091 +0000 UTC m=+27.719621646" watchObservedRunningTime="2026-07-09 09:34:29.549902341 +0000 UTC m=+27.721060897"
Jul 09 09:34:30.382550 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:30.382465    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:30.382748 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:30.382616    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqzsx" podUID="72ab801d-5d43-4fbc-a26a-60fab96be57b"
Jul 09 09:34:30.503546 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:30.503520    2549 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 09 09:34:31.107585 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:31.107515    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ljjkv"]
Jul 09 09:34:31.107712 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:31.107627    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:31.107749 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:31.107712    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljjkv" podUID="2ab646db-3779-4dbf-a300-a7b166d929ab"
Jul 09 09:34:31.110686 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:31.110661    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ccqx8"]
Jul 09 09:34:31.110838 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:31.110753    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:31.110838 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:31.110830    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccqx8" podUID="282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee"
Jul 09 09:34:31.111279 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:31.111256    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rqzsx"]
Jul 09 09:34:31.111357 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:31.111342    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:31.111441 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:31.111425    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqzsx" podUID="72ab801d-5d43-4fbc-a26a-60fab96be57b"
Jul 09 09:34:31.439237 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:31.439165    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:34:31.508397 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:31.508367    2549 generic.go:356] "Generic (PLEG): container finished" podID="eae6a875-8c48-46ae-bac2-cbf1b0cfc34f" containerID="ef0d9d2968178f5046334e076c61c8dc841eaa4a27688d2868c365e2b27c32fb" exitCode=0
Jul 09 09:34:31.508843 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:31.508451    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9h8vf" event={"ID":"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f","Type":"ContainerDied","Data":"ef0d9d2968178f5046334e076c61c8dc841eaa4a27688d2868c365e2b27c32fb"}
Jul 09 09:34:32.383885 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:32.383724    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:32.384054 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:32.383976    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:32.384054 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:32.383988    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljjkv" podUID="2ab646db-3779-4dbf-a300-a7b166d929ab"
Jul 09 09:34:32.384233 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:32.384203    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqzsx" podUID="72ab801d-5d43-4fbc-a26a-60fab96be57b"
Jul 09 09:34:33.379734 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:33.379708    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:33.380169 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:33.379798    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccqx8" podUID="282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee"
Jul 09 09:34:33.515272 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:33.515197    2549 generic.go:356] "Generic (PLEG): container finished" podID="eae6a875-8c48-46ae-bac2-cbf1b0cfc34f" containerID="47767a5081c950f7a39c38c186422f29ae98e36fbf059957f536428c07bf315f" exitCode=0
Jul 09 09:34:33.515272 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:33.515239    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9h8vf" event={"ID":"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f","Type":"ContainerDied","Data":"47767a5081c950f7a39c38c186422f29ae98e36fbf059957f536428c07bf315f"}
Jul 09 09:34:34.380035 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:34.379998    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:34.380707 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:34.379998    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:34.380707 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:34.380126    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqzsx" podUID="72ab801d-5d43-4fbc-a26a-60fab96be57b"
Jul 09 09:34:34.380707 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:34.380171    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljjkv" podUID="2ab646db-3779-4dbf-a300-a7b166d929ab"
Jul 09 09:34:35.053964 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:35.053929    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8xp94_83ccd32a-ea2e-4ad9-95c7-f79c33043432/dns-node-resolver/0.log"
Jul 09 09:34:35.380289 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:35.380212    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:35.380717 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:35.380321    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccqx8" podUID="282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee"
Jul 09 09:34:36.000629 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:36.000598    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72ab801d-5d43-4fbc-a26a-60fab96be57b-metrics-certs\") pod \"network-metrics-daemon-rqzsx\" (UID: \"72ab801d-5d43-4fbc-a26a-60fab96be57b\") " pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:36.000779 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:36.000730    2549 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 09:34:36.000840 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:36.000794    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72ab801d-5d43-4fbc-a26a-60fab96be57b-metrics-certs podName:72ab801d-5d43-4fbc-a26a-60fab96be57b nodeName:}" failed. No retries permitted until 2026-07-09 09:35:08.000777852 +0000 UTC m=+66.171936407 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72ab801d-5d43-4fbc-a26a-60fab96be57b-metrics-certs") pod "network-metrics-daemon-rqzsx" (UID: "72ab801d-5d43-4fbc-a26a-60fab96be57b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 09:34:36.025852 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:36.025825    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nz6rq_2e407356-17fe-4a9f-81da-417042050209/node-ca/0.log"
Jul 09 09:34:36.101193 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:36.101162    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6299\" (UniqueName: \"kubernetes.io/projected/2ab646db-3779-4dbf-a300-a7b166d929ab-kube-api-access-t6299\") pod \"network-check-target-ljjkv\" (UID: \"2ab646db-3779-4dbf-a300-a7b166d929ab\") " pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:36.101348 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:36.101230    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-original-pull-secret\") pod \"global-pull-secret-syncer-ccqx8\" (UID: \"282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee\") " pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:36.101348 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:36.101333    2549 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 09:34:36.101451 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:36.101355    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 09:34:36.101451 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:36.101380    2549 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 09:34:36.101451 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:36.101393    2549 projected.go:196] Error preparing data for projected volume kube-api-access-t6299 for pod openshift-network-diagnostics/network-check-target-ljjkv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 09:34:36.101451 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:36.101407    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-original-pull-secret podName:282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee nodeName:}" failed. No retries permitted until 2026-07-09 09:35:08.101386818 +0000 UTC m=+66.272545372 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-original-pull-secret") pod "global-pull-secret-syncer-ccqx8" (UID: "282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee") : object "kube-system"/"original-pull-secret" not registered
Jul 09 09:34:36.101451 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:36.101444    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ab646db-3779-4dbf-a300-a7b166d929ab-kube-api-access-t6299 podName:2ab646db-3779-4dbf-a300-a7b166d929ab nodeName:}" failed. No retries permitted until 2026-07-09 09:35:08.101426709 +0000 UTC m=+66.272585261 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-t6299" (UniqueName: "kubernetes.io/projected/2ab646db-3779-4dbf-a300-a7b166d929ab-kube-api-access-t6299") pod "network-check-target-ljjkv" (UID: "2ab646db-3779-4dbf-a300-a7b166d929ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 09:34:36.379947 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:36.379913    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:36.380107 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:36.380009    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljjkv" podUID="2ab646db-3779-4dbf-a300-a7b166d929ab"
Jul 09 09:34:36.380107 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:36.380063    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:36.380226 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:36.380158    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqzsx" podUID="72ab801d-5d43-4fbc-a26a-60fab96be57b"
Jul 09 09:34:37.380198 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:37.380159    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:37.380730 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:37.380294    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccqx8" podUID="282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee"
Jul 09 09:34:38.380497 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:38.380463    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:38.381030 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:38.380584    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljjkv" podUID="2ab646db-3779-4dbf-a300-a7b166d929ab"
Jul 09 09:34:38.381030 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:38.380652    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:38.381030 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:38.380790    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqzsx" podUID="72ab801d-5d43-4fbc-a26a-60fab96be57b"
Jul 09 09:34:39.380287 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:39.380262    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:39.380423 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:39.380351    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccqx8" podUID="282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee"
Jul 09 09:34:39.531769 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:39.531738    2549 generic.go:356] "Generic (PLEG): container finished" podID="eae6a875-8c48-46ae-bac2-cbf1b0cfc34f" containerID="115a37139e5a90e6a4fb59187614dab28e1fb39084745f8df7aa698d2883d3ca" exitCode=0
Jul 09 09:34:39.532108 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:39.531798    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9h8vf" event={"ID":"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f","Type":"ContainerDied","Data":"115a37139e5a90e6a4fb59187614dab28e1fb39084745f8df7aa698d2883d3ca"}
Jul 09 09:34:40.380453 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:40.380280    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:40.380634 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:40.380336    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:40.380634 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:40.380518    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqzsx" podUID="72ab801d-5d43-4fbc-a26a-60fab96be57b"
Jul 09 09:34:40.380634 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:40.380599    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljjkv" podUID="2ab646db-3779-4dbf-a300-a7b166d929ab"
Jul 09 09:34:40.536461 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:40.536423    2549 generic.go:356] "Generic (PLEG): container finished" podID="eae6a875-8c48-46ae-bac2-cbf1b0cfc34f" containerID="3e2f2fa11d7b7088cd2f258d711f802de722cd7177efa1378fbedf74148bcbce" exitCode=0
Jul 09 09:34:40.536911 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:40.536502    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9h8vf" event={"ID":"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f","Type":"ContainerDied","Data":"3e2f2fa11d7b7088cd2f258d711f802de722cd7177efa1378fbedf74148bcbce"}
Jul 09 09:34:41.380282 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:41.380255    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:41.380596 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:41.380360    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccqx8" podUID="282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee"
Jul 09 09:34:41.541881 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:41.541829    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9h8vf" event={"ID":"eae6a875-8c48-46ae-bac2-cbf1b0cfc34f","Type":"ContainerStarted","Data":"298ae34993763aad6899a03f6d3b6bfd172bff8bb3e83ff2ec1485d336552118"}
Jul 09 09:34:41.571413 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:41.571358    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9h8vf" podStartSLOduration=8.803807192 podStartE2EDuration="39.571342918s" totalImagesPullingTime="30.767535726s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:34:02 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:34:41.56953726 +0000 UTC m=+39.740695818" watchObservedRunningTime="2026-07-09 09:34:41.571342918 +0000 UTC m=+39.742501473"
Jul 09 09:34:42.381082 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:42.381050    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:42.381240 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:42.381121    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:42.381240 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:42.381129    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljjkv" podUID="2ab646db-3779-4dbf-a300-a7b166d929ab"
Jul 09 09:34:42.381240 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:42.381201    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqzsx" podUID="72ab801d-5d43-4fbc-a26a-60fab96be57b"
Jul 09 09:34:43.379539 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:43.379513    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:43.379917 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:43.379638    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccqx8" podUID="282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee"
Jul 09 09:34:44.380460 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:44.380427    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:44.380846 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:44.380432    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:44.380846 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:44.380560    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljjkv" podUID="2ab646db-3779-4dbf-a300-a7b166d929ab"
Jul 09 09:34:44.380846 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:44.380611    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqzsx" podUID="72ab801d-5d43-4fbc-a26a-60fab96be57b"
Jul 09 09:34:45.380442 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:45.380410    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:45.380610 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:45.380510    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccqx8" podUID="282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee"
Jul 09 09:34:46.379926 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:46.379898    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:46.380082 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:46.379999    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljjkv" podUID="2ab646db-3779-4dbf-a300-a7b166d929ab"
Jul 09 09:34:46.380082 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:46.380063    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:46.380201 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:46.380179    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqzsx" podUID="72ab801d-5d43-4fbc-a26a-60fab96be57b"
Jul 09 09:34:47.380362 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:47.380336    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:47.380712 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:47.380422    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccqx8" podUID="282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee"
Jul 09 09:34:48.380384 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:48.380345    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:48.380776 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:48.380461    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljjkv" podUID="2ab646db-3779-4dbf-a300-a7b166d929ab"
Jul 09 09:34:48.380776 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:48.380513    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:48.380776 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:48.380613    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqzsx" podUID="72ab801d-5d43-4fbc-a26a-60fab96be57b"
Jul 09 09:34:49.380375 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:49.380337    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:49.380541 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:49.380448    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccqx8" podUID="282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee"
Jul 09 09:34:50.379683 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:50.379648    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:50.379683 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:50.379655    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:50.379917 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:50.379820    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljjkv" podUID="2ab646db-3779-4dbf-a300-a7b166d929ab"
Jul 09 09:34:50.379917 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:50.379762    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqzsx" podUID="72ab801d-5d43-4fbc-a26a-60fab96be57b"
Jul 09 09:34:51.380317 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:51.380288    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:51.380668 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:51.380383    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccqx8" podUID="282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee"
Jul 09 09:34:52.380773 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:52.380744    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:52.381166 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:52.380827    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqzsx" podUID="72ab801d-5d43-4fbc-a26a-60fab96be57b"
Jul 09 09:34:52.381166 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:52.380906    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:52.381166 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:52.380982    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljjkv" podUID="2ab646db-3779-4dbf-a300-a7b166d929ab"
Jul 09 09:34:53.380330 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:53.380301    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:53.380488 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:53.380393    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccqx8" podUID="282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee"
Jul 09 09:34:54.379904 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:54.379852    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:54.380347 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:54.379855    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:54.380347 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:54.379993    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rqzsx" podUID="72ab801d-5d43-4fbc-a26a-60fab96be57b"
Jul 09 09:34:54.380347 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:34:54.380051    2549 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljjkv" podUID="2ab646db-3779-4dbf-a300-a7b166d929ab"
Jul 09 09:34:55.075954 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.075881    2549 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-199.ec2.internal" event="NodeReady"
Jul 09 09:34:55.076093 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.075986    2549 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 09 09:34:55.153183 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.153154    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fvmrl"]
Jul 09 09:34:55.175249 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.175221    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-67wzn"]
Jul 09 09:34:55.175373 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.175334    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fvmrl"
Jul 09 09:34:55.178567 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.178551    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 09 09:34:55.178690 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.178589    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 09 09:34:55.178810 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.178796    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-999kb\""
Jul 09 09:34:55.179047 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.179032    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 09 09:34:55.179687 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.179673    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 09 09:34:55.195165 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.195147    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fvmrl"]
Jul 09 09:34:55.195246 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.195179    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-67wzn"]
Jul 09 09:34:55.195246 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.195243    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-67wzn"
Jul 09 09:34:55.198276 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.198257    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-87rgw\""
Jul 09 09:34:55.199401 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.199378    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 09 09:34:55.199503 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.199432    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 09 09:34:55.199570 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.199390    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 09 09:34:55.240907 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.240886    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-scljx"]
Jul 09 09:34:55.253572 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.253548    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a04fe9a6-e137-44cc-b31b-8dddd4e3a1f3-cert\") pod \"ingress-canary-67wzn\" (UID: \"a04fe9a6-e137-44cc-b31b-8dddd4e3a1f3\") " pod="openshift-ingress-canary/ingress-canary-67wzn"
Jul 09 09:34:55.253659 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.253587    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3-data-volume\") pod \"insights-runtime-extractor-fvmrl\" (UID: \"fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3\") " pod="openshift-insights/insights-runtime-extractor-fvmrl"
Jul 09 09:34:55.253659 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.253610    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3-crio-socket\") pod \"insights-runtime-extractor-fvmrl\" (UID: \"fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3\") " pod="openshift-insights/insights-runtime-extractor-fvmrl"
Jul 09 09:34:55.253659 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.253628    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fvmrl\" (UID: \"fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3\") " pod="openshift-insights/insights-runtime-extractor-fvmrl"
Jul 09 09:34:55.253817 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.253708    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fvmrl\" (UID: \"fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3\") " pod="openshift-insights/insights-runtime-extractor-fvmrl"
Jul 09 09:34:55.253817 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.253743    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdh9w\" (UniqueName: \"kubernetes.io/projected/fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3-kube-api-access-sdh9w\") pod \"insights-runtime-extractor-fvmrl\" (UID: \"fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3\") " pod="openshift-insights/insights-runtime-extractor-fvmrl"
Jul 09 09:34:55.253817 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.253770    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xc76v\" (UniqueName: \"kubernetes.io/projected/a04fe9a6-e137-44cc-b31b-8dddd4e3a1f3-kube-api-access-xc76v\") pod \"ingress-canary-67wzn\" (UID: \"a04fe9a6-e137-44cc-b31b-8dddd4e3a1f3\") " pod="openshift-ingress-canary/ingress-canary-67wzn"
Jul 09 09:34:55.255195 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.255182    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-scljx"
Jul 09 09:34:55.258474 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.258450    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-scljx"]
Jul 09 09:34:55.258588 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.258570    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 09 09:34:55.258724 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.258667    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 09 09:34:55.258821 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.258807    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mgngk\""
Jul 09 09:34:55.354787 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.354740    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3-data-volume\") pod \"insights-runtime-extractor-fvmrl\" (UID: \"fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3\") " pod="openshift-insights/insights-runtime-extractor-fvmrl"
Jul 09 09:34:55.354787 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.354771    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8mvj\" (UniqueName: \"kubernetes.io/projected/685afdc3-9a1c-4ec4-85f2-1cefd4b7c2e6-kube-api-access-w8mvj\") pod \"dns-default-scljx\" (UID: \"685afdc3-9a1c-4ec4-85f2-1cefd4b7c2e6\") " pod="openshift-dns/dns-default-scljx"
Jul 09 09:34:55.354787 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.354788    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/685afdc3-9a1c-4ec4-85f2-1cefd4b7c2e6-metrics-tls\") pod \"dns-default-scljx\" (UID: \"685afdc3-9a1c-4ec4-85f2-1cefd4b7c2e6\") " pod="openshift-dns/dns-default-scljx"
Jul 09 09:34:55.354992 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.354810    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3-crio-socket\") pod \"insights-runtime-extractor-fvmrl\" (UID: \"fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3\") " pod="openshift-insights/insights-runtime-extractor-fvmrl"
Jul 09 09:34:55.354992 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.354830    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fvmrl\" (UID: \"fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3\") " pod="openshift-insights/insights-runtime-extractor-fvmrl"
Jul 09 09:34:55.354992 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.354938    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3-crio-socket\") pod \"insights-runtime-extractor-fvmrl\" (UID: \"fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3\") " pod="openshift-insights/insights-runtime-extractor-fvmrl"
Jul 09 09:34:55.354992 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.354976    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fvmrl\" (UID: \"fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3\") " pod="openshift-insights/insights-runtime-extractor-fvmrl"
Jul 09 09:34:55.355132 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.355004    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/685afdc3-9a1c-4ec4-85f2-1cefd4b7c2e6-tmp-dir\") pod \"dns-default-scljx\" (UID: \"685afdc3-9a1c-4ec4-85f2-1cefd4b7c2e6\") " pod="openshift-dns/dns-default-scljx"
Jul 09 09:34:55.355132 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.355031    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdh9w\" (UniqueName: \"kubernetes.io/projected/fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3-kube-api-access-sdh9w\") pod \"insights-runtime-extractor-fvmrl\" (UID: \"fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3\") " pod="openshift-insights/insights-runtime-extractor-fvmrl"
Jul 09 09:34:55.355132 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.355035    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3-data-volume\") pod \"insights-runtime-extractor-fvmrl\" (UID: \"fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3\") " pod="openshift-insights/insights-runtime-extractor-fvmrl"
Jul 09 09:34:55.355132 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.355055    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xc76v\" (UniqueName: \"kubernetes.io/projected/a04fe9a6-e137-44cc-b31b-8dddd4e3a1f3-kube-api-access-xc76v\") pod \"ingress-canary-67wzn\" (UID: \"a04fe9a6-e137-44cc-b31b-8dddd4e3a1f3\") " pod="openshift-ingress-canary/ingress-canary-67wzn"
Jul 09 09:34:55.355132 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.355115    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a04fe9a6-e137-44cc-b31b-8dddd4e3a1f3-cert\") pod \"ingress-canary-67wzn\" (UID: \"a04fe9a6-e137-44cc-b31b-8dddd4e3a1f3\") " pod="openshift-ingress-canary/ingress-canary-67wzn"
Jul 09 09:34:55.355336 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.355139    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/685afdc3-9a1c-4ec4-85f2-1cefd4b7c2e6-config-volume\") pod \"dns-default-scljx\" (UID: \"685afdc3-9a1c-4ec4-85f2-1cefd4b7c2e6\") " pod="openshift-dns/dns-default-scljx"
Jul 09 09:34:55.355440 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.355422    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fvmrl\" (UID: \"fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3\") " pod="openshift-insights/insights-runtime-extractor-fvmrl"
Jul 09 09:34:55.358637 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.358613    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fvmrl\" (UID: \"fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3\") " pod="openshift-insights/insights-runtime-extractor-fvmrl"
Jul 09 09:34:55.358794 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.358774    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a04fe9a6-e137-44cc-b31b-8dddd4e3a1f3-cert\") pod \"ingress-canary-67wzn\" (UID: \"a04fe9a6-e137-44cc-b31b-8dddd4e3a1f3\") " pod="openshift-ingress-canary/ingress-canary-67wzn"
Jul 09 09:34:55.365172 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.365154    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdh9w\" (UniqueName: \"kubernetes.io/projected/fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3-kube-api-access-sdh9w\") pod \"insights-runtime-extractor-fvmrl\" (UID: \"fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3\") " pod="openshift-insights/insights-runtime-extractor-fvmrl"
Jul 09 09:34:55.365538 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.365523    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xc76v\" (UniqueName: \"kubernetes.io/projected/a04fe9a6-e137-44cc-b31b-8dddd4e3a1f3-kube-api-access-xc76v\") pod \"ingress-canary-67wzn\" (UID: \"a04fe9a6-e137-44cc-b31b-8dddd4e3a1f3\") " pod="openshift-ingress-canary/ingress-canary-67wzn"
Jul 09 09:34:55.379401 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.379383    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:34:55.382362 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.382345    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 09:34:55.455838 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.455815    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8mvj\" (UniqueName: \"kubernetes.io/projected/685afdc3-9a1c-4ec4-85f2-1cefd4b7c2e6-kube-api-access-w8mvj\") pod \"dns-default-scljx\" (UID: \"685afdc3-9a1c-4ec4-85f2-1cefd4b7c2e6\") " pod="openshift-dns/dns-default-scljx"
Jul 09 09:34:55.456007 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.455849    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/685afdc3-9a1c-4ec4-85f2-1cefd4b7c2e6-metrics-tls\") pod \"dns-default-scljx\" (UID: \"685afdc3-9a1c-4ec4-85f2-1cefd4b7c2e6\") " pod="openshift-dns/dns-default-scljx"
Jul 09 09:34:55.456007 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.455925    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/685afdc3-9a1c-4ec4-85f2-1cefd4b7c2e6-tmp-dir\") pod \"dns-default-scljx\" (UID: \"685afdc3-9a1c-4ec4-85f2-1cefd4b7c2e6\") " pod="openshift-dns/dns-default-scljx"
Jul 09 09:34:55.456007 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.455996    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/685afdc3-9a1c-4ec4-85f2-1cefd4b7c2e6-config-volume\") pod \"dns-default-scljx\" (UID: \"685afdc3-9a1c-4ec4-85f2-1cefd4b7c2e6\") " pod="openshift-dns/dns-default-scljx"
Jul 09 09:34:55.456325 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.456306    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/685afdc3-9a1c-4ec4-85f2-1cefd4b7c2e6-tmp-dir\") pod \"dns-default-scljx\" (UID: \"685afdc3-9a1c-4ec4-85f2-1cefd4b7c2e6\") " pod="openshift-dns/dns-default-scljx"
Jul 09 09:34:55.456511 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.456491    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/685afdc3-9a1c-4ec4-85f2-1cefd4b7c2e6-config-volume\") pod \"dns-default-scljx\" (UID: \"685afdc3-9a1c-4ec4-85f2-1cefd4b7c2e6\") " pod="openshift-dns/dns-default-scljx"
Jul 09 09:34:55.458002 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.457984    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/685afdc3-9a1c-4ec4-85f2-1cefd4b7c2e6-metrics-tls\") pod \"dns-default-scljx\" (UID: \"685afdc3-9a1c-4ec4-85f2-1cefd4b7c2e6\") " pod="openshift-dns/dns-default-scljx"
Jul 09 09:34:55.466103 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.466083    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8mvj\" (UniqueName: \"kubernetes.io/projected/685afdc3-9a1c-4ec4-85f2-1cefd4b7c2e6-kube-api-access-w8mvj\") pod \"dns-default-scljx\" (UID: \"685afdc3-9a1c-4ec4-85f2-1cefd4b7c2e6\") " pod="openshift-dns/dns-default-scljx"
Jul 09 09:34:55.484025 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.484007    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fvmrl"
Jul 09 09:34:55.503137 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.503118    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-67wzn"
Jul 09 09:34:55.565628 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.565596    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-scljx"
Jul 09 09:34:55.702733 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.702706    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fvmrl"]
Jul 09 09:34:55.706931 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:55.706906    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbc5cbd7_ae05_4a1b_a0cc_7d5a4be5c1d3.slice/crio-02576423b4b43dfa11730c178c2399c9ce11cf3be9a67bc92c1e506a1147038f WatchSource:0}: Error finding container 02576423b4b43dfa11730c178c2399c9ce11cf3be9a67bc92c1e506a1147038f: Status 404 returned error can't find the container with id 02576423b4b43dfa11730c178c2399c9ce11cf3be9a67bc92c1e506a1147038f
Jul 09 09:34:55.716899 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.716848    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-67wzn"]
Jul 09 09:34:55.721718 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:55.721699    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda04fe9a6_e137_44cc_b31b_8dddd4e3a1f3.slice/crio-22794d9c7cc7ee0478a3eda9a215ef7c46288cd4417889f0b277f84f686e8424 WatchSource:0}: Error finding container 22794d9c7cc7ee0478a3eda9a215ef7c46288cd4417889f0b277f84f686e8424: Status 404 returned error can't find the container with id 22794d9c7cc7ee0478a3eda9a215ef7c46288cd4417889f0b277f84f686e8424
Jul 09 09:34:55.734168 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:55.734149    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-scljx"]
Jul 09 09:34:55.737261 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:34:55.737241    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod685afdc3_9a1c_4ec4_85f2_1cefd4b7c2e6.slice/crio-94292193a4d095e9d34db18b84a810b57a8ac802836aadcaf48fa93cedffc96c WatchSource:0}: Error finding container 94292193a4d095e9d34db18b84a810b57a8ac802836aadcaf48fa93cedffc96c: Status 404 returned error can't find the container with id 94292193a4d095e9d34db18b84a810b57a8ac802836aadcaf48fa93cedffc96c
Jul 09 09:34:56.380034 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:56.379707    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:34:56.380902 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:56.380464    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:34:56.383534 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:56.383430    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 09:34:56.384032 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:56.384008    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 09:34:56.384584 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:56.384517    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-j2x9h\""
Jul 09 09:34:56.384773 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:56.384751    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 09:34:56.385055 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:56.385038    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qlfc8\""
Jul 09 09:34:56.578661 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:56.578554    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-scljx" event={"ID":"685afdc3-9a1c-4ec4-85f2-1cefd4b7c2e6","Type":"ContainerStarted","Data":"94292193a4d095e9d34db18b84a810b57a8ac802836aadcaf48fa93cedffc96c"}
Jul 09 09:34:56.582181 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:56.582091    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-67wzn" event={"ID":"a04fe9a6-e137-44cc-b31b-8dddd4e3a1f3","Type":"ContainerStarted","Data":"22794d9c7cc7ee0478a3eda9a215ef7c46288cd4417889f0b277f84f686e8424"}
Jul 09 09:34:56.586364 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:34:56.585730    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fvmrl" event={"ID":"fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3","Type":"ContainerStarted","Data":"02576423b4b43dfa11730c178c2399c9ce11cf3be9a67bc92c1e506a1147038f"}
Jul 09 09:35:01.599810 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:01.599761    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-scljx" event={"ID":"685afdc3-9a1c-4ec4-85f2-1cefd4b7c2e6","Type":"ContainerStarted","Data":"bfeb3e81f3e316a88d85ffff7b4437a3ae998aad70c8b09ad9d52a4f73b87567"}
Jul 09 09:35:01.600278 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:01.599829    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-scljx" event={"ID":"685afdc3-9a1c-4ec4-85f2-1cefd4b7c2e6","Type":"ContainerStarted","Data":"d4da1c5614fb1e7c093ce102818dca09fa7e0d687a2f1904d782aec6f5c68738"}
Jul 09 09:35:01.600278 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:01.599861    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-scljx"
Jul 09 09:35:01.601240 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:01.601211    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-67wzn" event={"ID":"a04fe9a6-e137-44cc-b31b-8dddd4e3a1f3","Type":"ContainerStarted","Data":"591a923fded5e6f44dbdd967d8c97fba7f084612b1adadac19c4b0e108c94f42"}
Jul 09 09:35:01.602774 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:01.602754    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fvmrl" event={"ID":"fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3","Type":"ContainerStarted","Data":"35d7cfed6b92d7e061d29396f8140eeabaa26ed29bd8c3e0639c95305df3a054"}
Jul 09 09:35:01.602897 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:01.602795    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fvmrl" event={"ID":"fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3","Type":"ContainerStarted","Data":"06f2713ff6e56a8b17edb51ab38812153917d65b025797b91faf04119b31ca65"}
Jul 09 09:35:01.623342 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:01.623301    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-scljx" podStartSLOduration=1.860838819 podStartE2EDuration="6.623287669s" totalImagesPullingTime="4.76244885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:34:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:35:01.621996035 +0000 UTC m=+59.793154602" watchObservedRunningTime="2026-07-09 09:35:01.623287669 +0000 UTC m=+59.794446203"
Jul 09 09:35:01.647361 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:01.647319    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-67wzn" podStartSLOduration=1.851701404 podStartE2EDuration="6.647308824s" totalImagesPullingTime="4.79560742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:34:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:35:01.646108286 +0000 UTC m=+59.817266842" watchObservedRunningTime="2026-07-09 09:35:01.647308824 +0000 UTC m=+59.818467380"
Jul 09 09:35:01.830731 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:01.830694    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-566fz"]
Jul 09 09:35:01.850263 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:01.850184    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-566fz"]
Jul 09 09:35:01.850406 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:01.850295    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-566fz"
Jul 09 09:35:01.853677 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:01.853650    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-v7s7d\""
Jul 09 09:35:01.853797 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:01.853650    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 09 09:35:01.910814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:01.910784    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e4dd43e4-2a1a-4232-9441-d19b5a5f76b6-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-566fz\" (UID: \"e4dd43e4-2a1a-4232-9441-d19b5a5f76b6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-566fz"
Jul 09 09:35:02.011266 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:02.011230    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e4dd43e4-2a1a-4232-9441-d19b5a5f76b6-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-566fz\" (UID: \"e4dd43e4-2a1a-4232-9441-d19b5a5f76b6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-566fz"
Jul 09 09:35:02.013825 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:02.013802    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e4dd43e4-2a1a-4232-9441-d19b5a5f76b6-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-566fz\" (UID: \"e4dd43e4-2a1a-4232-9441-d19b5a5f76b6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-566fz"
Jul 09 09:35:02.161059 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:02.160970    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-566fz"
Jul 09 09:35:02.316433 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:02.316408    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-566fz"]
Jul 09 09:35:02.320021 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:35:02.319993    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4dd43e4_2a1a_4232_9441_d19b5a5f76b6.slice/crio-b9f2e6523c73001026bb7d9135325d8b5e2aa51d8066eaa52a3291832fd0e262 WatchSource:0}: Error finding container b9f2e6523c73001026bb7d9135325d8b5e2aa51d8066eaa52a3291832fd0e262: Status 404 returned error can't find the container with id b9f2e6523c73001026bb7d9135325d8b5e2aa51d8066eaa52a3291832fd0e262
Jul 09 09:35:02.529802 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:02.529771    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j2nr7"
Jul 09 09:35:02.606730 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:02.606684    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-566fz" event={"ID":"e4dd43e4-2a1a-4232-9441-d19b5a5f76b6","Type":"ContainerStarted","Data":"b9f2e6523c73001026bb7d9135325d8b5e2aa51d8066eaa52a3291832fd0e262"}
Jul 09 09:35:04.619469 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:04.619435    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fvmrl" event={"ID":"fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3","Type":"ContainerStarted","Data":"900def02be0651a0a86e3428c256a5d9cf8d47159dd983ab14bfac13efe38a9d"}
Jul 09 09:35:04.621245 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:04.621215    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-566fz" event={"ID":"e4dd43e4-2a1a-4232-9441-d19b5a5f76b6","Type":"ContainerStarted","Data":"dee3c9d61d045e6d40c9b70104b4f17bcb4dfe61782502a3c58508d67a697275"}
Jul 09 09:35:04.621365 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:04.621345    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-566fz"
Jul 09 09:35:04.625923 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:04.625902    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-566fz"
Jul 09 09:35:04.651572 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:04.651532    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fvmrl" podStartSLOduration=1.356293555 podStartE2EDuration="9.65152145s" totalImagesPullingTime="8.295227895s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:34:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:35:04.651356164 +0000 UTC m=+62.822514731" watchObservedRunningTime="2026-07-09 09:35:04.65152145 +0000 UTC m=+62.822680002"
Jul 09 09:35:04.687314 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:04.687279    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-566fz" podStartSLOduration=1.8602584960000002 podStartE2EDuration="3.687267669s" totalImagesPullingTime="1.827009173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:35:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:35:04.687251185 +0000 UTC m=+62.858409742" watchObservedRunningTime="2026-07-09 09:35:04.687267669 +0000 UTC m=+62.858426224"
Jul 09 09:35:04.877605 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:04.877528    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-765fd786f7-pf2bx"]
Jul 09 09:35:04.879793 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:04.879777    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-765fd786f7-pf2bx"
Jul 09 09:35:04.882400 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:35:04.882374    2549 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"service-ca\" is forbidden: User \"system:node:ip-10-0-137-199.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-137-199.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console\"/\"service-ca\"" type="*v1.ConfigMap"
Jul 09 09:35:04.882638 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:35:04.882619    2549 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"console-serving-cert\" is forbidden: User \"system:node:ip-10-0-137-199.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-137-199.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console\"/\"console-serving-cert\"" type="*v1.Secret"
Jul 09 09:35:04.882961 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:35:04.882936    2549 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-137-199.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-137-199.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Jul 09 09:35:04.882961 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:35:04.882937    2549 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"console-config\" is forbidden: User \"system:node:ip-10-0-137-199.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-137-199.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console\"/\"console-config\"" type="*v1.ConfigMap"
Jul 09 09:35:04.883082 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:35:04.882977    2549 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"console-dockercfg-jtpds\" is forbidden: User \"system:node:ip-10-0-137-199.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-137-199.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console\"/\"console-dockercfg-jtpds\"" type="*v1.Secret"
Jul 09 09:35:04.883082 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:35:04.883030    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"console-765fd786f7-pf2bx\" is forbidden: User \"system:node:ip-10-0-137-199.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-137-199.ec2.internal' and this object" podUID="69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe" pod="openshift-console/console-765fd786f7-pf2bx"
Jul 09 09:35:04.883163 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:35:04.883142    2549 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"oauth-serving-cert\" is forbidden: User \"system:node:ip-10-0-137-199.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-137-199.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" type="*v1.ConfigMap"
Jul 09 09:35:04.883396 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:35:04.883377    2549 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-137-199.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-137-199.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Jul 09 09:35:04.890698 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:35:04.890677    2549 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"console-oauth-config\" is forbidden: User \"system:node:ip-10-0-137-199.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-137-199.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console\"/\"console-oauth-config\"" type="*v1.Secret"
Jul 09 09:35:04.916757 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:04.916734    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-765fd786f7-pf2bx"]
Jul 09 09:35:04.936096 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:04.936073    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-console-oauth-config\") pod \"console-765fd786f7-pf2bx\" (UID: \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\") " pod="openshift-console/console-765fd786f7-pf2bx"
Jul 09 09:35:04.936176 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:04.936098    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t96nz\" (UniqueName: \"kubernetes.io/projected/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-kube-api-access-t96nz\") pod \"console-765fd786f7-pf2bx\" (UID: \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\") " pod="openshift-console/console-765fd786f7-pf2bx"
Jul 09 09:35:04.936176 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:04.936131    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-console-serving-cert\") pod \"console-765fd786f7-pf2bx\" (UID: \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\") " pod="openshift-console/console-765fd786f7-pf2bx"
Jul 09 09:35:04.936265 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:04.936246    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-service-ca\") pod \"console-765fd786f7-pf2bx\" (UID: \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\") " pod="openshift-console/console-765fd786f7-pf2bx"
Jul 09 09:35:04.936299 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:04.936275    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-oauth-serving-cert\") pod \"console-765fd786f7-pf2bx\" (UID: \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\") " pod="openshift-console/console-765fd786f7-pf2bx"
Jul 09 09:35:04.936334 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:04.936299    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-console-config\") pod \"console-765fd786f7-pf2bx\" (UID: \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\") " pod="openshift-console/console-765fd786f7-pf2bx"
Jul 09 09:35:05.037532 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.037495    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-service-ca\") pod \"console-765fd786f7-pf2bx\" (UID: \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\") " pod="openshift-console/console-765fd786f7-pf2bx"
Jul 09 09:35:05.037532 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.037538    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-oauth-serving-cert\") pod \"console-765fd786f7-pf2bx\" (UID: \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\") " pod="openshift-console/console-765fd786f7-pf2bx"
Jul 09 09:35:05.037654 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.037562    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-console-config\") pod \"console-765fd786f7-pf2bx\" (UID: \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\") " pod="openshift-console/console-765fd786f7-pf2bx"
Jul 09 09:35:05.037654 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.037591    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-console-oauth-config\") pod \"console-765fd786f7-pf2bx\" (UID: \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\") " pod="openshift-console/console-765fd786f7-pf2bx"
Jul 09 09:35:05.037654 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.037614    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t96nz\" (UniqueName: \"kubernetes.io/projected/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-kube-api-access-t96nz\") pod \"console-765fd786f7-pf2bx\" (UID: \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\") " pod="openshift-console/console-765fd786f7-pf2bx"
Jul 09 09:35:05.037746 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.037723    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-console-serving-cert\") pod \"console-765fd786f7-pf2bx\" (UID: \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\") " pod="openshift-console/console-765fd786f7-pf2bx"
Jul 09 09:35:05.087187 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.087164    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-8vkwv"]
Jul 09 09:35:05.089031 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.089016    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8vkwv"
Jul 09 09:35:05.094125 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.094108    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 09 09:35:05.094284 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.094124    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 09 09:35:05.094284 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.094157    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 09 09:35:05.094459 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.094407    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 09 09:35:05.094535 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.094462    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-bj4fd\""
Jul 09 09:35:05.094992 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.094976    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 09 09:35:05.118423 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.118406    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-8vkwv"]
Jul 09 09:35:05.138622 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.138557    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/94a5d665-56aa-49cf-8cf6-c9ad52a2227e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-8vkwv\" (UID: \"94a5d665-56aa-49cf-8cf6-c9ad52a2227e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8vkwv"
Jul 09 09:35:05.138622 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.138587    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/94a5d665-56aa-49cf-8cf6-c9ad52a2227e-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-8vkwv\" (UID: \"94a5d665-56aa-49cf-8cf6-c9ad52a2227e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8vkwv"
Jul 09 09:35:05.138801 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.138672    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvd64\" (UniqueName: \"kubernetes.io/projected/94a5d665-56aa-49cf-8cf6-c9ad52a2227e-kube-api-access-cvd64\") pod \"prometheus-operator-567f7b8b7c-8vkwv\" (UID: \"94a5d665-56aa-49cf-8cf6-c9ad52a2227e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8vkwv"
Jul 09 09:35:05.138801 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.138751    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/94a5d665-56aa-49cf-8cf6-c9ad52a2227e-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-8vkwv\" (UID: \"94a5d665-56aa-49cf-8cf6-c9ad52a2227e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8vkwv"
Jul 09 09:35:05.239593 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.239568    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/94a5d665-56aa-49cf-8cf6-c9ad52a2227e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-8vkwv\" (UID: \"94a5d665-56aa-49cf-8cf6-c9ad52a2227e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8vkwv"
Jul 09 09:35:05.239696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.239604    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/94a5d665-56aa-49cf-8cf6-c9ad52a2227e-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-8vkwv\" (UID: \"94a5d665-56aa-49cf-8cf6-c9ad52a2227e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8vkwv"
Jul 09 09:35:05.239696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.239661    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvd64\" (UniqueName: \"kubernetes.io/projected/94a5d665-56aa-49cf-8cf6-c9ad52a2227e-kube-api-access-cvd64\") pod \"prometheus-operator-567f7b8b7c-8vkwv\" (UID: \"94a5d665-56aa-49cf-8cf6-c9ad52a2227e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8vkwv"
Jul 09 09:35:05.239799 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.239722    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/94a5d665-56aa-49cf-8cf6-c9ad52a2227e-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-8vkwv\" (UID: \"94a5d665-56aa-49cf-8cf6-c9ad52a2227e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8vkwv"
Jul 09 09:35:05.240331 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.240312    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/94a5d665-56aa-49cf-8cf6-c9ad52a2227e-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-8vkwv\" (UID: \"94a5d665-56aa-49cf-8cf6-c9ad52a2227e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8vkwv"
Jul 09 09:35:05.241948 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.241920    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/94a5d665-56aa-49cf-8cf6-c9ad52a2227e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-8vkwv\" (UID: \"94a5d665-56aa-49cf-8cf6-c9ad52a2227e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8vkwv"
Jul 09 09:35:05.242033 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.241987    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/94a5d665-56aa-49cf-8cf6-c9ad52a2227e-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-8vkwv\" (UID: \"94a5d665-56aa-49cf-8cf6-c9ad52a2227e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8vkwv"
Jul 09 09:35:05.254126 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.254105    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvd64\" (UniqueName: \"kubernetes.io/projected/94a5d665-56aa-49cf-8cf6-c9ad52a2227e-kube-api-access-cvd64\") pod \"prometheus-operator-567f7b8b7c-8vkwv\" (UID: \"94a5d665-56aa-49cf-8cf6-c9ad52a2227e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8vkwv"
Jul 09 09:35:05.397638 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.397574    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8vkwv"
Jul 09 09:35:05.525731 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.525704    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-8vkwv"]
Jul 09 09:35:05.529627 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:35:05.529601    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94a5d665_56aa_49cf_8cf6_c9ad52a2227e.slice/crio-1fe16a1ae2f101a9b31fbaeada9b0727fda3e31a80d472853ca896ad818810b9 WatchSource:0}: Error finding container 1fe16a1ae2f101a9b31fbaeada9b0727fda3e31a80d472853ca896ad818810b9: Status 404 returned error can't find the container with id 1fe16a1ae2f101a9b31fbaeada9b0727fda3e31a80d472853ca896ad818810b9
Jul 09 09:35:05.624944 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.624915    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8vkwv" event={"ID":"94a5d665-56aa-49cf-8cf6-c9ad52a2227e","Type":"ContainerStarted","Data":"1fe16a1ae2f101a9b31fbaeada9b0727fda3e31a80d472853ca896ad818810b9"}
Jul 09 09:35:05.869000 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.868971    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 09 09:35:05.880150 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:05.880123    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-console-serving-cert\") pod \"console-765fd786f7-pf2bx\" (UID: \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\") " pod="openshift-console/console-765fd786f7-pf2bx"
Jul 09 09:35:06.038063 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:35:06.038034    2549 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: failed to sync secret cache: timed out waiting for the condition
Jul 09 09:35:06.038063 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:35:06.038062    2549 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: failed to sync configmap cache: timed out waiting for the condition
Jul 09 09:35:06.038236 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:35:06.038106    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-console-oauth-config podName:69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe nodeName:}" failed. No retries permitted until 2026-07-09 09:35:06.5380917 +0000 UTC m=+64.709250247 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-console-oauth-config") pod "console-765fd786f7-pf2bx" (UID: "69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe") : failed to sync secret cache: timed out waiting for the condition
Jul 09 09:35:06.038236 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:35:06.038034    2549 configmap.go:193] Couldn't get configMap openshift-console/console-config: failed to sync configmap cache: timed out waiting for the condition
Jul 09 09:35:06.038236 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:35:06.038132    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-oauth-serving-cert podName:69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe nodeName:}" failed. No retries permitted until 2026-07-09 09:35:06.538115461 +0000 UTC m=+64.709274017 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-oauth-serving-cert") pod "console-765fd786f7-pf2bx" (UID: "69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe") : failed to sync configmap cache: timed out waiting for the condition
Jul 09 09:35:06.038236 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:35:06.038161    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-console-config podName:69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe nodeName:}" failed. No retries permitted until 2026-07-09 09:35:06.538150868 +0000 UTC m=+64.709309403 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-console-config") pod "console-765fd786f7-pf2bx" (UID: "69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe") : failed to sync configmap cache: timed out waiting for the condition
Jul 09 09:35:06.038236 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:35:06.038048    2549 configmap.go:193] Couldn't get configMap openshift-console/service-ca: failed to sync configmap cache: timed out waiting for the condition
Jul 09 09:35:06.038236 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:35:06.038191    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-service-ca podName:69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe nodeName:}" failed. No retries permitted until 2026-07-09 09:35:06.538183851 +0000 UTC m=+64.709342385 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-service-ca") pod "console-765fd786f7-pf2bx" (UID: "69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe") : failed to sync configmap cache: timed out waiting for the condition
Jul 09 09:35:06.053974 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:35:06.053955    2549 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition
Jul 09 09:35:06.056370 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.056355    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 09 09:35:06.155993 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.155937    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 09 09:35:06.164899 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:35:06.164878    2549 projected.go:196] Error preparing data for projected volume kube-api-access-t96nz for pod openshift-console/console-765fd786f7-pf2bx: failed to sync configmap cache: timed out waiting for the condition
Jul 09 09:35:06.164994 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:35:06.164943    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-kube-api-access-t96nz podName:69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe nodeName:}" failed. No retries permitted until 2026-07-09 09:35:06.664925726 +0000 UTC m=+64.836084277 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-t96nz" (UniqueName: "kubernetes.io/projected/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-kube-api-access-t96nz") pod "console-765fd786f7-pf2bx" (UID: "69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe") : failed to sync configmap cache: timed out waiting for the condition
Jul 09 09:35:06.208803 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.208773    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 09 09:35:06.250909 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.250884    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 09 09:35:06.271487 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.271462    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 09 09:35:06.379037 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.379010    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 09 09:35:06.434857 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.434793    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57475f5bfb-lg5kw"]
Jul 09 09:35:06.436709 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.436690    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57475f5bfb-lg5kw"
Jul 09 09:35:06.449503 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.449478    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 09 09:35:06.450017 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.449999    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57475f5bfb-lg5kw"]
Jul 09 09:35:06.461170 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.461148    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jtpds\""
Jul 09 09:35:06.552397 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.552367    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0fb00168-57af-42e0-8397-c79457b74799-console-oauth-config\") pod \"console-57475f5bfb-lg5kw\" (UID: \"0fb00168-57af-42e0-8397-c79457b74799\") " pod="openshift-console/console-57475f5bfb-lg5kw"
Jul 09 09:35:06.552581 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.552410    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0fb00168-57af-42e0-8397-c79457b74799-trusted-ca-bundle\") pod \"console-57475f5bfb-lg5kw\" (UID: \"0fb00168-57af-42e0-8397-c79457b74799\") " pod="openshift-console/console-57475f5bfb-lg5kw"
Jul 09 09:35:06.552581 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.552442    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0fb00168-57af-42e0-8397-c79457b74799-service-ca\") pod \"console-57475f5bfb-lg5kw\" (UID: \"0fb00168-57af-42e0-8397-c79457b74799\") " pod="openshift-console/console-57475f5bfb-lg5kw"
Jul 09 09:35:06.552581 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.552525    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0fb00168-57af-42e0-8397-c79457b74799-oauth-serving-cert\") pod \"console-57475f5bfb-lg5kw\" (UID: \"0fb00168-57af-42e0-8397-c79457b74799\") " pod="openshift-console/console-57475f5bfb-lg5kw"
Jul 09 09:35:06.552698 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.552582    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0fb00168-57af-42e0-8397-c79457b74799-console-serving-cert\") pod \"console-57475f5bfb-lg5kw\" (UID: \"0fb00168-57af-42e0-8397-c79457b74799\") " pod="openshift-console/console-57475f5bfb-lg5kw"
Jul 09 09:35:06.552698 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.552614    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-service-ca\") pod \"console-765fd786f7-pf2bx\" (UID: \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\") " pod="openshift-console/console-765fd786f7-pf2bx"
Jul 09 09:35:06.552698 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.552633    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bgjm\" (UniqueName: \"kubernetes.io/projected/0fb00168-57af-42e0-8397-c79457b74799-kube-api-access-4bgjm\") pod \"console-57475f5bfb-lg5kw\" (UID: \"0fb00168-57af-42e0-8397-c79457b74799\") " pod="openshift-console/console-57475f5bfb-lg5kw"
Jul 09 09:35:06.552698 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.552651    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-console-config\") pod \"console-765fd786f7-pf2bx\" (UID: \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\") " pod="openshift-console/console-765fd786f7-pf2bx"
Jul 09 09:35:06.552698 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.552684    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-console-oauth-config\") pod \"console-765fd786f7-pf2bx\" (UID: \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\") " pod="openshift-console/console-765fd786f7-pf2bx"
Jul 09 09:35:06.552958 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.552876    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-oauth-serving-cert\") pod \"console-765fd786f7-pf2bx\" (UID: \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\") " pod="openshift-console/console-765fd786f7-pf2bx"
Jul 09 09:35:06.552958 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.552907    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0fb00168-57af-42e0-8397-c79457b74799-console-config\") pod \"console-57475f5bfb-lg5kw\" (UID: \"0fb00168-57af-42e0-8397-c79457b74799\") " pod="openshift-console/console-57475f5bfb-lg5kw"
Jul 09 09:35:06.553432 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.553410    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-service-ca\") pod \"console-765fd786f7-pf2bx\" (UID: \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\") " pod="openshift-console/console-765fd786f7-pf2bx"
Jul 09 09:35:06.553558 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.553476    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-console-config\") pod \"console-765fd786f7-pf2bx\" (UID: \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\") " pod="openshift-console/console-765fd786f7-pf2bx"
Jul 09 09:35:06.553558 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.553498    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-oauth-serving-cert\") pod \"console-765fd786f7-pf2bx\" (UID: \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\") " pod="openshift-console/console-765fd786f7-pf2bx"
Jul 09 09:35:06.555521 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.555496    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-console-oauth-config\") pod \"console-765fd786f7-pf2bx\" (UID: \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\") " pod="openshift-console/console-765fd786f7-pf2bx"
Jul 09 09:35:06.654094 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.654061    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0fb00168-57af-42e0-8397-c79457b74799-console-config\") pod \"console-57475f5bfb-lg5kw\" (UID: \"0fb00168-57af-42e0-8397-c79457b74799\") " pod="openshift-console/console-57475f5bfb-lg5kw"
Jul 09 09:35:06.654566 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.654147    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0fb00168-57af-42e0-8397-c79457b74799-console-oauth-config\") pod \"console-57475f5bfb-lg5kw\" (UID: \"0fb00168-57af-42e0-8397-c79457b74799\") " pod="openshift-console/console-57475f5bfb-lg5kw"
Jul 09 09:35:06.654566 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.654187    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0fb00168-57af-42e0-8397-c79457b74799-trusted-ca-bundle\") pod \"console-57475f5bfb-lg5kw\" (UID: \"0fb00168-57af-42e0-8397-c79457b74799\") " pod="openshift-console/console-57475f5bfb-lg5kw"
Jul 09 09:35:06.654566 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.654212    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0fb00168-57af-42e0-8397-c79457b74799-service-ca\") pod \"console-57475f5bfb-lg5kw\" (UID: \"0fb00168-57af-42e0-8397-c79457b74799\") " pod="openshift-console/console-57475f5bfb-lg5kw"
Jul 09 09:35:06.654566 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.654250    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0fb00168-57af-42e0-8397-c79457b74799-oauth-serving-cert\") pod \"console-57475f5bfb-lg5kw\" (UID: \"0fb00168-57af-42e0-8397-c79457b74799\") " pod="openshift-console/console-57475f5bfb-lg5kw"
Jul 09 09:35:06.654566 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.654291    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0fb00168-57af-42e0-8397-c79457b74799-console-serving-cert\") pod \"console-57475f5bfb-lg5kw\" (UID: \"0fb00168-57af-42e0-8397-c79457b74799\") " pod="openshift-console/console-57475f5bfb-lg5kw"
Jul 09 09:35:06.654566 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.654324    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bgjm\" (UniqueName: \"kubernetes.io/projected/0fb00168-57af-42e0-8397-c79457b74799-kube-api-access-4bgjm\") pod \"console-57475f5bfb-lg5kw\" (UID: \"0fb00168-57af-42e0-8397-c79457b74799\") " pod="openshift-console/console-57475f5bfb-lg5kw"
Jul 09 09:35:06.655401 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.654900    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0fb00168-57af-42e0-8397-c79457b74799-console-config\") pod \"console-57475f5bfb-lg5kw\" (UID: \"0fb00168-57af-42e0-8397-c79457b74799\") " pod="openshift-console/console-57475f5bfb-lg5kw"
Jul 09 09:35:06.655401 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.655353    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0fb00168-57af-42e0-8397-c79457b74799-oauth-serving-cert\") pod \"console-57475f5bfb-lg5kw\" (UID: \"0fb00168-57af-42e0-8397-c79457b74799\") " pod="openshift-console/console-57475f5bfb-lg5kw"
Jul 09 09:35:06.655401 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.655363    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0fb00168-57af-42e0-8397-c79457b74799-service-ca\") pod \"console-57475f5bfb-lg5kw\" (UID: \"0fb00168-57af-42e0-8397-c79457b74799\") " pod="openshift-console/console-57475f5bfb-lg5kw"
Jul 09 09:35:06.655693 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.655667    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0fb00168-57af-42e0-8397-c79457b74799-trusted-ca-bundle\") pod \"console-57475f5bfb-lg5kw\" (UID: \"0fb00168-57af-42e0-8397-c79457b74799\") " pod="openshift-console/console-57475f5bfb-lg5kw"
Jul 09 09:35:06.656969 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.656945    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0fb00168-57af-42e0-8397-c79457b74799-console-serving-cert\") pod \"console-57475f5bfb-lg5kw\" (UID: \"0fb00168-57af-42e0-8397-c79457b74799\") " pod="openshift-console/console-57475f5bfb-lg5kw"
Jul 09 09:35:06.657070 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.656950    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0fb00168-57af-42e0-8397-c79457b74799-console-oauth-config\") pod \"console-57475f5bfb-lg5kw\" (UID: \"0fb00168-57af-42e0-8397-c79457b74799\") " pod="openshift-console/console-57475f5bfb-lg5kw"
Jul 09 09:35:06.663523 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.663503    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bgjm\" (UniqueName: \"kubernetes.io/projected/0fb00168-57af-42e0-8397-c79457b74799-kube-api-access-4bgjm\") pod \"console-57475f5bfb-lg5kw\" (UID: \"0fb00168-57af-42e0-8397-c79457b74799\") " pod="openshift-console/console-57475f5bfb-lg5kw"
Jul 09 09:35:06.748842 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.748811    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57475f5bfb-lg5kw"
Jul 09 09:35:06.755429 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.755404    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t96nz\" (UniqueName: \"kubernetes.io/projected/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-kube-api-access-t96nz\") pod \"console-765fd786f7-pf2bx\" (UID: \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\") " pod="openshift-console/console-765fd786f7-pf2bx"
Jul 09 09:35:06.757991 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.757967    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t96nz\" (UniqueName: \"kubernetes.io/projected/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-kube-api-access-t96nz\") pod \"console-765fd786f7-pf2bx\" (UID: \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\") " pod="openshift-console/console-765fd786f7-pf2bx"
Jul 09 09:35:06.988710 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:06.988679    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-765fd786f7-pf2bx"
Jul 09 09:35:07.017640 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:07.017601    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57475f5bfb-lg5kw"]
Jul 09 09:35:07.025451 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:35:07.025216    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fb00168_57af_42e0_8397_c79457b74799.slice/crio-79f1e412047fd8e7de2f94db38a951e2c4f69228d0ff77637ec03c22f82a9c2b WatchSource:0}: Error finding container 79f1e412047fd8e7de2f94db38a951e2c4f69228d0ff77637ec03c22f82a9c2b: Status 404 returned error can't find the container with id 79f1e412047fd8e7de2f94db38a951e2c4f69228d0ff77637ec03c22f82a9c2b
Jul 09 09:35:07.120483 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:07.120420    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-765fd786f7-pf2bx"]
Jul 09 09:35:07.123121 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:35:07.123097    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69a05f8e_6f4f_40d4_9ca0_2ac466a16dbe.slice/crio-bd70623ccc6f753524dc49725a9301f904fdcf2a99491fd710260d276c8ff0b8 WatchSource:0}: Error finding container bd70623ccc6f753524dc49725a9301f904fdcf2a99491fd710260d276c8ff0b8: Status 404 returned error can't find the container with id bd70623ccc6f753524dc49725a9301f904fdcf2a99491fd710260d276c8ff0b8
Jul 09 09:35:07.634734 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:07.634697    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8vkwv" event={"ID":"94a5d665-56aa-49cf-8cf6-c9ad52a2227e","Type":"ContainerStarted","Data":"383e61aff7c81e7f5442f20226d68bee4504849587979be37cb834b72651cc1d"}
Jul 09 09:35:07.634944 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:07.634771    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8vkwv" event={"ID":"94a5d665-56aa-49cf-8cf6-c9ad52a2227e","Type":"ContainerStarted","Data":"5a8b06462d5bc1ccbe43ef06fcb5d4c165325f8a226b207c08a23e0e40814d92"}
Jul 09 09:35:07.636622 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:07.636592    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57475f5bfb-lg5kw" event={"ID":"0fb00168-57af-42e0-8397-c79457b74799","Type":"ContainerStarted","Data":"79f1e412047fd8e7de2f94db38a951e2c4f69228d0ff77637ec03c22f82a9c2b"}
Jul 09 09:35:07.637978 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:07.637956    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-765fd786f7-pf2bx" event={"ID":"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe","Type":"ContainerStarted","Data":"bd70623ccc6f753524dc49725a9301f904fdcf2a99491fd710260d276c8ff0b8"}
Jul 09 09:35:07.655786 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:07.655739    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-8vkwv" podStartSLOduration=1.260458428 podStartE2EDuration="2.655722701s" totalImagesPullingTime="1.395264273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:35:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:35:07.654276351 +0000 UTC m=+65.825434908" watchObservedRunningTime="2026-07-09 09:35:07.655722701 +0000 UTC m=+65.826881256"
Jul 09 09:35:08.066744 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:08.066709    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72ab801d-5d43-4fbc-a26a-60fab96be57b-metrics-certs\") pod \"network-metrics-daemon-rqzsx\" (UID: \"72ab801d-5d43-4fbc-a26a-60fab96be57b\") " pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:35:08.069552 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:08.069526    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 09:35:08.080584 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:08.080536    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72ab801d-5d43-4fbc-a26a-60fab96be57b-metrics-certs\") pod \"network-metrics-daemon-rqzsx\" (UID: \"72ab801d-5d43-4fbc-a26a-60fab96be57b\") " pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:35:08.100183 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:08.100154    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qlfc8\""
Jul 09 09:35:08.108541 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:08.108517    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rqzsx"
Jul 09 09:35:08.167969 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:08.167192    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6299\" (UniqueName: \"kubernetes.io/projected/2ab646db-3779-4dbf-a300-a7b166d929ab-kube-api-access-t6299\") pod \"network-check-target-ljjkv\" (UID: \"2ab646db-3779-4dbf-a300-a7b166d929ab\") " pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:35:08.167969 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:08.167262    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-original-pull-secret\") pod \"global-pull-secret-syncer-ccqx8\" (UID: \"282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee\") " pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:35:08.170701 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:08.170677    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 09:35:08.170935 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:08.170918    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 09:35:08.180448 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:08.180423    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee-original-pull-secret\") pod \"global-pull-secret-syncer-ccqx8\" (UID: \"282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee\") " pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:35:08.180970 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:08.180952    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 09:35:08.190728 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:08.190706    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6299\" (UniqueName: \"kubernetes.io/projected/2ab646db-3779-4dbf-a300-a7b166d929ab-kube-api-access-t6299\") pod \"network-check-target-ljjkv\" (UID: \"2ab646db-3779-4dbf-a300-a7b166d929ab\") " pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:35:08.257846 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:08.257818    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rqzsx"]
Jul 09 09:35:08.262553 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:35:08.262516    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72ab801d_5d43_4fbc_a26a_60fab96be57b.slice/crio-a6c7eee32efaddecc1c512a2631dd3d6808b28444f4d5fdcbd02a304b8d10b54 WatchSource:0}: Error finding container a6c7eee32efaddecc1c512a2631dd3d6808b28444f4d5fdcbd02a304b8d10b54: Status 404 returned error can't find the container with id a6c7eee32efaddecc1c512a2631dd3d6808b28444f4d5fdcbd02a304b8d10b54
Jul 09 09:35:08.287307 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:08.287269    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccqx8"
Jul 09 09:35:08.410256 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:08.410209    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-j2x9h\""
Jul 09 09:35:08.417859 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:08.417829    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:35:08.439346 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:08.439304    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ccqx8"]
Jul 09 09:35:08.444773 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:35:08.444741    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod282fa84e_5b6f_4cb1_b6bf_f50d9ad479ee.slice/crio-c41502902cee74ca97372afa4d327711f4471e5f75fcc2b14798d6674b2bf5d4 WatchSource:0}: Error finding container c41502902cee74ca97372afa4d327711f4471e5f75fcc2b14798d6674b2bf5d4: Status 404 returned error can't find the container with id c41502902cee74ca97372afa4d327711f4471e5f75fcc2b14798d6674b2bf5d4
Jul 09 09:35:08.568060 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:08.568027    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ljjkv"]
Jul 09 09:35:08.570938 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:35:08.570908    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ab646db_3779_4dbf_a300_a7b166d929ab.slice/crio-6be1c162519ce70db236d60e186e928aa6c03d4e1828f9999537062cc89cce42 WatchSource:0}: Error finding container 6be1c162519ce70db236d60e186e928aa6c03d4e1828f9999537062cc89cce42: Status 404 returned error can't find the container with id 6be1c162519ce70db236d60e186e928aa6c03d4e1828f9999537062cc89cce42
Jul 09 09:35:08.646245 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:08.646168    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rqzsx" event={"ID":"72ab801d-5d43-4fbc-a26a-60fab96be57b","Type":"ContainerStarted","Data":"a6c7eee32efaddecc1c512a2631dd3d6808b28444f4d5fdcbd02a304b8d10b54"}
Jul 09 09:35:08.647879 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:08.647800    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ccqx8" event={"ID":"282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee","Type":"ContainerStarted","Data":"c41502902cee74ca97372afa4d327711f4471e5f75fcc2b14798d6674b2bf5d4"}
Jul 09 09:35:08.649616 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:08.649565    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ljjkv" event={"ID":"2ab646db-3779-4dbf-a300-a7b166d929ab","Type":"ContainerStarted","Data":"6be1c162519ce70db236d60e186e928aa6c03d4e1828f9999537062cc89cce42"}
Jul 09 09:35:09.455438 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.451937    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-pprmx"]
Jul 09 09:35:09.487750 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.487718    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-57lfj"]
Jul 09 09:35:09.489274 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.489252    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pprmx"
Jul 09 09:35:09.498070 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.497821    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 09 09:35:09.498446 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.498273    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 09 09:35:09.499153 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.498603    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-dgq9x\""
Jul 09 09:35:09.504326 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.503705    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-pprmx"]
Jul 09 09:35:09.504326 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.503750    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-tv629"]
Jul 09 09:35:09.505154 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.504570    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-57lfj"
Jul 09 09:35:09.508900 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.507992    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-hjjvr\""
Jul 09 09:35:09.508900 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.508472    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 09 09:35:09.508900 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.508677    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 09 09:35:09.509441 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.509251    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 09 09:35:09.525827 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.525780    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-57lfj"]
Jul 09 09:35:09.525978 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.525922    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tv629"
Jul 09 09:35:09.530095 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.528700    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 09 09:35:09.530095 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.529131    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 09 09:35:09.530095 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.529426    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 09 09:35:09.530095 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.529675    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-l2sn9\""
Jul 09 09:35:09.580463 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.580315    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37e2f403-c4b2-42d0-8613-8755b43545b8-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-57lfj\" (UID: \"37e2f403-c4b2-42d0-8613-8755b43545b8\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-57lfj"
Jul 09 09:35:09.580463 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.580352    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/37e2f403-c4b2-42d0-8613-8755b43545b8-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-57lfj\" (UID: \"37e2f403-c4b2-42d0-8613-8755b43545b8\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-57lfj"
Jul 09 09:35:09.580463 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.580409    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/37e2f403-c4b2-42d0-8613-8755b43545b8-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-57lfj\" (UID: \"37e2f403-c4b2-42d0-8613-8755b43545b8\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-57lfj"
Jul 09 09:35:09.580744 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.580517    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac99270f-01d2-4095-b9f6-84129e22918e-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-pprmx\" (UID: \"ac99270f-01d2-4095-b9f6-84129e22918e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pprmx"
Jul 09 09:35:09.580744 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.580569    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ac99270f-01d2-4095-b9f6-84129e22918e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-pprmx\" (UID: \"ac99270f-01d2-4095-b9f6-84129e22918e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pprmx"
Jul 09 09:35:09.580744 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.580615    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqqct\" (UniqueName: \"kubernetes.io/projected/ac99270f-01d2-4095-b9f6-84129e22918e-kube-api-access-qqqct\") pod \"openshift-state-metrics-54cf96dfcf-pprmx\" (UID: \"ac99270f-01d2-4095-b9f6-84129e22918e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pprmx"
Jul 09 09:35:09.580744 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.580643    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/37e2f403-c4b2-42d0-8613-8755b43545b8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-57lfj\" (UID: \"37e2f403-c4b2-42d0-8613-8755b43545b8\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-57lfj"
Jul 09 09:35:09.580744 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.580669    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw4zl\" (UniqueName: \"kubernetes.io/projected/37e2f403-c4b2-42d0-8613-8755b43545b8-kube-api-access-tw4zl\") pod \"kube-state-metrics-7c9dbb8b7d-57lfj\" (UID: \"37e2f403-c4b2-42d0-8613-8755b43545b8\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-57lfj"
Jul 09 09:35:09.580744 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.580697    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ac99270f-01d2-4095-b9f6-84129e22918e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-pprmx\" (UID: \"ac99270f-01d2-4095-b9f6-84129e22918e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pprmx"
Jul 09 09:35:09.580744 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.580724    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/37e2f403-c4b2-42d0-8613-8755b43545b8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-57lfj\" (UID: \"37e2f403-c4b2-42d0-8613-8755b43545b8\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-57lfj"
Jul 09 09:35:09.681424 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.681390    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqqct\" (UniqueName: \"kubernetes.io/projected/ac99270f-01d2-4095-b9f6-84129e22918e-kube-api-access-qqqct\") pod \"openshift-state-metrics-54cf96dfcf-pprmx\" (UID: \"ac99270f-01d2-4095-b9f6-84129e22918e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pprmx"
Jul 09 09:35:09.681586 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.681435    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25727dba-8eab-4eb5-85e2-6afa43578f4a-sys\") pod \"node-exporter-tv629\" (UID: \"25727dba-8eab-4eb5-85e2-6afa43578f4a\") " pod="openshift-monitoring/node-exporter-tv629"
Jul 09 09:35:09.681586 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.681466    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/37e2f403-c4b2-42d0-8613-8755b43545b8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-57lfj\" (UID: \"37e2f403-c4b2-42d0-8613-8755b43545b8\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-57lfj"
Jul 09 09:35:09.681586 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.681494    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw4zl\" (UniqueName: \"kubernetes.io/projected/37e2f403-c4b2-42d0-8613-8755b43545b8-kube-api-access-tw4zl\") pod \"kube-state-metrics-7c9dbb8b7d-57lfj\" (UID: \"37e2f403-c4b2-42d0-8613-8755b43545b8\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-57lfj"
Jul 09 09:35:09.681586 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.681523    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ac99270f-01d2-4095-b9f6-84129e22918e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-pprmx\" (UID: \"ac99270f-01d2-4095-b9f6-84129e22918e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pprmx"
Jul 09 09:35:09.681586 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.681550    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/37e2f403-c4b2-42d0-8613-8755b43545b8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-57lfj\" (UID: \"37e2f403-c4b2-42d0-8613-8755b43545b8\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-57lfj"
Jul 09 09:35:09.681586 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.681582    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25727dba-8eab-4eb5-85e2-6afa43578f4a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tv629\" (UID: \"25727dba-8eab-4eb5-85e2-6afa43578f4a\") " pod="openshift-monitoring/node-exporter-tv629"
Jul 09 09:35:09.681928 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.681611    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/25727dba-8eab-4eb5-85e2-6afa43578f4a-node-exporter-tls\") pod \"node-exporter-tv629\" (UID: \"25727dba-8eab-4eb5-85e2-6afa43578f4a\") " pod="openshift-monitoring/node-exporter-tv629"
Jul 09 09:35:09.681928 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.681641    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/25727dba-8eab-4eb5-85e2-6afa43578f4a-node-exporter-accelerators-collector-config\") pod \"node-exporter-tv629\" (UID: \"25727dba-8eab-4eb5-85e2-6afa43578f4a\") " pod="openshift-monitoring/node-exporter-tv629"
Jul 09 09:35:09.681928 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.681680    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37e2f403-c4b2-42d0-8613-8755b43545b8-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-57lfj\" (UID: \"37e2f403-c4b2-42d0-8613-8755b43545b8\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-57lfj"
Jul 09 09:35:09.681928 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.681702    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25727dba-8eab-4eb5-85e2-6afa43578f4a-metrics-client-ca\") pod \"node-exporter-tv629\" (UID: \"25727dba-8eab-4eb5-85e2-6afa43578f4a\") " pod="openshift-monitoring/node-exporter-tv629"
Jul 09 09:35:09.681928 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.681727    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/37e2f403-c4b2-42d0-8613-8755b43545b8-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-57lfj\" (UID: \"37e2f403-c4b2-42d0-8613-8755b43545b8\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-57lfj"
Jul 09 09:35:09.681928 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.681754    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47x7d\" (UniqueName: \"kubernetes.io/projected/25727dba-8eab-4eb5-85e2-6afa43578f4a-kube-api-access-47x7d\") pod \"node-exporter-tv629\" (UID: \"25727dba-8eab-4eb5-85e2-6afa43578f4a\") " pod="openshift-monitoring/node-exporter-tv629"
Jul 09 09:35:09.681928 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.681802    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/37e2f403-c4b2-42d0-8613-8755b43545b8-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-57lfj\" (UID: \"37e2f403-c4b2-42d0-8613-8755b43545b8\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-57lfj"
Jul 09 09:35:09.681928 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.681829    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/25727dba-8eab-4eb5-85e2-6afa43578f4a-node-exporter-textfile\") pod \"node-exporter-tv629\" (UID: \"25727dba-8eab-4eb5-85e2-6afa43578f4a\") " pod="openshift-monitoring/node-exporter-tv629"
Jul 09 09:35:09.681928 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.681854    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/25727dba-8eab-4eb5-85e2-6afa43578f4a-node-exporter-wtmp\") pod \"node-exporter-tv629\" (UID: \"25727dba-8eab-4eb5-85e2-6afa43578f4a\") " pod="openshift-monitoring/node-exporter-tv629"
Jul 09 09:35:09.681928 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.681895    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac99270f-01d2-4095-b9f6-84129e22918e-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-pprmx\" (UID: \"ac99270f-01d2-4095-b9f6-84129e22918e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pprmx"
Jul 09 09:35:09.681928 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.681925    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ac99270f-01d2-4095-b9f6-84129e22918e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-pprmx\" (UID: \"ac99270f-01d2-4095-b9f6-84129e22918e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pprmx"
Jul 09 09:35:09.682523 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.681963    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/25727dba-8eab-4eb5-85e2-6afa43578f4a-root\") pod \"node-exporter-tv629\" (UID: \"25727dba-8eab-4eb5-85e2-6afa43578f4a\") " pod="openshift-monitoring/node-exporter-tv629"
Jul 09 09:35:09.683434 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.682990    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/37e2f403-c4b2-42d0-8613-8755b43545b8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-57lfj\" (UID: \"37e2f403-c4b2-42d0-8613-8755b43545b8\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-57lfj"
Jul 09 09:35:09.683434 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:35:09.683282    2549 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 09 09:35:09.683434 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:35:09.683389    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37e2f403-c4b2-42d0-8613-8755b43545b8-kube-state-metrics-tls podName:37e2f403-c4b2-42d0-8613-8755b43545b8 nodeName:}" failed. No retries permitted until 2026-07-09 09:35:10.183367981 +0000 UTC m=+68.354526520 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/37e2f403-c4b2-42d0-8613-8755b43545b8-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-57lfj" (UID: "37e2f403-c4b2-42d0-8613-8755b43545b8") : secret "kube-state-metrics-tls" not found
Jul 09 09:35:09.683685 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:35:09.683607    2549 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 09 09:35:09.683685 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:35:09.683680    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac99270f-01d2-4095-b9f6-84129e22918e-openshift-state-metrics-tls podName:ac99270f-01d2-4095-b9f6-84129e22918e nodeName:}" failed. No retries permitted until 2026-07-09 09:35:10.183660614 +0000 UTC m=+68.354819163 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/ac99270f-01d2-4095-b9f6-84129e22918e-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-pprmx" (UID: "ac99270f-01d2-4095-b9f6-84129e22918e") : secret "openshift-state-metrics-tls" not found
Jul 09 09:35:09.683914 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.683889    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37e2f403-c4b2-42d0-8613-8755b43545b8-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-57lfj\" (UID: \"37e2f403-c4b2-42d0-8613-8755b43545b8\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-57lfj"
Jul 09 09:35:09.684033 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.683889    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ac99270f-01d2-4095-b9f6-84129e22918e-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-pprmx\" (UID: \"ac99270f-01d2-4095-b9f6-84129e22918e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pprmx"
Jul 09 09:35:09.684203 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.684163    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/37e2f403-c4b2-42d0-8613-8755b43545b8-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-57lfj\" (UID: \"37e2f403-c4b2-42d0-8613-8755b43545b8\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-57lfj"
Jul 09 09:35:09.686441 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.686400    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/37e2f403-c4b2-42d0-8613-8755b43545b8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-57lfj\" (UID: \"37e2f403-c4b2-42d0-8613-8755b43545b8\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-57lfj"
Jul 09 09:35:09.687526 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.687502    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ac99270f-01d2-4095-b9f6-84129e22918e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-pprmx\" (UID: \"ac99270f-01d2-4095-b9f6-84129e22918e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pprmx"
Jul 09 09:35:09.691796 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.691774    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqqct\" (UniqueName: \"kubernetes.io/projected/ac99270f-01d2-4095-b9f6-84129e22918e-kube-api-access-qqqct\") pod \"openshift-state-metrics-54cf96dfcf-pprmx\" (UID: \"ac99270f-01d2-4095-b9f6-84129e22918e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pprmx"
Jul 09 09:35:09.693857 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.693741    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw4zl\" (UniqueName: \"kubernetes.io/projected/37e2f403-c4b2-42d0-8613-8755b43545b8-kube-api-access-tw4zl\") pod \"kube-state-metrics-7c9dbb8b7d-57lfj\" (UID: \"37e2f403-c4b2-42d0-8613-8755b43545b8\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-57lfj"
Jul 09 09:35:09.784014 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.783273    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47x7d\" (UniqueName: \"kubernetes.io/projected/25727dba-8eab-4eb5-85e2-6afa43578f4a-kube-api-access-47x7d\") pod \"node-exporter-tv629\" (UID: \"25727dba-8eab-4eb5-85e2-6afa43578f4a\") " pod="openshift-monitoring/node-exporter-tv629"
Jul 09 09:35:09.784014 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.783471    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/25727dba-8eab-4eb5-85e2-6afa43578f4a-node-exporter-textfile\") pod \"node-exporter-tv629\" (UID: \"25727dba-8eab-4eb5-85e2-6afa43578f4a\") " pod="openshift-monitoring/node-exporter-tv629"
Jul 09 09:35:09.784014 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.783543    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/25727dba-8eab-4eb5-85e2-6afa43578f4a-node-exporter-wtmp\") pod \"node-exporter-tv629\" (UID: \"25727dba-8eab-4eb5-85e2-6afa43578f4a\") " pod="openshift-monitoring/node-exporter-tv629"
Jul 09 09:35:09.784014 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.783627    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/25727dba-8eab-4eb5-85e2-6afa43578f4a-root\") pod \"node-exporter-tv629\" (UID: \"25727dba-8eab-4eb5-85e2-6afa43578f4a\") " pod="openshift-monitoring/node-exporter-tv629"
Jul 09 09:35:09.784014 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.783656    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25727dba-8eab-4eb5-85e2-6afa43578f4a-sys\") pod \"node-exporter-tv629\" (UID: \"25727dba-8eab-4eb5-85e2-6afa43578f4a\") " pod="openshift-monitoring/node-exporter-tv629"
Jul 09 09:35:09.784014 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.783695    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25727dba-8eab-4eb5-85e2-6afa43578f4a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tv629\" (UID: \"25727dba-8eab-4eb5-85e2-6afa43578f4a\") " pod="openshift-monitoring/node-exporter-tv629"
Jul 09 09:35:09.784014 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.783724    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/25727dba-8eab-4eb5-85e2-6afa43578f4a-node-exporter-tls\") pod \"node-exporter-tv629\" (UID: \"25727dba-8eab-4eb5-85e2-6afa43578f4a\") " pod="openshift-monitoring/node-exporter-tv629"
Jul 09 09:35:09.784014 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.783758    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/25727dba-8eab-4eb5-85e2-6afa43578f4a-node-exporter-accelerators-collector-config\") pod \"node-exporter-tv629\" (UID: \"25727dba-8eab-4eb5-85e2-6afa43578f4a\") " pod="openshift-monitoring/node-exporter-tv629"
Jul 09 09:35:09.784014 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.783799    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/25727dba-8eab-4eb5-85e2-6afa43578f4a-node-exporter-wtmp\") pod \"node-exporter-tv629\" (UID: \"25727dba-8eab-4eb5-85e2-6afa43578f4a\") " pod="openshift-monitoring/node-exporter-tv629"
Jul 09 09:35:09.784014 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.783806    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25727dba-8eab-4eb5-85e2-6afa43578f4a-metrics-client-ca\") pod \"node-exporter-tv629\" (UID: \"25727dba-8eab-4eb5-85e2-6afa43578f4a\") " pod="openshift-monitoring/node-exporter-tv629"
Jul 09 09:35:09.784014 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.783947    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/25727dba-8eab-4eb5-85e2-6afa43578f4a-node-exporter-textfile\") pod \"node-exporter-tv629\" (UID: \"25727dba-8eab-4eb5-85e2-6afa43578f4a\") " pod="openshift-monitoring/node-exporter-tv629"
Jul 09 09:35:09.784698 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.784310    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25727dba-8eab-4eb5-85e2-6afa43578f4a-metrics-client-ca\") pod \"node-exporter-tv629\" (UID: \"25727dba-8eab-4eb5-85e2-6afa43578f4a\") " pod="openshift-monitoring/node-exporter-tv629"
Jul 09 09:35:09.784698 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.784366    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/25727dba-8eab-4eb5-85e2-6afa43578f4a-root\") pod \"node-exporter-tv629\" (UID: \"25727dba-8eab-4eb5-85e2-6afa43578f4a\") " pod="openshift-monitoring/node-exporter-tv629"
Jul 09 09:35:09.784698 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.784408    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25727dba-8eab-4eb5-85e2-6afa43578f4a-sys\") pod \"node-exporter-tv629\" (UID: \"25727dba-8eab-4eb5-85e2-6afa43578f4a\") " pod="openshift-monitoring/node-exporter-tv629"
Jul 09 09:35:09.785526 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.785473    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/25727dba-8eab-4eb5-85e2-6afa43578f4a-node-exporter-accelerators-collector-config\") pod \"node-exporter-tv629\" (UID: \"25727dba-8eab-4eb5-85e2-6afa43578f4a\") " pod="openshift-monitoring/node-exporter-tv629"
Jul 09 09:35:09.788232 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.788191    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/25727dba-8eab-4eb5-85e2-6afa43578f4a-node-exporter-tls\") pod \"node-exporter-tv629\" (UID: \"25727dba-8eab-4eb5-85e2-6afa43578f4a\") " pod="openshift-monitoring/node-exporter-tv629"
Jul 09 09:35:09.791543 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.791519    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25727dba-8eab-4eb5-85e2-6afa43578f4a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tv629\" (UID: \"25727dba-8eab-4eb5-85e2-6afa43578f4a\") " pod="openshift-monitoring/node-exporter-tv629"
Jul 09 09:35:09.792161 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.792141    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47x7d\" (UniqueName: \"kubernetes.io/projected/25727dba-8eab-4eb5-85e2-6afa43578f4a-kube-api-access-47x7d\") pod \"node-exporter-tv629\" (UID: \"25727dba-8eab-4eb5-85e2-6afa43578f4a\") " pod="openshift-monitoring/node-exporter-tv629"
Jul 09 09:35:09.838606 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:09.838573    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tv629"
Jul 09 09:35:10.187853 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.187632    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/37e2f403-c4b2-42d0-8613-8755b43545b8-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-57lfj\" (UID: \"37e2f403-c4b2-42d0-8613-8755b43545b8\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-57lfj"
Jul 09 09:35:10.187853 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.187695    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ac99270f-01d2-4095-b9f6-84129e22918e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-pprmx\" (UID: \"ac99270f-01d2-4095-b9f6-84129e22918e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pprmx"
Jul 09 09:35:10.190719 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.190663    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ac99270f-01d2-4095-b9f6-84129e22918e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-pprmx\" (UID: \"ac99270f-01d2-4095-b9f6-84129e22918e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pprmx"
Jul 09 09:35:10.191068 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.191026    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/37e2f403-c4b2-42d0-8613-8755b43545b8-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-57lfj\" (UID: \"37e2f403-c4b2-42d0-8613-8755b43545b8\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-57lfj"
Jul 09 09:35:10.402780 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.402661    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pprmx"
Jul 09 09:35:10.418448 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.418246    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-57lfj"
Jul 09 09:35:10.540860 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.540826    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 09:35:10.545744 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.545720    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.554645 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.554320    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 09 09:35:10.554645 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.554338    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 09 09:35:10.554645 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.554424    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 09 09:35:10.554645 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.554578    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 09 09:35:10.554645 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.554610    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 09 09:35:10.554645 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.554641    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 09 09:35:10.555052 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.554579    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 09 09:35:10.555052 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.554773    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 09 09:35:10.555052 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.554831    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 09 09:35:10.555052 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.554896    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-plk4q\""
Jul 09 09:35:10.647454 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.647421    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 09:35:10.693217 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.692954    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.694231 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.693513    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.694231 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.693570    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/247c41df-4a8f-42d5-80c7-207e2d27dea9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.694231 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.693596    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9s92p\" (UniqueName: \"kubernetes.io/projected/247c41df-4a8f-42d5-80c7-207e2d27dea9-kube-api-access-9s92p\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.694231 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.693656    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-config-volume\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.694231 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.693722    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/247c41df-4a8f-42d5-80c7-207e2d27dea9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.694231 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.693746    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/247c41df-4a8f-42d5-80c7-207e2d27dea9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.694231 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.693793    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.694231 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.693820    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/247c41df-4a8f-42d5-80c7-207e2d27dea9-config-out\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.694231 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.693859    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.694231 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.693903    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/247c41df-4a8f-42d5-80c7-207e2d27dea9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.694231 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.693962    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-web-config\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.694231 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.694009    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.795640 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.795402    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.795640 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.795449    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.795640 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.795497    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/247c41df-4a8f-42d5-80c7-207e2d27dea9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.795640 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.795522    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9s92p\" (UniqueName: \"kubernetes.io/projected/247c41df-4a8f-42d5-80c7-207e2d27dea9-kube-api-access-9s92p\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.795993 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.795645    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-config-volume\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.795993 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.795688    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/247c41df-4a8f-42d5-80c7-207e2d27dea9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.795993 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.795708    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/247c41df-4a8f-42d5-80c7-207e2d27dea9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.795993 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.795737    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.795993 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.795760    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/247c41df-4a8f-42d5-80c7-207e2d27dea9-config-out\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.795993 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.795786    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.795993 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.795808    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/247c41df-4a8f-42d5-80c7-207e2d27dea9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.795993 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.795942    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-web-config\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.796388 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.796008    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.798440 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.798338    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/247c41df-4a8f-42d5-80c7-207e2d27dea9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.799672 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.799641    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/247c41df-4a8f-42d5-80c7-207e2d27dea9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.799672 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.799660    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/247c41df-4a8f-42d5-80c7-207e2d27dea9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.804743 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.804708    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-web-config\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.806822 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.805768    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.807521 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.807044    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-config-volume\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.809538 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.809326    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/247c41df-4a8f-42d5-80c7-207e2d27dea9-config-out\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.810180 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.810157    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.813591 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.812828    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.813591 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.813212    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.813591 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.813557    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.814115 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.814068    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9s92p\" (UniqueName: \"kubernetes.io/projected/247c41df-4a8f-42d5-80c7-207e2d27dea9-kube-api-access-9s92p\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.814653 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.814616    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/247c41df-4a8f-42d5-80c7-207e2d27dea9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.859335 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:10.859095    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:35:10.968197 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:35:10.968137    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25727dba_8eab_4eb5_85e2_6afa43578f4a.slice/crio-d1be54312a229a3ff5a6168c1d75e74483e173d7cbd168527ff3cf4cdbbf3fa6 WatchSource:0}: Error finding container d1be54312a229a3ff5a6168c1d75e74483e173d7cbd168527ff3cf4cdbbf3fa6: Status 404 returned error can't find the container with id d1be54312a229a3ff5a6168c1d75e74483e173d7cbd168527ff3cf4cdbbf3fa6
Jul 09 09:35:11.165675 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.165533    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 09:35:11.383098 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.383018    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-pprmx"]
Jul 09 09:35:11.383966 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.383928    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-57lfj"]
Jul 09 09:35:11.557554 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.557518    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"]
Jul 09 09:35:11.561567 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.561540    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"
Jul 09 09:35:11.564730 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.564530    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 09 09:35:11.564730 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.564589    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 09 09:35:11.564730 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.564602    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 09 09:35:11.564730 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.564653    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-57aeeqke0q8ae\""
Jul 09 09:35:11.564730 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.564658    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 09 09:35:11.564730 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.564606    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 09 09:35:11.565111 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.564965    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-lrjkp\""
Jul 09 09:35:11.568249 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.568228    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"]
Jul 09 09:35:11.609556 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.609525    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-scljx"
Jul 09 09:35:11.665388 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.665292    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-765fd786f7-pf2bx" event={"ID":"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe","Type":"ContainerStarted","Data":"28ab5f943e650321dbb7105439154c67a1f45b6c9964e986a300a182fbadabc2"}
Jul 09 09:35:11.667049 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.667020    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rqzsx" event={"ID":"72ab801d-5d43-4fbc-a26a-60fab96be57b","Type":"ContainerStarted","Data":"72cbe6021b880ccc320acb42b96e2876ef378f4128ad01e8f46ce3cf94eb01be"}
Jul 09 09:35:11.668855 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.668831    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57475f5bfb-lg5kw" event={"ID":"0fb00168-57af-42e0-8397-c79457b74799","Type":"ContainerStarted","Data":"ca1aee54f2f05f8d88b1400b919cb5fc9eb38193a73fa02ffdad3ccf7b798c82"}
Jul 09 09:35:11.670432 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.670411    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tv629" event={"ID":"25727dba-8eab-4eb5-85e2-6afa43578f4a","Type":"ContainerStarted","Data":"d1be54312a229a3ff5a6168c1d75e74483e173d7cbd168527ff3cf4cdbbf3fa6"}
Jul 09 09:35:11.688151 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.688095    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-765fd786f7-pf2bx" podStartSLOduration=3.853178288 podStartE2EDuration="7.688078267s" totalImagesPullingTime="3.834899979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:35:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:35:11.687613184 +0000 UTC m=+69.858771738" watchObservedRunningTime="2026-07-09 09:35:11.688078267 +0000 UTC m=+69.859236829"
Jul 09 09:35:11.704649 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.704624    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/cf8a07e1-0685-4dcc-b0f9-f3228e2438c1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5dc479b7c8-ln75k\" (UID: \"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1\") " pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"
Jul 09 09:35:11.704765 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.704655    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cf8a07e1-0685-4dcc-b0f9-f3228e2438c1-secret-grpc-tls\") pod \"thanos-querier-5dc479b7c8-ln75k\" (UID: \"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1\") " pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"
Jul 09 09:35:11.704765 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.704681    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf8a07e1-0685-4dcc-b0f9-f3228e2438c1-metrics-client-ca\") pod \"thanos-querier-5dc479b7c8-ln75k\" (UID: \"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1\") " pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"
Jul 09 09:35:11.704946 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.704832    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cf8a07e1-0685-4dcc-b0f9-f3228e2438c1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5dc479b7c8-ln75k\" (UID: \"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1\") " pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"
Jul 09 09:35:11.704946 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.704899    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hlcc\" (UniqueName: \"kubernetes.io/projected/cf8a07e1-0685-4dcc-b0f9-f3228e2438c1-kube-api-access-6hlcc\") pod \"thanos-querier-5dc479b7c8-ln75k\" (UID: \"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1\") " pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"
Jul 09 09:35:11.704946 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.704931    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/cf8a07e1-0685-4dcc-b0f9-f3228e2438c1-secret-thanos-querier-tls\") pod \"thanos-querier-5dc479b7c8-ln75k\" (UID: \"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1\") " pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"
Jul 09 09:35:11.705107 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.704972    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cf8a07e1-0685-4dcc-b0f9-f3228e2438c1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5dc479b7c8-ln75k\" (UID: \"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1\") " pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"
Jul 09 09:35:11.705107 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.705041    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/cf8a07e1-0685-4dcc-b0f9-f3228e2438c1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5dc479b7c8-ln75k\" (UID: \"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1\") " pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"
Jul 09 09:35:11.707157 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.707118    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57475f5bfb-lg5kw" podStartSLOduration=1.7620817610000001 podStartE2EDuration="5.707105583s" totalImagesPullingTime="3.945023822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:35:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:35:11.706128239 +0000 UTC m=+69.877286797" watchObservedRunningTime="2026-07-09 09:35:11.707105583 +0000 UTC m=+69.878264138"
Jul 09 09:35:11.805851 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.805810    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hlcc\" (UniqueName: \"kubernetes.io/projected/cf8a07e1-0685-4dcc-b0f9-f3228e2438c1-kube-api-access-6hlcc\") pod \"thanos-querier-5dc479b7c8-ln75k\" (UID: \"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1\") " pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"
Jul 09 09:35:11.806046 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.805876    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/cf8a07e1-0685-4dcc-b0f9-f3228e2438c1-secret-thanos-querier-tls\") pod \"thanos-querier-5dc479b7c8-ln75k\" (UID: \"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1\") " pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"
Jul 09 09:35:11.806046 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.805967    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cf8a07e1-0685-4dcc-b0f9-f3228e2438c1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5dc479b7c8-ln75k\" (UID: \"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1\") " pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"
Jul 09 09:35:11.807396 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.806198    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/cf8a07e1-0685-4dcc-b0f9-f3228e2438c1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5dc479b7c8-ln75k\" (UID: \"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1\") " pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"
Jul 09 09:35:11.807396 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.806254    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/cf8a07e1-0685-4dcc-b0f9-f3228e2438c1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5dc479b7c8-ln75k\" (UID: \"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1\") " pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"
Jul 09 09:35:11.807396 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.806302    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cf8a07e1-0685-4dcc-b0f9-f3228e2438c1-secret-grpc-tls\") pod \"thanos-querier-5dc479b7c8-ln75k\" (UID: \"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1\") " pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"
Jul 09 09:35:11.807396 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.806335    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf8a07e1-0685-4dcc-b0f9-f3228e2438c1-metrics-client-ca\") pod \"thanos-querier-5dc479b7c8-ln75k\" (UID: \"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1\") " pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"
Jul 09 09:35:11.807396 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.806576    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cf8a07e1-0685-4dcc-b0f9-f3228e2438c1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5dc479b7c8-ln75k\" (UID: \"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1\") " pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"
Jul 09 09:35:11.807396 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.807257    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf8a07e1-0685-4dcc-b0f9-f3228e2438c1-metrics-client-ca\") pod \"thanos-querier-5dc479b7c8-ln75k\" (UID: \"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1\") " pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"
Jul 09 09:35:11.809453 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.809419    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/cf8a07e1-0685-4dcc-b0f9-f3228e2438c1-secret-thanos-querier-tls\") pod \"thanos-querier-5dc479b7c8-ln75k\" (UID: \"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1\") " pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"
Jul 09 09:35:11.809453 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.809427    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cf8a07e1-0685-4dcc-b0f9-f3228e2438c1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5dc479b7c8-ln75k\" (UID: \"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1\") " pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"
Jul 09 09:35:11.810102 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.810077    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/cf8a07e1-0685-4dcc-b0f9-f3228e2438c1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5dc479b7c8-ln75k\" (UID: \"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1\") " pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"
Jul 09 09:35:11.810300 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.810277    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cf8a07e1-0685-4dcc-b0f9-f3228e2438c1-secret-grpc-tls\") pod \"thanos-querier-5dc479b7c8-ln75k\" (UID: \"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1\") " pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"
Jul 09 09:35:11.810382 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.810360    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/cf8a07e1-0685-4dcc-b0f9-f3228e2438c1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5dc479b7c8-ln75k\" (UID: \"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1\") " pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"
Jul 09 09:35:11.810608 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.810587    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cf8a07e1-0685-4dcc-b0f9-f3228e2438c1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5dc479b7c8-ln75k\" (UID: \"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1\") " pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"
Jul 09 09:35:11.821728 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.821709    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hlcc\" (UniqueName: \"kubernetes.io/projected/cf8a07e1-0685-4dcc-b0f9-f3228e2438c1-kube-api-access-6hlcc\") pod \"thanos-querier-5dc479b7c8-ln75k\" (UID: \"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1\") " pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"
Jul 09 09:35:11.873664 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:11.873639    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"
Jul 09 09:35:13.622182 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:35:13.622139    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod247c41df_4a8f_42d5_80c7_207e2d27dea9.slice/crio-1755cda988399c0681bf0a1c4dc1b9f62ae3d112a3e281aac19221328396d8eb WatchSource:0}: Error finding container 1755cda988399c0681bf0a1c4dc1b9f62ae3d112a3e281aac19221328396d8eb: Status 404 returned error can't find the container with id 1755cda988399c0681bf0a1c4dc1b9f62ae3d112a3e281aac19221328396d8eb
Jul 09 09:35:13.622949 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:35:13.622917    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37e2f403_c4b2_42d0_8613_8755b43545b8.slice/crio-dca2cbc977b118dcce43118fab40f45cd8ca36c0bca6c97055fdbb413fbe643c WatchSource:0}: Error finding container dca2cbc977b118dcce43118fab40f45cd8ca36c0bca6c97055fdbb413fbe643c: Status 404 returned error can't find the container with id dca2cbc977b118dcce43118fab40f45cd8ca36c0bca6c97055fdbb413fbe643c
Jul 09 09:35:13.624019 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:35:13.623996    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac99270f_01d2_4095_b9f6_84129e22918e.slice/crio-387bdd2c75b4342a88f8d522b1b221dd485fca0046300aa5f8137dbab991337c WatchSource:0}: Error finding container 387bdd2c75b4342a88f8d522b1b221dd485fca0046300aa5f8137dbab991337c: Status 404 returned error can't find the container with id 387bdd2c75b4342a88f8d522b1b221dd485fca0046300aa5f8137dbab991337c
Jul 09 09:35:13.678827 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:13.678793    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-57lfj" event={"ID":"37e2f403-c4b2-42d0-8613-8755b43545b8","Type":"ContainerStarted","Data":"dca2cbc977b118dcce43118fab40f45cd8ca36c0bca6c97055fdbb413fbe643c"}
Jul 09 09:35:13.680375 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:13.680350    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"247c41df-4a8f-42d5-80c7-207e2d27dea9","Type":"ContainerStarted","Data":"1755cda988399c0681bf0a1c4dc1b9f62ae3d112a3e281aac19221328396d8eb"}
Jul 09 09:35:13.681562 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:13.681527    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pprmx" event={"ID":"ac99270f-01d2-4095-b9f6-84129e22918e","Type":"ContainerStarted","Data":"387bdd2c75b4342a88f8d522b1b221dd485fca0046300aa5f8137dbab991337c"}
Jul 09 09:35:13.872750 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:13.872700    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6485c4445-bcqjm"]
Jul 09 09:35:13.876075 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:13.876055    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6485c4445-bcqjm"
Jul 09 09:35:13.878825 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:13.878804    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 09 09:35:13.880304 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:13.879729    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 09 09:35:13.880304 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:13.879825    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 09 09:35:13.880304 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:13.879842    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 09 09:35:13.880304 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:13.879966    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-abn354jsgluqm\""
Jul 09 09:35:13.880304 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:13.879967    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-q564x\""
Jul 09 09:35:13.888945 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:13.888859    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6485c4445-bcqjm"]
Jul 09 09:35:14.026856 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.026824    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4bcbdffd-45cc-4293-a061-a8d0785819b4-client-ca-bundle\") pod \"metrics-server-6485c4445-bcqjm\" (UID: \"4bcbdffd-45cc-4293-a061-a8d0785819b4\") " pod="openshift-monitoring/metrics-server-6485c4445-bcqjm"
Jul 09 09:35:14.026956 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.026916    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bcbdffd-45cc-4293-a061-a8d0785819b4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6485c4445-bcqjm\" (UID: \"4bcbdffd-45cc-4293-a061-a8d0785819b4\") " pod="openshift-monitoring/metrics-server-6485c4445-bcqjm"
Jul 09 09:35:14.027023 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.027000    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4bcbdffd-45cc-4293-a061-a8d0785819b4-secret-metrics-server-tls\") pod \"metrics-server-6485c4445-bcqjm\" (UID: \"4bcbdffd-45cc-4293-a061-a8d0785819b4\") " pod="openshift-monitoring/metrics-server-6485c4445-bcqjm"
Jul 09 09:35:14.027082 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.027032    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4bcbdffd-45cc-4293-a061-a8d0785819b4-secret-metrics-server-client-certs\") pod \"metrics-server-6485c4445-bcqjm\" (UID: \"4bcbdffd-45cc-4293-a061-a8d0785819b4\") " pod="openshift-monitoring/metrics-server-6485c4445-bcqjm"
Jul 09 09:35:14.027082 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.027070    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4bcbdffd-45cc-4293-a061-a8d0785819b4-metrics-server-audit-profiles\") pod \"metrics-server-6485c4445-bcqjm\" (UID: \"4bcbdffd-45cc-4293-a061-a8d0785819b4\") " pod="openshift-monitoring/metrics-server-6485c4445-bcqjm"
Jul 09 09:35:14.027178 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.027104    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nvkt\" (UniqueName: \"kubernetes.io/projected/4bcbdffd-45cc-4293-a061-a8d0785819b4-kube-api-access-6nvkt\") pod \"metrics-server-6485c4445-bcqjm\" (UID: \"4bcbdffd-45cc-4293-a061-a8d0785819b4\") " pod="openshift-monitoring/metrics-server-6485c4445-bcqjm"
Jul 09 09:35:14.027384 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.027358    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4bcbdffd-45cc-4293-a061-a8d0785819b4-audit-log\") pod \"metrics-server-6485c4445-bcqjm\" (UID: \"4bcbdffd-45cc-4293-a061-a8d0785819b4\") " pod="openshift-monitoring/metrics-server-6485c4445-bcqjm"
Jul 09 09:35:14.042281 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.042215    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"]
Jul 09 09:35:14.045032 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:35:14.044993    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf8a07e1_0685_4dcc_b0f9_f3228e2438c1.slice/crio-a751d3a76cafab1baead9aeddcde14b7b8ffd21cd163eab0997917f94862d537 WatchSource:0}: Error finding container a751d3a76cafab1baead9aeddcde14b7b8ffd21cd163eab0997917f94862d537: Status 404 returned error can't find the container with id a751d3a76cafab1baead9aeddcde14b7b8ffd21cd163eab0997917f94862d537
Jul 09 09:35:14.128239 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.128209    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4bcbdffd-45cc-4293-a061-a8d0785819b4-secret-metrics-server-tls\") pod \"metrics-server-6485c4445-bcqjm\" (UID: \"4bcbdffd-45cc-4293-a061-a8d0785819b4\") " pod="openshift-monitoring/metrics-server-6485c4445-bcqjm"
Jul 09 09:35:14.128367 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.128246    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4bcbdffd-45cc-4293-a061-a8d0785819b4-secret-metrics-server-client-certs\") pod \"metrics-server-6485c4445-bcqjm\" (UID: \"4bcbdffd-45cc-4293-a061-a8d0785819b4\") " pod="openshift-monitoring/metrics-server-6485c4445-bcqjm"
Jul 09 09:35:14.128367 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.128296    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4bcbdffd-45cc-4293-a061-a8d0785819b4-metrics-server-audit-profiles\") pod \"metrics-server-6485c4445-bcqjm\" (UID: \"4bcbdffd-45cc-4293-a061-a8d0785819b4\") " pod="openshift-monitoring/metrics-server-6485c4445-bcqjm"
Jul 09 09:35:14.128367 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.128318    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nvkt\" (UniqueName: \"kubernetes.io/projected/4bcbdffd-45cc-4293-a061-a8d0785819b4-kube-api-access-6nvkt\") pod \"metrics-server-6485c4445-bcqjm\" (UID: \"4bcbdffd-45cc-4293-a061-a8d0785819b4\") " pod="openshift-monitoring/metrics-server-6485c4445-bcqjm"
Jul 09 09:35:14.128367 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.128350    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4bcbdffd-45cc-4293-a061-a8d0785819b4-audit-log\") pod \"metrics-server-6485c4445-bcqjm\" (UID: \"4bcbdffd-45cc-4293-a061-a8d0785819b4\") " pod="openshift-monitoring/metrics-server-6485c4445-bcqjm"
Jul 09 09:35:14.128553 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.128425    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4bcbdffd-45cc-4293-a061-a8d0785819b4-client-ca-bundle\") pod \"metrics-server-6485c4445-bcqjm\" (UID: \"4bcbdffd-45cc-4293-a061-a8d0785819b4\") " pod="openshift-monitoring/metrics-server-6485c4445-bcqjm"
Jul 09 09:35:14.128553 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.128461    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bcbdffd-45cc-4293-a061-a8d0785819b4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6485c4445-bcqjm\" (UID: \"4bcbdffd-45cc-4293-a061-a8d0785819b4\") " pod="openshift-monitoring/metrics-server-6485c4445-bcqjm"
Jul 09 09:35:14.129172 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.129105    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4bcbdffd-45cc-4293-a061-a8d0785819b4-audit-log\") pod \"metrics-server-6485c4445-bcqjm\" (UID: \"4bcbdffd-45cc-4293-a061-a8d0785819b4\") " pod="openshift-monitoring/metrics-server-6485c4445-bcqjm"
Jul 09 09:35:14.129297 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.129219    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bcbdffd-45cc-4293-a061-a8d0785819b4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6485c4445-bcqjm\" (UID: \"4bcbdffd-45cc-4293-a061-a8d0785819b4\") " pod="openshift-monitoring/metrics-server-6485c4445-bcqjm"
Jul 09 09:35:14.129464 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.129444    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4bcbdffd-45cc-4293-a061-a8d0785819b4-metrics-server-audit-profiles\") pod \"metrics-server-6485c4445-bcqjm\" (UID: \"4bcbdffd-45cc-4293-a061-a8d0785819b4\") " pod="openshift-monitoring/metrics-server-6485c4445-bcqjm"
Jul 09 09:35:14.131177 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.131154    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4bcbdffd-45cc-4293-a061-a8d0785819b4-secret-metrics-server-tls\") pod \"metrics-server-6485c4445-bcqjm\" (UID: \"4bcbdffd-45cc-4293-a061-a8d0785819b4\") " pod="openshift-monitoring/metrics-server-6485c4445-bcqjm"
Jul 09 09:35:14.131273 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.131188    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4bcbdffd-45cc-4293-a061-a8d0785819b4-client-ca-bundle\") pod \"metrics-server-6485c4445-bcqjm\" (UID: \"4bcbdffd-45cc-4293-a061-a8d0785819b4\") " pod="openshift-monitoring/metrics-server-6485c4445-bcqjm"
Jul 09 09:35:14.132318 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.132301    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4bcbdffd-45cc-4293-a061-a8d0785819b4-secret-metrics-server-client-certs\") pod \"metrics-server-6485c4445-bcqjm\" (UID: \"4bcbdffd-45cc-4293-a061-a8d0785819b4\") " pod="openshift-monitoring/metrics-server-6485c4445-bcqjm"
Jul 09 09:35:14.136246 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.136228    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nvkt\" (UniqueName: \"kubernetes.io/projected/4bcbdffd-45cc-4293-a061-a8d0785819b4-kube-api-access-6nvkt\") pod \"metrics-server-6485c4445-bcqjm\" (UID: \"4bcbdffd-45cc-4293-a061-a8d0785819b4\") " pod="openshift-monitoring/metrics-server-6485c4445-bcqjm"
Jul 09 09:35:14.206842 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.206625    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6485c4445-bcqjm"
Jul 09 09:35:14.224639 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.224611    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-dxx4z"]
Jul 09 09:35:14.228363 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.228293    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-dxx4z"
Jul 09 09:35:14.233931 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.232242    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 09 09:35:14.233931 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.232276    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-4glcf\""
Jul 09 09:35:14.237213 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.237190    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-dxx4z"]
Jul 09 09:35:14.240786 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.239853    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-765fd786f7-pf2bx"]
Jul 09 09:35:14.271262 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.270735    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c445df65c-kmk8b"]
Jul 09 09:35:14.274302 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.274281    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c445df65c-kmk8b"
Jul 09 09:35:14.288455 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.288428    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c445df65c-kmk8b"]
Jul 09 09:35:14.331189 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.330720    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdb4w\" (UniqueName: \"kubernetes.io/projected/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-kube-api-access-pdb4w\") pod \"console-7c445df65c-kmk8b\" (UID: \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\") " pod="openshift-console/console-7c445df65c-kmk8b"
Jul 09 09:35:14.331189 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.330764    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-trusted-ca-bundle\") pod \"console-7c445df65c-kmk8b\" (UID: \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\") " pod="openshift-console/console-7c445df65c-kmk8b"
Jul 09 09:35:14.331189 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.330794    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-service-ca\") pod \"console-7c445df65c-kmk8b\" (UID: \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\") " pod="openshift-console/console-7c445df65c-kmk8b"
Jul 09 09:35:14.331189 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.330825    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-console-oauth-config\") pod \"console-7c445df65c-kmk8b\" (UID: \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\") " pod="openshift-console/console-7c445df65c-kmk8b"
Jul 09 09:35:14.331189 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.330853    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-console-config\") pod \"console-7c445df65c-kmk8b\" (UID: \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\") " pod="openshift-console/console-7c445df65c-kmk8b"
Jul 09 09:35:14.331189 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.330918    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-oauth-serving-cert\") pod \"console-7c445df65c-kmk8b\" (UID: \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\") " pod="openshift-console/console-7c445df65c-kmk8b"
Jul 09 09:35:14.331189 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.330973    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r29sn\" (UniqueName: \"kubernetes.io/projected/fa21ce94-f749-4993-9894-dfd2a9988132-kube-api-access-r29sn\") pod \"monitoring-plugin-d5dcff7cf-dxx4z\" (UID: \"fa21ce94-f749-4993-9894-dfd2a9988132\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-dxx4z"
Jul 09 09:35:14.331189 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.331023    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-console-serving-cert\") pod \"console-7c445df65c-kmk8b\" (UID: \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\") " pod="openshift-console/console-7c445df65c-kmk8b"
Jul 09 09:35:14.331189 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.331058    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fa21ce94-f749-4993-9894-dfd2a9988132-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-dxx4z\" (UID: \"fa21ce94-f749-4993-9894-dfd2a9988132\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-dxx4z"
Jul 09 09:35:14.378546 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.378402    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6485c4445-bcqjm"]
Jul 09 09:35:14.382532 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:35:14.382502    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bcbdffd_45cc_4293_a061_a8d0785819b4.slice/crio-de1c8373cec24e752db937be82d00d119541659aebd7237fa2308f99dbb9ada2 WatchSource:0}: Error finding container de1c8373cec24e752db937be82d00d119541659aebd7237fa2308f99dbb9ada2: Status 404 returned error can't find the container with id de1c8373cec24e752db937be82d00d119541659aebd7237fa2308f99dbb9ada2
Jul 09 09:35:14.431596 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.431571    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r29sn\" (UniqueName: \"kubernetes.io/projected/fa21ce94-f749-4993-9894-dfd2a9988132-kube-api-access-r29sn\") pod \"monitoring-plugin-d5dcff7cf-dxx4z\" (UID: \"fa21ce94-f749-4993-9894-dfd2a9988132\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-dxx4z"
Jul 09 09:35:14.431737 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.431624    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-console-serving-cert\") pod \"console-7c445df65c-kmk8b\" (UID: \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\") " pod="openshift-console/console-7c445df65c-kmk8b"
Jul 09 09:35:14.431737 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.431656    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fa21ce94-f749-4993-9894-dfd2a9988132-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-dxx4z\" (UID: \"fa21ce94-f749-4993-9894-dfd2a9988132\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-dxx4z"
Jul 09 09:35:14.431737 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.431684    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdb4w\" (UniqueName: \"kubernetes.io/projected/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-kube-api-access-pdb4w\") pod \"console-7c445df65c-kmk8b\" (UID: \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\") " pod="openshift-console/console-7c445df65c-kmk8b"
Jul 09 09:35:14.431737 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.431708    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-trusted-ca-bundle\") pod \"console-7c445df65c-kmk8b\" (UID: \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\") " pod="openshift-console/console-7c445df65c-kmk8b"
Jul 09 09:35:14.431737 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.431733    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-service-ca\") pod \"console-7c445df65c-kmk8b\" (UID: \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\") " pod="openshift-console/console-7c445df65c-kmk8b"
Jul 09 09:35:14.432042 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.431904    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-console-oauth-config\") pod \"console-7c445df65c-kmk8b\" (UID: \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\") " pod="openshift-console/console-7c445df65c-kmk8b"
Jul 09 09:35:14.432042 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.431946    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-console-config\") pod \"console-7c445df65c-kmk8b\" (UID: \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\") " pod="openshift-console/console-7c445df65c-kmk8b"
Jul 09 09:35:14.432042 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.432006    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-oauth-serving-cert\") pod \"console-7c445df65c-kmk8b\" (UID: \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\") " pod="openshift-console/console-7c445df65c-kmk8b"
Jul 09 09:35:14.432565 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.432513    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-service-ca\") pod \"console-7c445df65c-kmk8b\" (UID: \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\") " pod="openshift-console/console-7c445df65c-kmk8b"
Jul 09 09:35:14.432690 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.432669    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-console-config\") pod \"console-7c445df65c-kmk8b\" (UID: \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\") " pod="openshift-console/console-7c445df65c-kmk8b"
Jul 09 09:35:14.432733 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.432717    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-oauth-serving-cert\") pod \"console-7c445df65c-kmk8b\" (UID: \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\") " pod="openshift-console/console-7c445df65c-kmk8b"
Jul 09 09:35:14.432773 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.432717    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-trusted-ca-bundle\") pod \"console-7c445df65c-kmk8b\" (UID: \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\") " pod="openshift-console/console-7c445df65c-kmk8b"
Jul 09 09:35:14.434208 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.434188    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-console-oauth-config\") pod \"console-7c445df65c-kmk8b\" (UID: \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\") " pod="openshift-console/console-7c445df65c-kmk8b"
Jul 09 09:35:14.434557 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.434539    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fa21ce94-f749-4993-9894-dfd2a9988132-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-dxx4z\" (UID: \"fa21ce94-f749-4993-9894-dfd2a9988132\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-dxx4z"
Jul 09 09:35:14.434596 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.434548    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-console-serving-cert\") pod \"console-7c445df65c-kmk8b\" (UID: \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\") " pod="openshift-console/console-7c445df65c-kmk8b"
Jul 09 09:35:14.441516 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.441496    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r29sn\" (UniqueName: \"kubernetes.io/projected/fa21ce94-f749-4993-9894-dfd2a9988132-kube-api-access-r29sn\") pod \"monitoring-plugin-d5dcff7cf-dxx4z\" (UID: \"fa21ce94-f749-4993-9894-dfd2a9988132\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-dxx4z"
Jul 09 09:35:14.441587 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.441553    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdb4w\" (UniqueName: \"kubernetes.io/projected/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-kube-api-access-pdb4w\") pod \"console-7c445df65c-kmk8b\" (UID: \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\") " pod="openshift-console/console-7c445df65c-kmk8b"
Jul 09 09:35:14.547500 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.547471    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-dxx4z"
Jul 09 09:35:14.586979 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.586685    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c445df65c-kmk8b"
Jul 09 09:35:14.691337 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.691295    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k" event={"ID":"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1","Type":"ContainerStarted","Data":"a751d3a76cafab1baead9aeddcde14b7b8ffd21cd163eab0997917f94862d537"}
Jul 09 09:35:14.693588 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.693515    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ccqx8" event={"ID":"282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee","Type":"ContainerStarted","Data":"711dc644fbd088df6001a1bc22d2326258138efc2b2447663a671ce3f8888325"}
Jul 09 09:35:14.696727 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.696357    2549 generic.go:356] "Generic (PLEG): container finished" podID="25727dba-8eab-4eb5-85e2-6afa43578f4a" containerID="d51af2ab8a25e5976de4f596b72fb88e52a4de9de0c26a8feef1868833b6ac4a" exitCode=0
Jul 09 09:35:14.696727 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.696450    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tv629" event={"ID":"25727dba-8eab-4eb5-85e2-6afa43578f4a","Type":"ContainerDied","Data":"d51af2ab8a25e5976de4f596b72fb88e52a4de9de0c26a8feef1868833b6ac4a"}
Jul 09 09:35:14.699753 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.699657    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pprmx" event={"ID":"ac99270f-01d2-4095-b9f6-84129e22918e","Type":"ContainerStarted","Data":"39c37fc20a9ec6bc96bad64e6257060afdaa176c1b19c2bb54d4390d3cf7ee2f"}
Jul 09 09:35:14.699753 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.699712    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pprmx" event={"ID":"ac99270f-01d2-4095-b9f6-84129e22918e","Type":"ContainerStarted","Data":"4cd2059c63a4da58a4b0154356e76546550bfbe306a659edd6a64466024308df"}
Jul 09 09:35:14.702517 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.702468    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-dxx4z"]
Jul 09 09:35:14.704571 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.704067    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ljjkv" event={"ID":"2ab646db-3779-4dbf-a300-a7b166d929ab","Type":"ContainerStarted","Data":"59370c3b03ff6d3c491e0f4392cc85f3f02772619199051a2ffbe69855d746b2"}
Jul 09 09:35:14.704571 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.704126    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:35:14.706520 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.706498    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rqzsx" event={"ID":"72ab801d-5d43-4fbc-a26a-60fab96be57b","Type":"ContainerStarted","Data":"a0bce4135a86999d299161b43d324b6fed2dc4658d042be3e68e458e2e53f2d2"}
Jul 09 09:35:14.707997 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.707948    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6485c4445-bcqjm" event={"ID":"4bcbdffd-45cc-4293-a061-a8d0785819b4","Type":"ContainerStarted","Data":"de1c8373cec24e752db937be82d00d119541659aebd7237fa2308f99dbb9ada2"}
Jul 09 09:35:14.714250 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.714207    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ccqx8" podStartSLOduration=67.284783582 podStartE2EDuration="1m12.714191342s" totalImagesPullingTime="5.42940776s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:34:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:35:14.712546377 +0000 UTC m=+72.883704960" watchObservedRunningTime="2026-07-09 09:35:14.714191342 +0000 UTC m=+72.885349897"
Jul 09 09:35:14.733549 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.733157    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rqzsx" podStartSLOduration=70.03612389 podStartE2EDuration="1m12.733140701s" totalImagesPullingTime="2.697016811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:34:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:35:14.731129564 +0000 UTC m=+72.902288120" watchObservedRunningTime="2026-07-09 09:35:14.733140701 +0000 UTC m=+72.904299260"
Jul 09 09:35:14.771505 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:14.771459    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-ljjkv" podStartSLOduration=67.468399145 podStartE2EDuration="1m12.771441683s" totalImagesPullingTime="5.303042538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:34:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:35:14.769960076 +0000 UTC m=+72.941118635" watchObservedRunningTime="2026-07-09 09:35:14.771441683 +0000 UTC m=+72.942600240"
Jul 09 09:35:14.897098 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:35:14.897018    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa21ce94_f749_4993_9894_dfd2a9988132.slice/crio-b81186fe6c88a165544e2de1ed2adde5e3514d9136bd03de62f304286405f557 WatchSource:0}: Error finding container b81186fe6c88a165544e2de1ed2adde5e3514d9136bd03de62f304286405f557: Status 404 returned error can't find the container with id b81186fe6c88a165544e2de1ed2adde5e3514d9136bd03de62f304286405f557
Jul 09 09:35:15.503377 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.503296    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c445df65c-kmk8b"]
Jul 09 09:35:15.720450 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.719565    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tv629" event={"ID":"25727dba-8eab-4eb5-85e2-6afa43578f4a","Type":"ContainerStarted","Data":"1c9515909e307559fdfcb75d1c436a0cf5cdfa8856fcf71177c25c647aaa8cc0"}
Jul 09 09:35:15.720450 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.719627    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tv629" event={"ID":"25727dba-8eab-4eb5-85e2-6afa43578f4a","Type":"ContainerStarted","Data":"a693b56f1d010a77c40eb5bee0d8c02e07e8a52d9e8775d8c24f2d38f54dba88"}
Jul 09 09:35:15.741219 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.740470    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-57lfj" event={"ID":"37e2f403-c4b2-42d0-8613-8755b43545b8","Type":"ContainerStarted","Data":"d6c99174b086fe29812aa1b1e0d268060caed94cce52df5cc99a0fe4e98c0016"}
Jul 09 09:35:15.741219 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.740567    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-57lfj" event={"ID":"37e2f403-c4b2-42d0-8613-8755b43545b8","Type":"ContainerStarted","Data":"ad0422c08944fef11166eacf5bab09943fc237287cbcebddfd03427236232a0b"}
Jul 09 09:35:15.741219 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.740594    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-57lfj" event={"ID":"37e2f403-c4b2-42d0-8613-8755b43545b8","Type":"ContainerStarted","Data":"3c54993bfabea4822a388a581a8f20a2357608c568ec2fcb80718f61b349a1c7"}
Jul 09 09:35:15.746530 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.746486    2549 generic.go:356] "Generic (PLEG): container finished" podID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerID="48519b062f7a608b6ba0d6c9eab648c15e881150a1ae50857a88f96ccc975c0e" exitCode=0
Jul 09 09:35:15.746896 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.746772    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"247c41df-4a8f-42d5-80c7-207e2d27dea9","Type":"ContainerDied","Data":"48519b062f7a608b6ba0d6c9eab648c15e881150a1ae50857a88f96ccc975c0e"}
Jul 09 09:35:15.752636 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.752051    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c445df65c-kmk8b" event={"ID":"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef","Type":"ContainerStarted","Data":"fa6e0dc90cc051b787346f06926fa223deb435e6fce60d1227d92c3ca9563e01"}
Jul 09 09:35:15.752636 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.752098    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c445df65c-kmk8b" event={"ID":"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef","Type":"ContainerStarted","Data":"ecdff7bf2b46728b05f563e2ba72dd17efe29eb5edb6e37e1002cb55d081ad34"}
Jul 09 09:35:15.752884 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.752825    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-tv629" podStartSLOduration=3.715752008 podStartE2EDuration="6.752805499s" totalImagesPullingTime="2.965438092s" totalInitContainerRuntime="71.615399ms" isStatefulPod=true podCreationTimestamp="2026-07-09 09:35:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:35:15.744709757 +0000 UTC m=+73.915868313" watchObservedRunningTime="2026-07-09 09:35:15.752805499 +0000 UTC m=+73.923964051"
Jul 09 09:35:15.757404 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.757370    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-dxx4z" event={"ID":"fa21ce94-f749-4993-9894-dfd2a9988132","Type":"ContainerStarted","Data":"b81186fe6c88a165544e2de1ed2adde5e3514d9136bd03de62f304286405f557"}
Jul 09 09:35:15.812294 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.810847    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-57lfj" podStartSLOduration=5.149551566 podStartE2EDuration="6.810826513s" totalImagesPullingTime="1.661274947s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:35:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:35:15.769963167 +0000 UTC m=+73.941121725" watchObservedRunningTime="2026-07-09 09:35:15.810826513 +0000 UTC m=+73.981985069"
Jul 09 09:35:15.812294 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.812120    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c445df65c-kmk8b" podStartSLOduration=1.812103473 podStartE2EDuration="1.812103473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:35:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:35:15.805593765 +0000 UTC m=+73.976752336" watchObservedRunningTime="2026-07-09 09:35:15.812103473 +0000 UTC m=+73.983262031"
Jul 09 09:35:15.824207 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.823778    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 09:35:15.829945 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.829921    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:15.835735 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.834978    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 09 09:35:15.837647 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.836564    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 09 09:35:15.837647 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.836702    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 09 09:35:15.837647 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.836753    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 09 09:35:15.837647 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.836912    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 09 09:35:15.837647 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.837022    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 09 09:35:15.837647 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.837116    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 09 09:35:15.837647 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.836570    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 09 09:35:15.837647 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.837463    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 09 09:35:15.845986 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.845626    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 09 09:35:15.846515 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.846098    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-kzbcv\""
Jul 09 09:35:15.846515 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.846315    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 09 09:35:15.846779 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.846698    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 09 09:35:15.847027 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.846937    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 09 09:35:15.852671 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.847207    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6j816k2kt5o32\""
Jul 09 09:35:15.857666 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.856630    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 09 09:35:15.871928 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.871907    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 09:35:15.946434 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.945573    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:15.946434 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.945617    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:15.946434 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.945649    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e611bfba-e12e-416c-bf0a-42055b9ed56e-config-out\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:15.946434 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.945677    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:15.946434 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.945706    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:15.946434 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.945737    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:15.946434 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.945763    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:15.946434 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.945787    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-config\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:15.946434 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.945812    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:15.946434 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.945881    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:15.946434 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.945909    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:15.946434 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.945936    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:15.946434 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.945968    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:15.946434 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.945995    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:15.946434 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.946025    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:15.946434 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.946048    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:15.947367 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.946079    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2tc7\" (UniqueName: \"kubernetes.io/projected/e611bfba-e12e-416c-bf0a-42055b9ed56e-kube-api-access-z2tc7\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:15.947367 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.946133    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:15.947367 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.946163    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-web-config\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:15.947367 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:15.946205    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e611bfba-e12e-416c-bf0a-42055b9ed56e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.048890 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.047493    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.048890 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.047549    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.048890 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.047577    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.048890 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.047605    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-config\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.048890 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.047631    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.048890 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.047681    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.048890 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.047709    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.048890 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.047737    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.048890 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.047765    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.048890 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.047788    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.048890 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.047818    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.048890 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.047841    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.048890 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.048633    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.052505 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.050844    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.052505 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.051439    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.053374 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.052700    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2tc7\" (UniqueName: \"kubernetes.io/projected/e611bfba-e12e-416c-bf0a-42055b9ed56e-kube-api-access-z2tc7\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.053374 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.052771    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.053374 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.052798    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-web-config\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.053374 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.052837    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e611bfba-e12e-416c-bf0a-42055b9ed56e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.053374 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.052921    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.053374 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.052942    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.053374 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.052969    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e611bfba-e12e-416c-bf0a-42055b9ed56e-config-out\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.053374 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.052996    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.055954 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.054401    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.058062 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.057592    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.058062 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.057656    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.058196 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.058066    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.060351 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.060304    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.065148 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.065125    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e611bfba-e12e-416c-bf0a-42055b9ed56e-config-out\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.066506 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.066041    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.066506 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.066081    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.066506 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.066248    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.066506 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.066382    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.066506 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.066465    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.066506 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.066502    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e611bfba-e12e-416c-bf0a-42055b9ed56e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.066856 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.066716    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.066856 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.066805    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-config\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.067024 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.067003    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.067495 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.067461    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-web-config\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.069154 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.069129    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2tc7\" (UniqueName: \"kubernetes.io/projected/e611bfba-e12e-416c-bf0a-42055b9ed56e-kube-api-access-z2tc7\") pod \"prometheus-k8s-0\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.163096 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.162787    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:16.749210 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.749175    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57475f5bfb-lg5kw"
Jul 09 09:35:16.749688 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.749281    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57475f5bfb-lg5kw"
Jul 09 09:35:16.755451 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.755431    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57475f5bfb-lg5kw"
Jul 09 09:35:16.770352 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.770331    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57475f5bfb-lg5kw"
Jul 09 09:35:16.989032 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:16.988973    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-765fd786f7-pf2bx"
Jul 09 09:35:17.732298 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:17.732268    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 09:35:17.734777 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:35:17.734743    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode611bfba_e12e_416c_bf0a_42055b9ed56e.slice/crio-58d3bdab5f7647daa562b6817b90391fb17802458c6988792789101f7d8577f0 WatchSource:0}: Error finding container 58d3bdab5f7647daa562b6817b90391fb17802458c6988792789101f7d8577f0: Status 404 returned error can't find the container with id 58d3bdab5f7647daa562b6817b90391fb17802458c6988792789101f7d8577f0
Jul 09 09:35:17.772477 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:17.772007    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6485c4445-bcqjm" event={"ID":"4bcbdffd-45cc-4293-a061-a8d0785819b4","Type":"ContainerStarted","Data":"544ff5c94b0ade1c4a64f2fc594ea8d160ff8f4fc33826afc24fc58285a6a3ab"}
Jul 09 09:35:17.774479 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:17.774452    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k" event={"ID":"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1","Type":"ContainerStarted","Data":"e38454d5d523d22f5ab4e9791b8e7069bdeecc9332fffb1da21f14f2154782b7"}
Jul 09 09:35:17.776574 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:17.776549    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e611bfba-e12e-416c-bf0a-42055b9ed56e","Type":"ContainerStarted","Data":"58d3bdab5f7647daa562b6817b90391fb17802458c6988792789101f7d8577f0"}
Jul 09 09:35:17.778887 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:17.778840    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pprmx" event={"ID":"ac99270f-01d2-4095-b9f6-84129e22918e","Type":"ContainerStarted","Data":"108a4151ed82c91c3445614f0ed7d4f36c2427ceda6a79ea50e5bfd0f679dfe6"}
Jul 09 09:35:17.780786 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:17.780764    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-dxx4z" event={"ID":"fa21ce94-f749-4993-9894-dfd2a9988132","Type":"ContainerStarted","Data":"a9d9436c781a96ad8c3c77e1bb017667c649ab57479b4f003a73fc63afc107bb"}
Jul 09 09:35:17.781340 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:17.781293    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-dxx4z"
Jul 09 09:35:17.788914 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:17.788897    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-dxx4z"
Jul 09 09:35:17.809160 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:17.809111    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6485c4445-bcqjm" podStartSLOduration=1.614825509 podStartE2EDuration="4.809088312s" totalImagesPullingTime="3.194262803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:35:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:35:17.789928074 +0000 UTC m=+75.961086631" watchObservedRunningTime="2026-07-09 09:35:17.809088312 +0000 UTC m=+75.980246869"
Jul 09 09:35:17.830812 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:17.830050    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-dxx4z" podStartSLOduration=1.105263876 podStartE2EDuration="3.830029884s" totalImagesPullingTime="2.724766008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:35:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:35:17.810449666 +0000 UTC m=+75.981608223" watchObservedRunningTime="2026-07-09 09:35:17.830029884 +0000 UTC m=+76.001188440"
Jul 09 09:35:18.789049 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:18.788992    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k" event={"ID":"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1","Type":"ContainerStarted","Data":"8f66931f29bf162e2902bfac6b481daa4f82bad6449436eeb9489a7f651e8eef"}
Jul 09 09:35:18.789500 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:18.789129    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k" event={"ID":"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1","Type":"ContainerStarted","Data":"2e6853713fa8cc9efdea70197bc4a754be9c92ee40f3efcf8122df3bbed5d9fc"}
Jul 09 09:35:18.790894 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:18.790848    2549 generic.go:356] "Generic (PLEG): container finished" podID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerID="2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c" exitCode=0
Jul 09 09:35:18.791009 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:18.790938    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e611bfba-e12e-416c-bf0a-42055b9ed56e","Type":"ContainerDied","Data":"2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c"}
Jul 09 09:35:18.829532 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:18.829473    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pprmx" podStartSLOduration=6.309302839 podStartE2EDuration="9.829428836s" totalImagesPullingTime="3.520125997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:35:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:35:17.830963439 +0000 UTC m=+76.002121996" watchObservedRunningTime="2026-07-09 09:35:18.829428836 +0000 UTC m=+77.000587392"
Jul 09 09:35:19.799087 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:19.799048    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k" event={"ID":"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1","Type":"ContainerStarted","Data":"cb0de795839b89a4be19a0bc398d0242df7564f95172d23959da5880eec923ac"}
Jul 09 09:35:19.799540 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:19.799131    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k" event={"ID":"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1","Type":"ContainerStarted","Data":"4744eb83afb2328eff54a607acfb8ee8ceea720b29d7552a18526fe1e6767431"}
Jul 09 09:35:19.799540 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:19.799175    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k" event={"ID":"cf8a07e1-0685-4dcc-b0f9-f3228e2438c1","Type":"ContainerStarted","Data":"583fead7f3f4e7e8be2ab019e790f57f6edc5db98a0be959938d250223a67d51"}
Jul 09 09:35:19.799540 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:19.799224    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"
Jul 09 09:35:19.802914 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:19.802882    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"247c41df-4a8f-42d5-80c7-207e2d27dea9","Type":"ContainerStarted","Data":"d7bae89e3430f29a1b8aa78d669b2cea6010754f749766c25e1a0d71cba21c23"}
Jul 09 09:35:19.803054 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:19.802960    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"247c41df-4a8f-42d5-80c7-207e2d27dea9","Type":"ContainerStarted","Data":"d9a2852995803729ffe185356960db3ac22b7fa47185cb85e4efaf0c0a4d26d5"}
Jul 09 09:35:19.803054 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:19.803004    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"247c41df-4a8f-42d5-80c7-207e2d27dea9","Type":"ContainerStarted","Data":"237e83b92fd9840c11df0cfb25b145c1625bde3a832068abf040f7818c7f7cc1"}
Jul 09 09:35:19.803054 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:19.803038    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"247c41df-4a8f-42d5-80c7-207e2d27dea9","Type":"ContainerStarted","Data":"abe4d96b8a959b907b5f511285f22fc519be2821f240d70fb95a19d0fc950456"}
Jul 09 09:35:19.803192 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:19.803067    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"247c41df-4a8f-42d5-80c7-207e2d27dea9","Type":"ContainerStarted","Data":"741cc0b6b1b5567a5115f43894b2955196ad9110ad427722a2325d3cdbd3daaf"}
Jul 09 09:35:19.803192 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:19.803103    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"247c41df-4a8f-42d5-80c7-207e2d27dea9","Type":"ContainerStarted","Data":"63ac463ed80afbd317c8a4ebbecdbd33c8523ae98da70527e6e5a71f6102ceb8"}
Jul 09 09:35:19.831260 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:19.831205    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k" podStartSLOduration=4.131178666 podStartE2EDuration="8.831185087s" totalImagesPullingTime="4.700006421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:35:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:35:19.827819341 +0000 UTC m=+77.998977897" watchObservedRunningTime="2026-07-09 09:35:19.831185087 +0000 UTC m=+78.002343644"
Jul 09 09:35:19.855948 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:19.855897    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.856359424 podStartE2EDuration="9.855855712s" totalImagesPullingTime="4.956299733s" totalInitContainerRuntime="43.196555ms" isStatefulPod=false podCreationTimestamp="2026-07-09 09:35:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:35:19.855032998 +0000 UTC m=+78.026191557" watchObservedRunningTime="2026-07-09 09:35:19.855855712 +0000 UTC m=+78.027014259"
Jul 09 09:35:22.827846 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:22.827812    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e611bfba-e12e-416c-bf0a-42055b9ed56e","Type":"ContainerStarted","Data":"bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5"}
Jul 09 09:35:22.828215 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:22.827924    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e611bfba-e12e-416c-bf0a-42055b9ed56e","Type":"ContainerStarted","Data":"588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50"}
Jul 09 09:35:22.828215 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:22.827956    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e611bfba-e12e-416c-bf0a-42055b9ed56e","Type":"ContainerStarted","Data":"fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda"}
Jul 09 09:35:22.828215 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:22.827986    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e611bfba-e12e-416c-bf0a-42055b9ed56e","Type":"ContainerStarted","Data":"51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a"}
Jul 09 09:35:22.828215 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:22.828008    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e611bfba-e12e-416c-bf0a-42055b9ed56e","Type":"ContainerStarted","Data":"d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769"}
Jul 09 09:35:22.828215 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:22.828032    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e611bfba-e12e-416c-bf0a-42055b9ed56e","Type":"ContainerStarted","Data":"a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5"}
Jul 09 09:35:22.868902 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:22.868832    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.61006898 podStartE2EDuration="7.868816811s" totalImagesPullingTime="3.234019024s" totalInitContainerRuntime="24.728807ms" isStatefulPod=false podCreationTimestamp="2026-07-09 09:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:35:22.866277287 +0000 UTC m=+81.037435843" watchObservedRunningTime="2026-07-09 09:35:22.868816811 +0000 UTC m=+81.039975367"
Jul 09 09:35:24.587562 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:24.587451    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c445df65c-kmk8b"
Jul 09 09:35:24.587562 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:24.587561    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c445df65c-kmk8b"
Jul 09 09:35:24.592243 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:24.592224    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c445df65c-kmk8b"
Jul 09 09:35:24.839228 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:24.839163    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c445df65c-kmk8b"
Jul 09 09:35:24.890170 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:24.890142    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57475f5bfb-lg5kw"]
Jul 09 09:35:25.817518 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:25.817488    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5dc479b7c8-ln75k"
Jul 09 09:35:26.163340 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:26.163263    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:35:34.207650 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:34.207617    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6485c4445-bcqjm"
Jul 09 09:35:34.208099 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:34.207708    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6485c4445-bcqjm"
Jul 09 09:35:39.279136 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:39.279063    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-765fd786f7-pf2bx" podUID="69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe" containerName="console" containerID="cri-o://28ab5f943e650321dbb7105439154c67a1f45b6c9964e986a300a182fbadabc2" gracePeriod=15
Jul 09 09:35:39.516891 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:39.516852    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-765fd786f7-pf2bx"
Jul 09 09:35:39.557937 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:39.557882    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-console-oauth-config\") pod \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\" (UID: \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\") "
Jul 09 09:35:39.557937 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:39.557924    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-console-serving-cert\") pod \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\" (UID: \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\") "
Jul 09 09:35:39.558090 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:39.557975    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t96nz\" (UniqueName: \"kubernetes.io/projected/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-kube-api-access-t96nz\") pod \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\" (UID: \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\") "
Jul 09 09:35:39.558145 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:39.558101    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-console-config\") pod \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\" (UID: \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\") "
Jul 09 09:35:39.558197 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:39.558173    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-service-ca\") pod \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\" (UID: \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\") "
Jul 09 09:35:39.558270 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:39.558249    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-oauth-serving-cert\") pod \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\" (UID: \"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe\") "
Jul 09 09:35:39.558599 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:39.558459    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-console-config" (OuterVolumeSpecName: "console-config") pod "69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe" (UID: "69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:35:39.558599 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:39.558571    2549 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-console-config\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:35:39.558774 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:39.558614    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-service-ca" (OuterVolumeSpecName: "service-ca") pod "69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe" (UID: "69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:35:39.558774 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:39.558670    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe" (UID: "69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:35:39.560206 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:39.560180    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-kube-api-access-t96nz" (OuterVolumeSpecName: "kube-api-access-t96nz") pod "69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe" (UID: "69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe"). InnerVolumeSpecName "kube-api-access-t96nz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:35:39.560389 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:39.560361    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe" (UID: "69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:35:39.560469 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:39.560416    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe" (UID: "69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:35:39.659968 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:39.659935    2549 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-service-ca\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:35:39.659968 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:39.659968    2549 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-oauth-serving-cert\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:35:39.659968 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:39.659977    2549 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-console-oauth-config\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:35:39.660158 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:39.659986    2549 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-console-serving-cert\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:35:39.660158 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:39.659994    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t96nz\" (UniqueName: \"kubernetes.io/projected/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe-kube-api-access-t96nz\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:35:39.901779 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:39.901692    2549 generic.go:356] "Generic (PLEG): container finished" podID="69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe" containerID="28ab5f943e650321dbb7105439154c67a1f45b6c9964e986a300a182fbadabc2" exitCode=0
Jul 09 09:35:39.901779 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:39.901739    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-765fd786f7-pf2bx"
Jul 09 09:35:39.901779 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:39.901768    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-765fd786f7-pf2bx" event={"ID":"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe","Type":"ContainerDied","Data":"28ab5f943e650321dbb7105439154c67a1f45b6c9964e986a300a182fbadabc2"}
Jul 09 09:35:39.902068 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:39.901833    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-765fd786f7-pf2bx" event={"ID":"69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe","Type":"ContainerDied","Data":"bd70623ccc6f753524dc49725a9301f904fdcf2a99491fd710260d276c8ff0b8"}
Jul 09 09:35:39.902068 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:39.901859    2549 scope.go:117] "RemoveContainer" containerID="28ab5f943e650321dbb7105439154c67a1f45b6c9964e986a300a182fbadabc2"
Jul 09 09:35:39.910483 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:39.910465    2549 scope.go:117] "RemoveContainer" containerID="28ab5f943e650321dbb7105439154c67a1f45b6c9964e986a300a182fbadabc2"
Jul 09 09:35:39.910717 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:35:39.910700    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28ab5f943e650321dbb7105439154c67a1f45b6c9964e986a300a182fbadabc2\": container with ID starting with 28ab5f943e650321dbb7105439154c67a1f45b6c9964e986a300a182fbadabc2 not found: ID does not exist" containerID="28ab5f943e650321dbb7105439154c67a1f45b6c9964e986a300a182fbadabc2"
Jul 09 09:35:39.910754 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:39.910724    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28ab5f943e650321dbb7105439154c67a1f45b6c9964e986a300a182fbadabc2"} err="failed to get container status \"28ab5f943e650321dbb7105439154c67a1f45b6c9964e986a300a182fbadabc2\": rpc error: code = NotFound desc = could not find container \"28ab5f943e650321dbb7105439154c67a1f45b6c9964e986a300a182fbadabc2\": container with ID starting with 28ab5f943e650321dbb7105439154c67a1f45b6c9964e986a300a182fbadabc2 not found: ID does not exist"
Jul 09 09:35:39.925542 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:39.925522    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-765fd786f7-pf2bx"]
Jul 09 09:35:39.936158 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:39.936138    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-765fd786f7-pf2bx"]
Jul 09 09:35:40.385422 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:40.385385    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe" path="/var/lib/kubelet/pods/69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe/volumes"
Jul 09 09:35:45.761003 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:45.760974    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-ljjkv"
Jul 09 09:35:49.917832 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:49.917777    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-57475f5bfb-lg5kw" podUID="0fb00168-57af-42e0-8397-c79457b74799" containerName="console" containerID="cri-o://ca1aee54f2f05f8d88b1400b919cb5fc9eb38193a73fa02ffdad3ccf7b798c82" gracePeriod=15
Jul 09 09:35:50.199363 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.199340    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57475f5bfb-lg5kw"
Jul 09 09:35:50.252821 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.252791    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0fb00168-57af-42e0-8397-c79457b74799-service-ca\") pod \"0fb00168-57af-42e0-8397-c79457b74799\" (UID: \"0fb00168-57af-42e0-8397-c79457b74799\") "
Jul 09 09:35:50.252974 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.252835    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bgjm\" (UniqueName: \"kubernetes.io/projected/0fb00168-57af-42e0-8397-c79457b74799-kube-api-access-4bgjm\") pod \"0fb00168-57af-42e0-8397-c79457b74799\" (UID: \"0fb00168-57af-42e0-8397-c79457b74799\") "
Jul 09 09:35:50.252974 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.252949    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0fb00168-57af-42e0-8397-c79457b74799-trusted-ca-bundle\") pod \"0fb00168-57af-42e0-8397-c79457b74799\" (UID: \"0fb00168-57af-42e0-8397-c79457b74799\") "
Jul 09 09:35:50.253100 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.252983    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0fb00168-57af-42e0-8397-c79457b74799-console-oauth-config\") pod \"0fb00168-57af-42e0-8397-c79457b74799\" (UID: \"0fb00168-57af-42e0-8397-c79457b74799\") "
Jul 09 09:35:50.253100 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.253000    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0fb00168-57af-42e0-8397-c79457b74799-console-serving-cert\") pod \"0fb00168-57af-42e0-8397-c79457b74799\" (UID: \"0fb00168-57af-42e0-8397-c79457b74799\") "
Jul 09 09:35:50.253100 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.253058    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0fb00168-57af-42e0-8397-c79457b74799-oauth-serving-cert\") pod \"0fb00168-57af-42e0-8397-c79457b74799\" (UID: \"0fb00168-57af-42e0-8397-c79457b74799\") "
Jul 09 09:35:50.253100 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.253086    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0fb00168-57af-42e0-8397-c79457b74799-console-config\") pod \"0fb00168-57af-42e0-8397-c79457b74799\" (UID: \"0fb00168-57af-42e0-8397-c79457b74799\") "
Jul 09 09:35:50.253304 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.253133    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0fb00168-57af-42e0-8397-c79457b74799-service-ca" (OuterVolumeSpecName: "service-ca") pod "0fb00168-57af-42e0-8397-c79457b74799" (UID: "0fb00168-57af-42e0-8397-c79457b74799"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:35:50.253403 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.253383    2549 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0fb00168-57af-42e0-8397-c79457b74799-service-ca\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:35:50.253465 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.253399    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0fb00168-57af-42e0-8397-c79457b74799-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0fb00168-57af-42e0-8397-c79457b74799" (UID: "0fb00168-57af-42e0-8397-c79457b74799"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:35:50.253553 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.253449    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0fb00168-57af-42e0-8397-c79457b74799-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0fb00168-57af-42e0-8397-c79457b74799" (UID: "0fb00168-57af-42e0-8397-c79457b74799"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:35:50.253620 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.253604    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0fb00168-57af-42e0-8397-c79457b74799-console-config" (OuterVolumeSpecName: "console-config") pod "0fb00168-57af-42e0-8397-c79457b74799" (UID: "0fb00168-57af-42e0-8397-c79457b74799"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:35:50.255200 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.255176    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fb00168-57af-42e0-8397-c79457b74799-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0fb00168-57af-42e0-8397-c79457b74799" (UID: "0fb00168-57af-42e0-8397-c79457b74799"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:35:50.255604 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.255581    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fb00168-57af-42e0-8397-c79457b74799-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0fb00168-57af-42e0-8397-c79457b74799" (UID: "0fb00168-57af-42e0-8397-c79457b74799"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:35:50.255664 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.255641    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fb00168-57af-42e0-8397-c79457b74799-kube-api-access-4bgjm" (OuterVolumeSpecName: "kube-api-access-4bgjm") pod "0fb00168-57af-42e0-8397-c79457b74799" (UID: "0fb00168-57af-42e0-8397-c79457b74799"). InnerVolumeSpecName "kube-api-access-4bgjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:35:50.354414 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.354383    2549 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0fb00168-57af-42e0-8397-c79457b74799-oauth-serving-cert\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:35:50.354414 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.354409    2549 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0fb00168-57af-42e0-8397-c79457b74799-console-config\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:35:50.354414 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.354419    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bgjm\" (UniqueName: \"kubernetes.io/projected/0fb00168-57af-42e0-8397-c79457b74799-kube-api-access-4bgjm\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:35:50.354614 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.354427    2549 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0fb00168-57af-42e0-8397-c79457b74799-trusted-ca-bundle\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:35:50.354614 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.354436    2549 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0fb00168-57af-42e0-8397-c79457b74799-console-oauth-config\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:35:50.354614 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.354444    2549 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0fb00168-57af-42e0-8397-c79457b74799-console-serving-cert\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:35:50.947985 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.947946    2549 generic.go:356] "Generic (PLEG): container finished" podID="0fb00168-57af-42e0-8397-c79457b74799" containerID="ca1aee54f2f05f8d88b1400b919cb5fc9eb38193a73fa02ffdad3ccf7b798c82" exitCode=0
Jul 09 09:35:50.948359 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.947994    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57475f5bfb-lg5kw"
Jul 09 09:35:50.948359 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.948035    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57475f5bfb-lg5kw" event={"ID":"0fb00168-57af-42e0-8397-c79457b74799","Type":"ContainerDied","Data":"ca1aee54f2f05f8d88b1400b919cb5fc9eb38193a73fa02ffdad3ccf7b798c82"}
Jul 09 09:35:50.948359 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.948092    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57475f5bfb-lg5kw" event={"ID":"0fb00168-57af-42e0-8397-c79457b74799","Type":"ContainerDied","Data":"79f1e412047fd8e7de2f94db38a951e2c4f69228d0ff77637ec03c22f82a9c2b"}
Jul 09 09:35:50.948359 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.948115    2549 scope.go:117] "RemoveContainer" containerID="ca1aee54f2f05f8d88b1400b919cb5fc9eb38193a73fa02ffdad3ccf7b798c82"
Jul 09 09:35:50.958046 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.958030    2549 scope.go:117] "RemoveContainer" containerID="ca1aee54f2f05f8d88b1400b919cb5fc9eb38193a73fa02ffdad3ccf7b798c82"
Jul 09 09:35:50.958309 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:35:50.958290    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca1aee54f2f05f8d88b1400b919cb5fc9eb38193a73fa02ffdad3ccf7b798c82\": container with ID starting with ca1aee54f2f05f8d88b1400b919cb5fc9eb38193a73fa02ffdad3ccf7b798c82 not found: ID does not exist" containerID="ca1aee54f2f05f8d88b1400b919cb5fc9eb38193a73fa02ffdad3ccf7b798c82"
Jul 09 09:35:50.958353 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.958318    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca1aee54f2f05f8d88b1400b919cb5fc9eb38193a73fa02ffdad3ccf7b798c82"} err="failed to get container status \"ca1aee54f2f05f8d88b1400b919cb5fc9eb38193a73fa02ffdad3ccf7b798c82\": rpc error: code = NotFound desc = could not find container \"ca1aee54f2f05f8d88b1400b919cb5fc9eb38193a73fa02ffdad3ccf7b798c82\": container with ID starting with ca1aee54f2f05f8d88b1400b919cb5fc9eb38193a73fa02ffdad3ccf7b798c82 not found: ID does not exist"
Jul 09 09:35:50.975022 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.975003    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57475f5bfb-lg5kw"]
Jul 09 09:35:50.981615 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:50.981593    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57475f5bfb-lg5kw"]
Jul 09 09:35:52.385263 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:52.385231    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0fb00168-57af-42e0-8397-c79457b74799" path="/var/lib/kubelet/pods/0fb00168-57af-42e0-8397-c79457b74799/volumes"
Jul 09 09:35:54.212089 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:54.212061    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6485c4445-bcqjm"
Jul 09 09:35:54.215662 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:35:54.215641    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6485c4445-bcqjm"
Jul 09 09:36:16.163283 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:16.163256    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:16.182795 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:16.182774    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:17.076473 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:17.076447    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:29.931184 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:29.931145    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 09:36:30.110037 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:30.109999    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerName="alertmanager" containerID="cri-o://63ac463ed80afbd317c8a4ebbecdbd33c8523ae98da70527e6e5a71f6102ceb8" gracePeriod=120
Jul 09 09:36:30.110244 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:30.110066    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerName="kube-rbac-proxy-web" containerID="cri-o://abe4d96b8a959b907b5f511285f22fc519be2821f240d70fb95a19d0fc950456" gracePeriod=120
Jul 09 09:36:30.110244 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:30.110066    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerName="kube-rbac-proxy-metric" containerID="cri-o://d9a2852995803729ffe185356960db3ac22b7fa47185cb85e4efaf0c0a4d26d5" gracePeriod=120
Jul 09 09:36:30.110761 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:30.110527    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerName="config-reloader" containerID="cri-o://741cc0b6b1b5567a5115f43894b2955196ad9110ad427722a2325d3cdbd3daaf" gracePeriod=120
Jul 09 09:36:30.110761 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:30.110650    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerName="kube-rbac-proxy" containerID="cri-o://237e83b92fd9840c11df0cfb25b145c1625bde3a832068abf040f7818c7f7cc1" gracePeriod=120
Jul 09 09:36:30.110761 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:30.110093    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerName="prom-label-proxy" containerID="cri-o://d7bae89e3430f29a1b8aa78d669b2cea6010754f749766c25e1a0d71cba21c23" gracePeriod=120
Jul 09 09:36:31.116985 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.116941    2549 generic.go:356] "Generic (PLEG): container finished" podID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerID="d7bae89e3430f29a1b8aa78d669b2cea6010754f749766c25e1a0d71cba21c23" exitCode=0
Jul 09 09:36:31.116985 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.116976    2549 generic.go:356] "Generic (PLEG): container finished" podID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerID="237e83b92fd9840c11df0cfb25b145c1625bde3a832068abf040f7818c7f7cc1" exitCode=0
Jul 09 09:36:31.116985 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.116984    2549 generic.go:356] "Generic (PLEG): container finished" podID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerID="741cc0b6b1b5567a5115f43894b2955196ad9110ad427722a2325d3cdbd3daaf" exitCode=0
Jul 09 09:36:31.116985 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.116990    2549 generic.go:356] "Generic (PLEG): container finished" podID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerID="63ac463ed80afbd317c8a4ebbecdbd33c8523ae98da70527e6e5a71f6102ceb8" exitCode=0
Jul 09 09:36:31.117440 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.117010    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"247c41df-4a8f-42d5-80c7-207e2d27dea9","Type":"ContainerDied","Data":"d7bae89e3430f29a1b8aa78d669b2cea6010754f749766c25e1a0d71cba21c23"}
Jul 09 09:36:31.117440 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.117077    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"247c41df-4a8f-42d5-80c7-207e2d27dea9","Type":"ContainerDied","Data":"237e83b92fd9840c11df0cfb25b145c1625bde3a832068abf040f7818c7f7cc1"}
Jul 09 09:36:31.117440 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.117102    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"247c41df-4a8f-42d5-80c7-207e2d27dea9","Type":"ContainerDied","Data":"741cc0b6b1b5567a5115f43894b2955196ad9110ad427722a2325d3cdbd3daaf"}
Jul 09 09:36:31.117440 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.117125    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"247c41df-4a8f-42d5-80c7-207e2d27dea9","Type":"ContainerDied","Data":"63ac463ed80afbd317c8a4ebbecdbd33c8523ae98da70527e6e5a71f6102ceb8"}
Jul 09 09:36:31.456224 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.456199    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:31.488365 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.488340    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-config-volume\") pod \"247c41df-4a8f-42d5-80c7-207e2d27dea9\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") "
Jul 09 09:36:31.488489 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.488370    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-secret-alertmanager-kube-rbac-proxy-web\") pod \"247c41df-4a8f-42d5-80c7-207e2d27dea9\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") "
Jul 09 09:36:31.488489 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.488389    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-secret-alertmanager-main-tls\") pod \"247c41df-4a8f-42d5-80c7-207e2d27dea9\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") "
Jul 09 09:36:31.488489 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.488414    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9s92p\" (UniqueName: \"kubernetes.io/projected/247c41df-4a8f-42d5-80c7-207e2d27dea9-kube-api-access-9s92p\") pod \"247c41df-4a8f-42d5-80c7-207e2d27dea9\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") "
Jul 09 09:36:31.488489 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.488432    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-web-config\") pod \"247c41df-4a8f-42d5-80c7-207e2d27dea9\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") "
Jul 09 09:36:31.488489 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.488460    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"247c41df-4a8f-42d5-80c7-207e2d27dea9\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") "
Jul 09 09:36:31.488489 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.488486    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/247c41df-4a8f-42d5-80c7-207e2d27dea9-alertmanager-main-db\") pod \"247c41df-4a8f-42d5-80c7-207e2d27dea9\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") "
Jul 09 09:36:31.488847 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.488522    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-secret-alertmanager-kube-rbac-proxy\") pod \"247c41df-4a8f-42d5-80c7-207e2d27dea9\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") "
Jul 09 09:36:31.488847 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.488572    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/247c41df-4a8f-42d5-80c7-207e2d27dea9-tls-assets\") pod \"247c41df-4a8f-42d5-80c7-207e2d27dea9\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") "
Jul 09 09:36:31.488847 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.488609    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/247c41df-4a8f-42d5-80c7-207e2d27dea9-alertmanager-trusted-ca-bundle\") pod \"247c41df-4a8f-42d5-80c7-207e2d27dea9\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") "
Jul 09 09:36:31.488847 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.488641    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/247c41df-4a8f-42d5-80c7-207e2d27dea9-config-out\") pod \"247c41df-4a8f-42d5-80c7-207e2d27dea9\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") "
Jul 09 09:36:31.488847 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.488669    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-cluster-tls-config\") pod \"247c41df-4a8f-42d5-80c7-207e2d27dea9\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") "
Jul 09 09:36:31.488847 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.488708    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/247c41df-4a8f-42d5-80c7-207e2d27dea9-metrics-client-ca\") pod \"247c41df-4a8f-42d5-80c7-207e2d27dea9\" (UID: \"247c41df-4a8f-42d5-80c7-207e2d27dea9\") "
Jul 09 09:36:31.489160 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.488838    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/247c41df-4a8f-42d5-80c7-207e2d27dea9-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "247c41df-4a8f-42d5-80c7-207e2d27dea9" (UID: "247c41df-4a8f-42d5-80c7-207e2d27dea9"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:36:31.489160 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.489103    2549 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/247c41df-4a8f-42d5-80c7-207e2d27dea9-alertmanager-main-db\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:31.489268 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.489196    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/247c41df-4a8f-42d5-80c7-207e2d27dea9-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "247c41df-4a8f-42d5-80c7-207e2d27dea9" (UID: "247c41df-4a8f-42d5-80c7-207e2d27dea9"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:36:31.489268 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.489223    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/247c41df-4a8f-42d5-80c7-207e2d27dea9-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "247c41df-4a8f-42d5-80c7-207e2d27dea9" (UID: "247c41df-4a8f-42d5-80c7-207e2d27dea9"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:36:31.491522 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.491485    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/247c41df-4a8f-42d5-80c7-207e2d27dea9-kube-api-access-9s92p" (OuterVolumeSpecName: "kube-api-access-9s92p") pod "247c41df-4a8f-42d5-80c7-207e2d27dea9" (UID: "247c41df-4a8f-42d5-80c7-207e2d27dea9"). InnerVolumeSpecName "kube-api-access-9s92p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:36:31.491961 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.491852    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "247c41df-4a8f-42d5-80c7-207e2d27dea9" (UID: "247c41df-4a8f-42d5-80c7-207e2d27dea9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:36:31.492184 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.492150    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/247c41df-4a8f-42d5-80c7-207e2d27dea9-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "247c41df-4a8f-42d5-80c7-207e2d27dea9" (UID: "247c41df-4a8f-42d5-80c7-207e2d27dea9"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:36:31.492184 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.492162    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "247c41df-4a8f-42d5-80c7-207e2d27dea9" (UID: "247c41df-4a8f-42d5-80c7-207e2d27dea9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:36:31.492315 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.492259    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-config-volume" (OuterVolumeSpecName: "config-volume") pod "247c41df-4a8f-42d5-80c7-207e2d27dea9" (UID: "247c41df-4a8f-42d5-80c7-207e2d27dea9"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:36:31.492404 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.492380    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "247c41df-4a8f-42d5-80c7-207e2d27dea9" (UID: "247c41df-4a8f-42d5-80c7-207e2d27dea9"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:36:31.492508 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.492485    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "247c41df-4a8f-42d5-80c7-207e2d27dea9" (UID: "247c41df-4a8f-42d5-80c7-207e2d27dea9"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:36:31.493050 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.493030    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/247c41df-4a8f-42d5-80c7-207e2d27dea9-config-out" (OuterVolumeSpecName: "config-out") pod "247c41df-4a8f-42d5-80c7-207e2d27dea9" (UID: "247c41df-4a8f-42d5-80c7-207e2d27dea9"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:36:31.496054 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.496009    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "247c41df-4a8f-42d5-80c7-207e2d27dea9" (UID: "247c41df-4a8f-42d5-80c7-207e2d27dea9"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:36:31.502075 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.502050    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-web-config" (OuterVolumeSpecName: "web-config") pod "247c41df-4a8f-42d5-80c7-207e2d27dea9" (UID: "247c41df-4a8f-42d5-80c7-207e2d27dea9"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:36:31.589478 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.589447    2549 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/247c41df-4a8f-42d5-80c7-207e2d27dea9-tls-assets\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:31.589478 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.589471    2549 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/247c41df-4a8f-42d5-80c7-207e2d27dea9-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:31.589478 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.589481    2549 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/247c41df-4a8f-42d5-80c7-207e2d27dea9-config-out\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:31.589683 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.589489    2549 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-cluster-tls-config\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:31.589683 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.589500    2549 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/247c41df-4a8f-42d5-80c7-207e2d27dea9-metrics-client-ca\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:31.589683 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.589509    2549 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-config-volume\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:31.589683 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.589517    2549 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:31.589683 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.589525    2549 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-secret-alertmanager-main-tls\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:31.589683 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.589535    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9s92p\" (UniqueName: \"kubernetes.io/projected/247c41df-4a8f-42d5-80c7-207e2d27dea9-kube-api-access-9s92p\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:31.589683 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.589542    2549 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-web-config\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:31.589683 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.589550    2549 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:31.589683 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:31.589559    2549 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/247c41df-4a8f-42d5-80c7-207e2d27dea9-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:32.124209 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.124177    2549 generic.go:356] "Generic (PLEG): container finished" podID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerID="d9a2852995803729ffe185356960db3ac22b7fa47185cb85e4efaf0c0a4d26d5" exitCode=0
Jul 09 09:36:32.124209 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.124200    2549 generic.go:356] "Generic (PLEG): container finished" podID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerID="abe4d96b8a959b907b5f511285f22fc519be2821f240d70fb95a19d0fc950456" exitCode=0
Jul 09 09:36:32.124620 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.124233    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"247c41df-4a8f-42d5-80c7-207e2d27dea9","Type":"ContainerDied","Data":"d9a2852995803729ffe185356960db3ac22b7fa47185cb85e4efaf0c0a4d26d5"}
Jul 09 09:36:32.124620 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.124259    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.124620 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.124288    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"247c41df-4a8f-42d5-80c7-207e2d27dea9","Type":"ContainerDied","Data":"abe4d96b8a959b907b5f511285f22fc519be2821f240d70fb95a19d0fc950456"}
Jul 09 09:36:32.124620 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.124316    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"247c41df-4a8f-42d5-80c7-207e2d27dea9","Type":"ContainerDied","Data":"1755cda988399c0681bf0a1c4dc1b9f62ae3d112a3e281aac19221328396d8eb"}
Jul 09 09:36:32.124620 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.124346    2549 scope.go:117] "RemoveContainer" containerID="d7bae89e3430f29a1b8aa78d669b2cea6010754f749766c25e1a0d71cba21c23"
Jul 09 09:36:32.134569 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.134544    2549 scope.go:117] "RemoveContainer" containerID="d9a2852995803729ffe185356960db3ac22b7fa47185cb85e4efaf0c0a4d26d5"
Jul 09 09:36:32.141270 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.141252    2549 scope.go:117] "RemoveContainer" containerID="237e83b92fd9840c11df0cfb25b145c1625bde3a832068abf040f7818c7f7cc1"
Jul 09 09:36:32.147570 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.147554    2549 scope.go:117] "RemoveContainer" containerID="abe4d96b8a959b907b5f511285f22fc519be2821f240d70fb95a19d0fc950456"
Jul 09 09:36:32.151336 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.151315    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 09:36:32.154817 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.154658    2549 scope.go:117] "RemoveContainer" containerID="741cc0b6b1b5567a5115f43894b2955196ad9110ad427722a2325d3cdbd3daaf"
Jul 09 09:36:32.157024 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.157003    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 09:36:32.161696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.161681    2549 scope.go:117] "RemoveContainer" containerID="63ac463ed80afbd317c8a4ebbecdbd33c8523ae98da70527e6e5a71f6102ceb8"
Jul 09 09:36:32.169269 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.169253    2549 scope.go:117] "RemoveContainer" containerID="48519b062f7a608b6ba0d6c9eab648c15e881150a1ae50857a88f96ccc975c0e"
Jul 09 09:36:32.175699 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.175681    2549 scope.go:117] "RemoveContainer" containerID="d7bae89e3430f29a1b8aa78d669b2cea6010754f749766c25e1a0d71cba21c23"
Jul 09 09:36:32.175975 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:36:32.175957    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7bae89e3430f29a1b8aa78d669b2cea6010754f749766c25e1a0d71cba21c23\": container with ID starting with d7bae89e3430f29a1b8aa78d669b2cea6010754f749766c25e1a0d71cba21c23 not found: ID does not exist" containerID="d7bae89e3430f29a1b8aa78d669b2cea6010754f749766c25e1a0d71cba21c23"
Jul 09 09:36:32.176022 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.175982    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7bae89e3430f29a1b8aa78d669b2cea6010754f749766c25e1a0d71cba21c23"} err="failed to get container status \"d7bae89e3430f29a1b8aa78d669b2cea6010754f749766c25e1a0d71cba21c23\": rpc error: code = NotFound desc = could not find container \"d7bae89e3430f29a1b8aa78d669b2cea6010754f749766c25e1a0d71cba21c23\": container with ID starting with d7bae89e3430f29a1b8aa78d669b2cea6010754f749766c25e1a0d71cba21c23 not found: ID does not exist"
Jul 09 09:36:32.176022 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.175997    2549 scope.go:117] "RemoveContainer" containerID="d9a2852995803729ffe185356960db3ac22b7fa47185cb85e4efaf0c0a4d26d5"
Jul 09 09:36:32.176248 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:36:32.176232    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9a2852995803729ffe185356960db3ac22b7fa47185cb85e4efaf0c0a4d26d5\": container with ID starting with d9a2852995803729ffe185356960db3ac22b7fa47185cb85e4efaf0c0a4d26d5 not found: ID does not exist" containerID="d9a2852995803729ffe185356960db3ac22b7fa47185cb85e4efaf0c0a4d26d5"
Jul 09 09:36:32.176292 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.176255    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9a2852995803729ffe185356960db3ac22b7fa47185cb85e4efaf0c0a4d26d5"} err="failed to get container status \"d9a2852995803729ffe185356960db3ac22b7fa47185cb85e4efaf0c0a4d26d5\": rpc error: code = NotFound desc = could not find container \"d9a2852995803729ffe185356960db3ac22b7fa47185cb85e4efaf0c0a4d26d5\": container with ID starting with d9a2852995803729ffe185356960db3ac22b7fa47185cb85e4efaf0c0a4d26d5 not found: ID does not exist"
Jul 09 09:36:32.176292 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.176271    2549 scope.go:117] "RemoveContainer" containerID="237e83b92fd9840c11df0cfb25b145c1625bde3a832068abf040f7818c7f7cc1"
Jul 09 09:36:32.176480 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:36:32.176466    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"237e83b92fd9840c11df0cfb25b145c1625bde3a832068abf040f7818c7f7cc1\": container with ID starting with 237e83b92fd9840c11df0cfb25b145c1625bde3a832068abf040f7818c7f7cc1 not found: ID does not exist" containerID="237e83b92fd9840c11df0cfb25b145c1625bde3a832068abf040f7818c7f7cc1"
Jul 09 09:36:32.176523 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.176482    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"237e83b92fd9840c11df0cfb25b145c1625bde3a832068abf040f7818c7f7cc1"} err="failed to get container status \"237e83b92fd9840c11df0cfb25b145c1625bde3a832068abf040f7818c7f7cc1\": rpc error: code = NotFound desc = could not find container \"237e83b92fd9840c11df0cfb25b145c1625bde3a832068abf040f7818c7f7cc1\": container with ID starting with 237e83b92fd9840c11df0cfb25b145c1625bde3a832068abf040f7818c7f7cc1 not found: ID does not exist"
Jul 09 09:36:32.176523 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.176494    2549 scope.go:117] "RemoveContainer" containerID="abe4d96b8a959b907b5f511285f22fc519be2821f240d70fb95a19d0fc950456"
Jul 09 09:36:32.176723 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:36:32.176709    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abe4d96b8a959b907b5f511285f22fc519be2821f240d70fb95a19d0fc950456\": container with ID starting with abe4d96b8a959b907b5f511285f22fc519be2821f240d70fb95a19d0fc950456 not found: ID does not exist" containerID="abe4d96b8a959b907b5f511285f22fc519be2821f240d70fb95a19d0fc950456"
Jul 09 09:36:32.176762 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.176726    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abe4d96b8a959b907b5f511285f22fc519be2821f240d70fb95a19d0fc950456"} err="failed to get container status \"abe4d96b8a959b907b5f511285f22fc519be2821f240d70fb95a19d0fc950456\": rpc error: code = NotFound desc = could not find container \"abe4d96b8a959b907b5f511285f22fc519be2821f240d70fb95a19d0fc950456\": container with ID starting with abe4d96b8a959b907b5f511285f22fc519be2821f240d70fb95a19d0fc950456 not found: ID does not exist"
Jul 09 09:36:32.176762 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.176737    2549 scope.go:117] "RemoveContainer" containerID="741cc0b6b1b5567a5115f43894b2955196ad9110ad427722a2325d3cdbd3daaf"
Jul 09 09:36:32.176960 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:36:32.176944    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"741cc0b6b1b5567a5115f43894b2955196ad9110ad427722a2325d3cdbd3daaf\": container with ID starting with 741cc0b6b1b5567a5115f43894b2955196ad9110ad427722a2325d3cdbd3daaf not found: ID does not exist" containerID="741cc0b6b1b5567a5115f43894b2955196ad9110ad427722a2325d3cdbd3daaf"
Jul 09 09:36:32.177009 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.176962    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"741cc0b6b1b5567a5115f43894b2955196ad9110ad427722a2325d3cdbd3daaf"} err="failed to get container status \"741cc0b6b1b5567a5115f43894b2955196ad9110ad427722a2325d3cdbd3daaf\": rpc error: code = NotFound desc = could not find container \"741cc0b6b1b5567a5115f43894b2955196ad9110ad427722a2325d3cdbd3daaf\": container with ID starting with 741cc0b6b1b5567a5115f43894b2955196ad9110ad427722a2325d3cdbd3daaf not found: ID does not exist"
Jul 09 09:36:32.177009 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.176973    2549 scope.go:117] "RemoveContainer" containerID="63ac463ed80afbd317c8a4ebbecdbd33c8523ae98da70527e6e5a71f6102ceb8"
Jul 09 09:36:32.177155 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:36:32.177141    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63ac463ed80afbd317c8a4ebbecdbd33c8523ae98da70527e6e5a71f6102ceb8\": container with ID starting with 63ac463ed80afbd317c8a4ebbecdbd33c8523ae98da70527e6e5a71f6102ceb8 not found: ID does not exist" containerID="63ac463ed80afbd317c8a4ebbecdbd33c8523ae98da70527e6e5a71f6102ceb8"
Jul 09 09:36:32.177193 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.177156    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63ac463ed80afbd317c8a4ebbecdbd33c8523ae98da70527e6e5a71f6102ceb8"} err="failed to get container status \"63ac463ed80afbd317c8a4ebbecdbd33c8523ae98da70527e6e5a71f6102ceb8\": rpc error: code = NotFound desc = could not find container \"63ac463ed80afbd317c8a4ebbecdbd33c8523ae98da70527e6e5a71f6102ceb8\": container with ID starting with 63ac463ed80afbd317c8a4ebbecdbd33c8523ae98da70527e6e5a71f6102ceb8 not found: ID does not exist"
Jul 09 09:36:32.177193 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.177168    2549 scope.go:117] "RemoveContainer" containerID="48519b062f7a608b6ba0d6c9eab648c15e881150a1ae50857a88f96ccc975c0e"
Jul 09 09:36:32.177357 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:36:32.177344    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48519b062f7a608b6ba0d6c9eab648c15e881150a1ae50857a88f96ccc975c0e\": container with ID starting with 48519b062f7a608b6ba0d6c9eab648c15e881150a1ae50857a88f96ccc975c0e not found: ID does not exist" containerID="48519b062f7a608b6ba0d6c9eab648c15e881150a1ae50857a88f96ccc975c0e"
Jul 09 09:36:32.177402 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.177359    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48519b062f7a608b6ba0d6c9eab648c15e881150a1ae50857a88f96ccc975c0e"} err="failed to get container status \"48519b062f7a608b6ba0d6c9eab648c15e881150a1ae50857a88f96ccc975c0e\": rpc error: code = NotFound desc = could not find container \"48519b062f7a608b6ba0d6c9eab648c15e881150a1ae50857a88f96ccc975c0e\": container with ID starting with 48519b062f7a608b6ba0d6c9eab648c15e881150a1ae50857a88f96ccc975c0e not found: ID does not exist"
Jul 09 09:36:32.177402 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.177370    2549 scope.go:117] "RemoveContainer" containerID="d7bae89e3430f29a1b8aa78d669b2cea6010754f749766c25e1a0d71cba21c23"
Jul 09 09:36:32.177548 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.177533    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7bae89e3430f29a1b8aa78d669b2cea6010754f749766c25e1a0d71cba21c23"} err="failed to get container status \"d7bae89e3430f29a1b8aa78d669b2cea6010754f749766c25e1a0d71cba21c23\": rpc error: code = NotFound desc = could not find container \"d7bae89e3430f29a1b8aa78d669b2cea6010754f749766c25e1a0d71cba21c23\": container with ID starting with d7bae89e3430f29a1b8aa78d669b2cea6010754f749766c25e1a0d71cba21c23 not found: ID does not exist"
Jul 09 09:36:32.177585 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.177548    2549 scope.go:117] "RemoveContainer" containerID="d9a2852995803729ffe185356960db3ac22b7fa47185cb85e4efaf0c0a4d26d5"
Jul 09 09:36:32.177740 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.177719    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9a2852995803729ffe185356960db3ac22b7fa47185cb85e4efaf0c0a4d26d5"} err="failed to get container status \"d9a2852995803729ffe185356960db3ac22b7fa47185cb85e4efaf0c0a4d26d5\": rpc error: code = NotFound desc = could not find container \"d9a2852995803729ffe185356960db3ac22b7fa47185cb85e4efaf0c0a4d26d5\": container with ID starting with d9a2852995803729ffe185356960db3ac22b7fa47185cb85e4efaf0c0a4d26d5 not found: ID does not exist"
Jul 09 09:36:32.177785 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.177742    2549 scope.go:117] "RemoveContainer" containerID="237e83b92fd9840c11df0cfb25b145c1625bde3a832068abf040f7818c7f7cc1"
Jul 09 09:36:32.178046 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.178024    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"237e83b92fd9840c11df0cfb25b145c1625bde3a832068abf040f7818c7f7cc1"} err="failed to get container status \"237e83b92fd9840c11df0cfb25b145c1625bde3a832068abf040f7818c7f7cc1\": rpc error: code = NotFound desc = could not find container \"237e83b92fd9840c11df0cfb25b145c1625bde3a832068abf040f7818c7f7cc1\": container with ID starting with 237e83b92fd9840c11df0cfb25b145c1625bde3a832068abf040f7818c7f7cc1 not found: ID does not exist"
Jul 09 09:36:32.178126 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.178048    2549 scope.go:117] "RemoveContainer" containerID="abe4d96b8a959b907b5f511285f22fc519be2821f240d70fb95a19d0fc950456"
Jul 09 09:36:32.178288 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.178270    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abe4d96b8a959b907b5f511285f22fc519be2821f240d70fb95a19d0fc950456"} err="failed to get container status \"abe4d96b8a959b907b5f511285f22fc519be2821f240d70fb95a19d0fc950456\": rpc error: code = NotFound desc = could not find container \"abe4d96b8a959b907b5f511285f22fc519be2821f240d70fb95a19d0fc950456\": container with ID starting with abe4d96b8a959b907b5f511285f22fc519be2821f240d70fb95a19d0fc950456 not found: ID does not exist"
Jul 09 09:36:32.178288 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.178288    2549 scope.go:117] "RemoveContainer" containerID="741cc0b6b1b5567a5115f43894b2955196ad9110ad427722a2325d3cdbd3daaf"
Jul 09 09:36:32.178478 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.178464    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"741cc0b6b1b5567a5115f43894b2955196ad9110ad427722a2325d3cdbd3daaf"} err="failed to get container status \"741cc0b6b1b5567a5115f43894b2955196ad9110ad427722a2325d3cdbd3daaf\": rpc error: code = NotFound desc = could not find container \"741cc0b6b1b5567a5115f43894b2955196ad9110ad427722a2325d3cdbd3daaf\": container with ID starting with 741cc0b6b1b5567a5115f43894b2955196ad9110ad427722a2325d3cdbd3daaf not found: ID does not exist"
Jul 09 09:36:32.178525 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.178480    2549 scope.go:117] "RemoveContainer" containerID="63ac463ed80afbd317c8a4ebbecdbd33c8523ae98da70527e6e5a71f6102ceb8"
Jul 09 09:36:32.178731 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.178710    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63ac463ed80afbd317c8a4ebbecdbd33c8523ae98da70527e6e5a71f6102ceb8"} err="failed to get container status \"63ac463ed80afbd317c8a4ebbecdbd33c8523ae98da70527e6e5a71f6102ceb8\": rpc error: code = NotFound desc = could not find container \"63ac463ed80afbd317c8a4ebbecdbd33c8523ae98da70527e6e5a71f6102ceb8\": container with ID starting with 63ac463ed80afbd317c8a4ebbecdbd33c8523ae98da70527e6e5a71f6102ceb8 not found: ID does not exist"
Jul 09 09:36:32.178731 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.178731    2549 scope.go:117] "RemoveContainer" containerID="48519b062f7a608b6ba0d6c9eab648c15e881150a1ae50857a88f96ccc975c0e"
Jul 09 09:36:32.179214 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.179183    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48519b062f7a608b6ba0d6c9eab648c15e881150a1ae50857a88f96ccc975c0e"} err="failed to get container status \"48519b062f7a608b6ba0d6c9eab648c15e881150a1ae50857a88f96ccc975c0e\": rpc error: code = NotFound desc = could not find container \"48519b062f7a608b6ba0d6c9eab648c15e881150a1ae50857a88f96ccc975c0e\": container with ID starting with 48519b062f7a608b6ba0d6c9eab648c15e881150a1ae50857a88f96ccc975c0e not found: ID does not exist"
Jul 09 09:36:32.190177 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.190154    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 09:36:32.190454 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.190442    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe" containerName="console"
Jul 09 09:36:32.190495 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.190456    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe" containerName="console"
Jul 09 09:36:32.190495 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.190465    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerName="kube-rbac-proxy-metric"
Jul 09 09:36:32.190495 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.190471    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerName="kube-rbac-proxy-metric"
Jul 09 09:36:32.190495 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.190480    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerName="init-config-reloader"
Jul 09 09:36:32.190495 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.190485    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerName="init-config-reloader"
Jul 09 09:36:32.190495 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.190493    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fb00168-57af-42e0-8397-c79457b74799" containerName="console"
Jul 09 09:36:32.190686 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.190499    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fb00168-57af-42e0-8397-c79457b74799" containerName="console"
Jul 09 09:36:32.190686 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.190506    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerName="config-reloader"
Jul 09 09:36:32.190686 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.190511    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerName="config-reloader"
Jul 09 09:36:32.190686 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.190518    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerName="kube-rbac-proxy"
Jul 09 09:36:32.190686 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.190524    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerName="kube-rbac-proxy"
Jul 09 09:36:32.190686 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.190530    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerName="kube-rbac-proxy-web"
Jul 09 09:36:32.190686 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.190535    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerName="kube-rbac-proxy-web"
Jul 09 09:36:32.190686 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.190542    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerName="prom-label-proxy"
Jul 09 09:36:32.190686 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.190547    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerName="prom-label-proxy"
Jul 09 09:36:32.190686 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.190555    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerName="alertmanager"
Jul 09 09:36:32.190686 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.190560    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerName="alertmanager"
Jul 09 09:36:32.190686 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.190612    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerName="kube-rbac-proxy-web"
Jul 09 09:36:32.190686 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.190620    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerName="alertmanager"
Jul 09 09:36:32.190686 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.190627    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="69a05f8e-6f4f-40d4-9ca0-2ac466a16dbe" containerName="console"
Jul 09 09:36:32.190686 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.190632    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerName="kube-rbac-proxy-metric"
Jul 09 09:36:32.190686 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.190638    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerName="kube-rbac-proxy"
Jul 09 09:36:32.190686 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.190642    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerName="config-reloader"
Jul 09 09:36:32.190686 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.190648    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="247c41df-4a8f-42d5-80c7-207e2d27dea9" containerName="prom-label-proxy"
Jul 09 09:36:32.190686 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.190654    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fb00168-57af-42e0-8397-c79457b74799" containerName="console"
Jul 09 09:36:32.194229 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.194211    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.199830 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.199810    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 09 09:36:32.199925 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.199814    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 09 09:36:32.199925 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.199888    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-plk4q\""
Jul 09 09:36:32.199925 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.199916    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 09 09:36:32.201286 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.201261    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 09 09:36:32.201399 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.201320    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 09 09:36:32.201470 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.201407    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 09 09:36:32.201470 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.201406    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 09 09:36:32.201587 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.201556    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 09 09:36:32.208338 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.208321    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 09 09:36:32.214341 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.214312    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 09:36:32.294259 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.294229    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.294417 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.294270    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-web-config\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.294417 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.294293    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.294417 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.294335    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nnb7\" (UniqueName: \"kubernetes.io/projected/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-kube-api-access-4nnb7\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.294417 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.294377    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.294417 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.294414    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.294594 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.294453    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.294594 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.294471    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.294594 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.294505    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-tls-assets\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.294594 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.294533    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-config-out\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.294594 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.294567    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.294594 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.294589    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-config-volume\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.294783 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.294620    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.388813 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.388737    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="247c41df-4a8f-42d5-80c7-207e2d27dea9" path="/var/lib/kubelet/pods/247c41df-4a8f-42d5-80c7-207e2d27dea9/volumes"
Jul 09 09:36:32.395137 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.395116    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.395206 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.395144    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.395206 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.395172    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.395206 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.395188    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.395352 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.395212    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-tls-assets\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.395352 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.395276    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-config-out\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.395352 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.395323    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.395567 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.395546    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.396786 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.395673    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-config-volume\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.396786 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.395735    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.396786 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.395792    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.396786 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.395823    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-web-config\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.396786 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.395844    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.396786 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.395881    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nnb7\" (UniqueName: \"kubernetes.io/projected/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-kube-api-access-4nnb7\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.396786 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.396234    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.396786 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.396748    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.398351 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.398323    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.398455 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.398364    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.398574 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.398551    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-config-out\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.398793 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.398769    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-tls-assets\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.398942 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.398856    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.399163 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.399142    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.399293 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.399274    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-web-config\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.399488 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.399468    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-config-volume\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.400227 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.400206    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.404651 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.404629    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nnb7\" (UniqueName: \"kubernetes.io/projected/12a35c5d-dbfa-4acd-bdc0-f651e73ab391-kube-api-access-4nnb7\") pod \"alertmanager-main-0\" (UID: \"12a35c5d-dbfa-4acd-bdc0-f651e73ab391\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.503848 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.503824    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 09:36:32.640735 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:32.640655    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 09:36:32.643360 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:36:32.643337    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12a35c5d_dbfa_4acd_bdc0_f651e73ab391.slice/crio-36fbbcad8a83f7bcdb03040508e27a18aac8db5841c0a74be8679c8fad80f7b2 WatchSource:0}: Error finding container 36fbbcad8a83f7bcdb03040508e27a18aac8db5841c0a74be8679c8fad80f7b2: Status 404 returned error can't find the container with id 36fbbcad8a83f7bcdb03040508e27a18aac8db5841c0a74be8679c8fad80f7b2
Jul 09 09:36:33.130637 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:33.130607    2549 generic.go:356] "Generic (PLEG): container finished" podID="12a35c5d-dbfa-4acd-bdc0-f651e73ab391" containerID="102ade5c7f93721e96d88205100dc645ba7bb62b33853c49012c33a0e6170067" exitCode=0
Jul 09 09:36:33.131030 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:33.130643    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12a35c5d-dbfa-4acd-bdc0-f651e73ab391","Type":"ContainerDied","Data":"102ade5c7f93721e96d88205100dc645ba7bb62b33853c49012c33a0e6170067"}
Jul 09 09:36:33.131030 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:33.130690    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12a35c5d-dbfa-4acd-bdc0-f651e73ab391","Type":"ContainerStarted","Data":"36fbbcad8a83f7bcdb03040508e27a18aac8db5841c0a74be8679c8fad80f7b2"}
Jul 09 09:36:34.044217 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.044185    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-77f97ff54d-d5rnf"]
Jul 09 09:36:34.047060 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.047043    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf"
Jul 09 09:36:34.049731 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.049707    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 09 09:36:34.049836 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.049765    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 09 09:36:34.049836 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.049706    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 09 09:36:34.050092 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.050075    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 09 09:36:34.050132 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.050110    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-2bv4g\""
Jul 09 09:36:34.050179 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.050151    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 09 09:36:34.057317 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.057297    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 09 09:36:34.061342 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.061321    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-77f97ff54d-d5rnf"]
Jul 09 09:36:34.111038 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.111003    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf4837df-9a6d-40ee-9c81-c22cee47a344-telemeter-trusted-ca-bundle\") pod \"telemeter-client-77f97ff54d-d5rnf\" (UID: \"cf4837df-9a6d-40ee-9c81-c22cee47a344\") " pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf"
Jul 09 09:36:34.111038 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.111041    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf4837df-9a6d-40ee-9c81-c22cee47a344-serving-certs-ca-bundle\") pod \"telemeter-client-77f97ff54d-d5rnf\" (UID: \"cf4837df-9a6d-40ee-9c81-c22cee47a344\") " pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf"
Jul 09 09:36:34.111276 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.111062    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/cf4837df-9a6d-40ee-9c81-c22cee47a344-secret-telemeter-client\") pod \"telemeter-client-77f97ff54d-d5rnf\" (UID: \"cf4837df-9a6d-40ee-9c81-c22cee47a344\") " pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf"
Jul 09 09:36:34.111276 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.111141    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnqwx\" (UniqueName: \"kubernetes.io/projected/cf4837df-9a6d-40ee-9c81-c22cee47a344-kube-api-access-pnqwx\") pod \"telemeter-client-77f97ff54d-d5rnf\" (UID: \"cf4837df-9a6d-40ee-9c81-c22cee47a344\") " pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf"
Jul 09 09:36:34.111276 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.111180    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/cf4837df-9a6d-40ee-9c81-c22cee47a344-federate-client-tls\") pod \"telemeter-client-77f97ff54d-d5rnf\" (UID: \"cf4837df-9a6d-40ee-9c81-c22cee47a344\") " pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf"
Jul 09 09:36:34.111276 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.111208    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cf4837df-9a6d-40ee-9c81-c22cee47a344-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-77f97ff54d-d5rnf\" (UID: \"cf4837df-9a6d-40ee-9c81-c22cee47a344\") " pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf"
Jul 09 09:36:34.111276 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.111261    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf4837df-9a6d-40ee-9c81-c22cee47a344-metrics-client-ca\") pod \"telemeter-client-77f97ff54d-d5rnf\" (UID: \"cf4837df-9a6d-40ee-9c81-c22cee47a344\") " pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf"
Jul 09 09:36:34.111518 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.111301    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/cf4837df-9a6d-40ee-9c81-c22cee47a344-telemeter-client-tls\") pod \"telemeter-client-77f97ff54d-d5rnf\" (UID: \"cf4837df-9a6d-40ee-9c81-c22cee47a344\") " pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf"
Jul 09 09:36:34.138131 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.138099    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12a35c5d-dbfa-4acd-bdc0-f651e73ab391","Type":"ContainerStarted","Data":"f43803fd003ec436763e2b53b2707bcadccd0c8e09d0d84d1ec663bce6773540"}
Jul 09 09:36:34.138534 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.138180    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12a35c5d-dbfa-4acd-bdc0-f651e73ab391","Type":"ContainerStarted","Data":"c753b64ac629def93d3afdbe38a9bb6c630a048236ae9ae320b6f80f0b4415e3"}
Jul 09 09:36:34.138534 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.138206    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12a35c5d-dbfa-4acd-bdc0-f651e73ab391","Type":"ContainerStarted","Data":"9babe3595d36a3727fb0448c5e9c8c835da555f2a1e126e95d44e987701511f7"}
Jul 09 09:36:34.138534 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.138236    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12a35c5d-dbfa-4acd-bdc0-f651e73ab391","Type":"ContainerStarted","Data":"ddb7969eb025b109fd56c29087c25afd4c749c883c50dcbf4c501c8096d08e62"}
Jul 09 09:36:34.138534 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.138259    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12a35c5d-dbfa-4acd-bdc0-f651e73ab391","Type":"ContainerStarted","Data":"a94de288a7770567c287ca2efc104817120164e63ea2cece37b544d57971b393"}
Jul 09 09:36:34.138534 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.138280    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"12a35c5d-dbfa-4acd-bdc0-f651e73ab391","Type":"ContainerStarted","Data":"0f18c7defd8217c26bd87f51c0be6df645217b142b94d854c76ef6c004d949a6"}
Jul 09 09:36:34.166951 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.166895    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.151215758 podStartE2EDuration="2.166857741s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.641983ms" isStatefulPod=false podCreationTimestamp="2026-07-09 09:36:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:36:34.164969567 +0000 UTC m=+152.336128124" watchObservedRunningTime="2026-07-09 09:36:34.166857741 +0000 UTC m=+152.338016297"
Jul 09 09:36:34.212019 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.211976    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf4837df-9a6d-40ee-9c81-c22cee47a344-telemeter-trusted-ca-bundle\") pod \"telemeter-client-77f97ff54d-d5rnf\" (UID: \"cf4837df-9a6d-40ee-9c81-c22cee47a344\") " pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf"
Jul 09 09:36:34.212179 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.212039    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf4837df-9a6d-40ee-9c81-c22cee47a344-serving-certs-ca-bundle\") pod \"telemeter-client-77f97ff54d-d5rnf\" (UID: \"cf4837df-9a6d-40ee-9c81-c22cee47a344\") " pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf"
Jul 09 09:36:34.212295 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.212256    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/cf4837df-9a6d-40ee-9c81-c22cee47a344-secret-telemeter-client\") pod \"telemeter-client-77f97ff54d-d5rnf\" (UID: \"cf4837df-9a6d-40ee-9c81-c22cee47a344\") " pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf"
Jul 09 09:36:34.212386 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.212368    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnqwx\" (UniqueName: \"kubernetes.io/projected/cf4837df-9a6d-40ee-9c81-c22cee47a344-kube-api-access-pnqwx\") pod \"telemeter-client-77f97ff54d-d5rnf\" (UID: \"cf4837df-9a6d-40ee-9c81-c22cee47a344\") " pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf"
Jul 09 09:36:34.212465 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.212442    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/cf4837df-9a6d-40ee-9c81-c22cee47a344-federate-client-tls\") pod \"telemeter-client-77f97ff54d-d5rnf\" (UID: \"cf4837df-9a6d-40ee-9c81-c22cee47a344\") " pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf"
Jul 09 09:36:34.212527 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.212488    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cf4837df-9a6d-40ee-9c81-c22cee47a344-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-77f97ff54d-d5rnf\" (UID: \"cf4837df-9a6d-40ee-9c81-c22cee47a344\") " pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf"
Jul 09 09:36:34.213449 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.212732    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf4837df-9a6d-40ee-9c81-c22cee47a344-metrics-client-ca\") pod \"telemeter-client-77f97ff54d-d5rnf\" (UID: \"cf4837df-9a6d-40ee-9c81-c22cee47a344\") " pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf"
Jul 09 09:36:34.213449 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.212755    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf4837df-9a6d-40ee-9c81-c22cee47a344-serving-certs-ca-bundle\") pod \"telemeter-client-77f97ff54d-d5rnf\" (UID: \"cf4837df-9a6d-40ee-9c81-c22cee47a344\") " pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf"
Jul 09 09:36:34.213449 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.212858    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/cf4837df-9a6d-40ee-9c81-c22cee47a344-telemeter-client-tls\") pod \"telemeter-client-77f97ff54d-d5rnf\" (UID: \"cf4837df-9a6d-40ee-9c81-c22cee47a344\") " pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf"
Jul 09 09:36:34.213449 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.212959    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf4837df-9a6d-40ee-9c81-c22cee47a344-telemeter-trusted-ca-bundle\") pod \"telemeter-client-77f97ff54d-d5rnf\" (UID: \"cf4837df-9a6d-40ee-9c81-c22cee47a344\") " pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf"
Jul 09 09:36:34.213765 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.213681    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf4837df-9a6d-40ee-9c81-c22cee47a344-metrics-client-ca\") pod \"telemeter-client-77f97ff54d-d5rnf\" (UID: \"cf4837df-9a6d-40ee-9c81-c22cee47a344\") " pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf"
Jul 09 09:36:34.216058 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.216030    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cf4837df-9a6d-40ee-9c81-c22cee47a344-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-77f97ff54d-d5rnf\" (UID: \"cf4837df-9a6d-40ee-9c81-c22cee47a344\") " pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf"
Jul 09 09:36:34.216337 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.216318    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/cf4837df-9a6d-40ee-9c81-c22cee47a344-federate-client-tls\") pod \"telemeter-client-77f97ff54d-d5rnf\" (UID: \"cf4837df-9a6d-40ee-9c81-c22cee47a344\") " pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf"
Jul 09 09:36:34.216422 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.216341    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/cf4837df-9a6d-40ee-9c81-c22cee47a344-secret-telemeter-client\") pod \"telemeter-client-77f97ff54d-d5rnf\" (UID: \"cf4837df-9a6d-40ee-9c81-c22cee47a344\") " pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf"
Jul 09 09:36:34.216584 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.216567    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/cf4837df-9a6d-40ee-9c81-c22cee47a344-telemeter-client-tls\") pod \"telemeter-client-77f97ff54d-d5rnf\" (UID: \"cf4837df-9a6d-40ee-9c81-c22cee47a344\") " pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf"
Jul 09 09:36:34.222472 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.222448    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnqwx\" (UniqueName: \"kubernetes.io/projected/cf4837df-9a6d-40ee-9c81-c22cee47a344-kube-api-access-pnqwx\") pod \"telemeter-client-77f97ff54d-d5rnf\" (UID: \"cf4837df-9a6d-40ee-9c81-c22cee47a344\") " pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf"
Jul 09 09:36:34.360129 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.360022    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf"
Jul 09 09:36:34.361768 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.361741    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 09:36:34.362154 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.362123    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerName="prometheus" containerID="cri-o://a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5" gracePeriod=600
Jul 09 09:36:34.362221 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.362136    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerName="kube-rbac-proxy" containerID="cri-o://588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50" gracePeriod=600
Jul 09 09:36:34.362221 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.362187    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerName="config-reloader" containerID="cri-o://d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769" gracePeriod=600
Jul 09 09:36:34.362221 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.362168    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerName="kube-rbac-proxy-thanos" containerID="cri-o://bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5" gracePeriod=600
Jul 09 09:36:34.362390 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.362282    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerName="kube-rbac-proxy-web" containerID="cri-o://fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda" gracePeriod=600
Jul 09 09:36:34.362481 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.362148    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerName="thanos-sidecar" containerID="cri-o://51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a" gracePeriod=600
Jul 09 09:36:34.530735 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.530709    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-77f97ff54d-d5rnf"]
Jul 09 09:36:34.532735 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:36:34.532706    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf4837df_9a6d_40ee_9c81_c22cee47a344.slice/crio-9fccc0d0f1624fec0b409ffc130191c0f7214162c1c67bd50fecd6226fb02035 WatchSource:0}: Error finding container 9fccc0d0f1624fec0b409ffc130191c0f7214162c1c67bd50fecd6226fb02035: Status 404 returned error can't find the container with id 9fccc0d0f1624fec0b409ffc130191c0f7214162c1c67bd50fecd6226fb02035
Jul 09 09:36:34.619365 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.619264    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:34.716340 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.716309    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"e611bfba-e12e-416c-bf0a-42055b9ed56e\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") "
Jul 09 09:36:34.716340 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.716342    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-thanos-prometheus-http-client-file\") pod \"e611bfba-e12e-416c-bf0a-42055b9ed56e\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") "
Jul 09 09:36:34.716616 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.716366    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-web-config\") pod \"e611bfba-e12e-416c-bf0a-42055b9ed56e\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") "
Jul 09 09:36:34.716616 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.716497    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-k8s-rulefiles-2\") pod \"e611bfba-e12e-416c-bf0a-42055b9ed56e\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") "
Jul 09 09:36:34.716616 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.716528    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-prometheus-k8s-tls\") pod \"e611bfba-e12e-416c-bf0a-42055b9ed56e\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") "
Jul 09 09:36:34.716616 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.716556    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-grpc-tls\") pod \"e611bfba-e12e-416c-bf0a-42055b9ed56e\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") "
Jul 09 09:36:34.716616 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.716580    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"e611bfba-e12e-416c-bf0a-42055b9ed56e\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") "
Jul 09 09:36:34.716616 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.716605    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-configmap-metrics-client-ca\") pod \"e611bfba-e12e-416c-bf0a-42055b9ed56e\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") "
Jul 09 09:36:34.717019 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.716638    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2tc7\" (UniqueName: \"kubernetes.io/projected/e611bfba-e12e-416c-bf0a-42055b9ed56e-kube-api-access-z2tc7\") pod \"e611bfba-e12e-416c-bf0a-42055b9ed56e\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") "
Jul 09 09:36:34.717019 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.716687    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-kube-rbac-proxy\") pod \"e611bfba-e12e-416c-bf0a-42055b9ed56e\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") "
Jul 09 09:36:34.717019 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.716733    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-configmap-kubelet-serving-ca-bundle\") pod \"e611bfba-e12e-416c-bf0a-42055b9ed56e\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") "
Jul 09 09:36:34.717019 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.716765    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e611bfba-e12e-416c-bf0a-42055b9ed56e-tls-assets\") pod \"e611bfba-e12e-416c-bf0a-42055b9ed56e\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") "
Jul 09 09:36:34.717019 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.716806    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "e611bfba-e12e-416c-bf0a-42055b9ed56e" (UID: "e611bfba-e12e-416c-bf0a-42055b9ed56e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:36:34.717019 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.716817    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-trusted-ca-bundle\") pod \"e611bfba-e12e-416c-bf0a-42055b9ed56e\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") "
Jul 09 09:36:34.717019 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.716914    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e611bfba-e12e-416c-bf0a-42055b9ed56e-config-out\") pod \"e611bfba-e12e-416c-bf0a-42055b9ed56e\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") "
Jul 09 09:36:34.717019 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.716948    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-config\") pod \"e611bfba-e12e-416c-bf0a-42055b9ed56e\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") "
Jul 09 09:36:34.717019 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.716982    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-k8s-rulefiles-0\") pod \"e611bfba-e12e-416c-bf0a-42055b9ed56e\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") "
Jul 09 09:36:34.717019 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.717019    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-configmap-serving-certs-ca-bundle\") pod \"e611bfba-e12e-416c-bf0a-42055b9ed56e\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") "
Jul 09 09:36:34.717498 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.717091    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-k8s-db\") pod \"e611bfba-e12e-416c-bf0a-42055b9ed56e\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") "
Jul 09 09:36:34.717498 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.717123    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-metrics-client-certs\") pod \"e611bfba-e12e-416c-bf0a-42055b9ed56e\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") "
Jul 09 09:36:34.717498 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.717143    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "e611bfba-e12e-416c-bf0a-42055b9ed56e" (UID: "e611bfba-e12e-416c-bf0a-42055b9ed56e"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:36:34.717498 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.717161    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-k8s-rulefiles-1\") pod \"e611bfba-e12e-416c-bf0a-42055b9ed56e\" (UID: \"e611bfba-e12e-416c-bf0a-42055b9ed56e\") "
Jul 09 09:36:34.717498 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.717466    2549 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:34.717498 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.717488    2549 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-trusted-ca-bundle\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:34.717792 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.717536    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "e611bfba-e12e-416c-bf0a-42055b9ed56e" (UID: "e611bfba-e12e-416c-bf0a-42055b9ed56e"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:36:34.717792 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.717701    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "e611bfba-e12e-416c-bf0a-42055b9ed56e" (UID: "e611bfba-e12e-416c-bf0a-42055b9ed56e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:36:34.718087 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.718056    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "e611bfba-e12e-416c-bf0a-42055b9ed56e" (UID: "e611bfba-e12e-416c-bf0a-42055b9ed56e"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:36:34.718919 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.718890    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "e611bfba-e12e-416c-bf0a-42055b9ed56e" (UID: "e611bfba-e12e-416c-bf0a-42055b9ed56e"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:36:34.719453 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.719350    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "e611bfba-e12e-416c-bf0a-42055b9ed56e" (UID: "e611bfba-e12e-416c-bf0a-42055b9ed56e"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:36:34.719453 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.719415    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "e611bfba-e12e-416c-bf0a-42055b9ed56e" (UID: "e611bfba-e12e-416c-bf0a-42055b9ed56e"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:36:34.720528 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.720355    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "e611bfba-e12e-416c-bf0a-42055b9ed56e" (UID: "e611bfba-e12e-416c-bf0a-42055b9ed56e"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:36:34.720528 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.720494    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "e611bfba-e12e-416c-bf0a-42055b9ed56e" (UID: "e611bfba-e12e-416c-bf0a-42055b9ed56e"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:36:34.721197 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.721020    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e611bfba-e12e-416c-bf0a-42055b9ed56e-config-out" (OuterVolumeSpecName: "config-out") pod "e611bfba-e12e-416c-bf0a-42055b9ed56e" (UID: "e611bfba-e12e-416c-bf0a-42055b9ed56e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:36:34.721197 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.721130    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "e611bfba-e12e-416c-bf0a-42055b9ed56e" (UID: "e611bfba-e12e-416c-bf0a-42055b9ed56e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:36:34.721333 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.721268    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "e611bfba-e12e-416c-bf0a-42055b9ed56e" (UID: "e611bfba-e12e-416c-bf0a-42055b9ed56e"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:36:34.721333 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.721270    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e611bfba-e12e-416c-bf0a-42055b9ed56e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e611bfba-e12e-416c-bf0a-42055b9ed56e" (UID: "e611bfba-e12e-416c-bf0a-42055b9ed56e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:36:34.721459 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.721352    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "e611bfba-e12e-416c-bf0a-42055b9ed56e" (UID: "e611bfba-e12e-416c-bf0a-42055b9ed56e"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:36:34.721459 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.721407    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "e611bfba-e12e-416c-bf0a-42055b9ed56e" (UID: "e611bfba-e12e-416c-bf0a-42055b9ed56e"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:36:34.721566 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.721468    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e611bfba-e12e-416c-bf0a-42055b9ed56e-kube-api-access-z2tc7" (OuterVolumeSpecName: "kube-api-access-z2tc7") pod "e611bfba-e12e-416c-bf0a-42055b9ed56e" (UID: "e611bfba-e12e-416c-bf0a-42055b9ed56e"). InnerVolumeSpecName "kube-api-access-z2tc7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:36:34.721724 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.721702    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "e611bfba-e12e-416c-bf0a-42055b9ed56e" (UID: "e611bfba-e12e-416c-bf0a-42055b9ed56e"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:36:34.722507 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.722483    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-config" (OuterVolumeSpecName: "config") pod "e611bfba-e12e-416c-bf0a-42055b9ed56e" (UID: "e611bfba-e12e-416c-bf0a-42055b9ed56e"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:36:34.729960 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.729931    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-web-config" (OuterVolumeSpecName: "web-config") pod "e611bfba-e12e-416c-bf0a-42055b9ed56e" (UID: "e611bfba-e12e-416c-bf0a-42055b9ed56e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:36:34.818507 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.818478    2549 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-kube-rbac-proxy\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:34.818507 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.818503    2549 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:34.818507 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.818514    2549 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e611bfba-e12e-416c-bf0a-42055b9ed56e-tls-assets\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:34.818697 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.818522    2549 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e611bfba-e12e-416c-bf0a-42055b9ed56e-config-out\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:34.818697 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.818531    2549 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-config\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:34.818697 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.818539    2549 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:34.818697 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.818547    2549 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:34.818697 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.818555    2549 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-k8s-db\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:34.818697 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.818563    2549 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-metrics-client-certs\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:34.818697 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.818572    2549 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:34.818697 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.818581    2549 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:34.818697 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.818590    2549 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-thanos-prometheus-http-client-file\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:34.818697 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.818598    2549 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-web-config\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:34.818697 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.818607    2549 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-prometheus-k8s-tls\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:34.818697 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.818615    2549 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-grpc-tls\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:34.818697 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.818623    2549 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e611bfba-e12e-416c-bf0a-42055b9ed56e-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:34.818697 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.818631    2549 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e611bfba-e12e-416c-bf0a-42055b9ed56e-configmap-metrics-client-ca\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:34.818697 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:34.818640    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2tc7\" (UniqueName: \"kubernetes.io/projected/e611bfba-e12e-416c-bf0a-42055b9ed56e-kube-api-access-z2tc7\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:36:35.144206 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.144159    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf" event={"ID":"cf4837df-9a6d-40ee-9c81-c22cee47a344","Type":"ContainerStarted","Data":"9fccc0d0f1624fec0b409ffc130191c0f7214162c1c67bd50fecd6226fb02035"}
Jul 09 09:36:35.147743 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.147714    2549 generic.go:356] "Generic (PLEG): container finished" podID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerID="bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5" exitCode=0
Jul 09 09:36:35.147743 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.147742    2549 generic.go:356] "Generic (PLEG): container finished" podID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerID="588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50" exitCode=0
Jul 09 09:36:35.147957 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.147756    2549 generic.go:356] "Generic (PLEG): container finished" podID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerID="fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda" exitCode=0
Jul 09 09:36:35.147957 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.147756    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e611bfba-e12e-416c-bf0a-42055b9ed56e","Type":"ContainerDied","Data":"bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5"}
Jul 09 09:36:35.147957 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.147804    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.147957 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.147831    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e611bfba-e12e-416c-bf0a-42055b9ed56e","Type":"ContainerDied","Data":"588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50"}
Jul 09 09:36:35.147957 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.147770    2549 generic.go:356] "Generic (PLEG): container finished" podID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerID="51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a" exitCode=0
Jul 09 09:36:35.147957 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.147857    2549 generic.go:356] "Generic (PLEG): container finished" podID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerID="d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769" exitCode=0
Jul 09 09:36:35.147957 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.147887    2549 generic.go:356] "Generic (PLEG): container finished" podID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerID="a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5" exitCode=0
Jul 09 09:36:35.147957 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.147892    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e611bfba-e12e-416c-bf0a-42055b9ed56e","Type":"ContainerDied","Data":"fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda"}
Jul 09 09:36:35.147957 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.147901    2549 scope.go:117] "RemoveContainer" containerID="bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5"
Jul 09 09:36:35.147957 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.147935    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e611bfba-e12e-416c-bf0a-42055b9ed56e","Type":"ContainerDied","Data":"51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a"}
Jul 09 09:36:35.148415 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.147976    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e611bfba-e12e-416c-bf0a-42055b9ed56e","Type":"ContainerDied","Data":"d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769"}
Jul 09 09:36:35.148415 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.148017    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e611bfba-e12e-416c-bf0a-42055b9ed56e","Type":"ContainerDied","Data":"a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5"}
Jul 09 09:36:35.148415 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.148059    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e611bfba-e12e-416c-bf0a-42055b9ed56e","Type":"ContainerDied","Data":"58d3bdab5f7647daa562b6817b90391fb17802458c6988792789101f7d8577f0"}
Jul 09 09:36:35.156149 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.156132    2549 scope.go:117] "RemoveContainer" containerID="588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50"
Jul 09 09:36:35.162893 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.162875    2549 scope.go:117] "RemoveContainer" containerID="fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda"
Jul 09 09:36:35.169362 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.169342    2549 scope.go:117] "RemoveContainer" containerID="51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a"
Jul 09 09:36:35.176068 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.176049    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 09:36:35.176317 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.176301    2549 scope.go:117] "RemoveContainer" containerID="d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769"
Jul 09 09:36:35.180961 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.180942    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 09:36:35.182914 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.182898    2549 scope.go:117] "RemoveContainer" containerID="a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5"
Jul 09 09:36:35.189603 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.189584    2549 scope.go:117] "RemoveContainer" containerID="2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c"
Jul 09 09:36:35.195708 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.195694    2549 scope.go:117] "RemoveContainer" containerID="bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5"
Jul 09 09:36:35.195980 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:36:35.195961    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5\": container with ID starting with bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5 not found: ID does not exist" containerID="bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5"
Jul 09 09:36:35.196037 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.195987    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5"} err="failed to get container status \"bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5\": rpc error: code = NotFound desc = could not find container \"bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5\": container with ID starting with bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5 not found: ID does not exist"
Jul 09 09:36:35.196037 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.196004    2549 scope.go:117] "RemoveContainer" containerID="588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50"
Jul 09 09:36:35.196255 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:36:35.196238    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50\": container with ID starting with 588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50 not found: ID does not exist" containerID="588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50"
Jul 09 09:36:35.196292 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.196260    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50"} err="failed to get container status \"588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50\": rpc error: code = NotFound desc = could not find container \"588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50\": container with ID starting with 588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50 not found: ID does not exist"
Jul 09 09:36:35.196292 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.196276    2549 scope.go:117] "RemoveContainer" containerID="fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda"
Jul 09 09:36:35.196470 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:36:35.196455    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda\": container with ID starting with fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda not found: ID does not exist" containerID="fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda"
Jul 09 09:36:35.196504 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.196474    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda"} err="failed to get container status \"fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda\": rpc error: code = NotFound desc = could not find container \"fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda\": container with ID starting with fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda not found: ID does not exist"
Jul 09 09:36:35.196504 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.196488    2549 scope.go:117] "RemoveContainer" containerID="51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a"
Jul 09 09:36:35.196667 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:36:35.196650    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a\": container with ID starting with 51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a not found: ID does not exist" containerID="51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a"
Jul 09 09:36:35.196709 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.196670    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a"} err="failed to get container status \"51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a\": rpc error: code = NotFound desc = could not find container \"51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a\": container with ID starting with 51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a not found: ID does not exist"
Jul 09 09:36:35.196709 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.196684    2549 scope.go:117] "RemoveContainer" containerID="d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769"
Jul 09 09:36:35.196915 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:36:35.196895    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769\": container with ID starting with d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769 not found: ID does not exist" containerID="d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769"
Jul 09 09:36:35.196987 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.196917    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769"} err="failed to get container status \"d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769\": rpc error: code = NotFound desc = could not find container \"d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769\": container with ID starting with d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769 not found: ID does not exist"
Jul 09 09:36:35.196987 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.196929    2549 scope.go:117] "RemoveContainer" containerID="a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5"
Jul 09 09:36:35.197146 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:36:35.197128    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5\": container with ID starting with a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5 not found: ID does not exist" containerID="a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5"
Jul 09 09:36:35.197195 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.197150    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5"} err="failed to get container status \"a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5\": rpc error: code = NotFound desc = could not find container \"a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5\": container with ID starting with a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5 not found: ID does not exist"
Jul 09 09:36:35.197195 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.197164    2549 scope.go:117] "RemoveContainer" containerID="2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c"
Jul 09 09:36:35.197340 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:36:35.197323    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c\": container with ID starting with 2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c not found: ID does not exist" containerID="2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c"
Jul 09 09:36:35.197393 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.197345    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c"} err="failed to get container status \"2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c\": rpc error: code = NotFound desc = could not find container \"2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c\": container with ID starting with 2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c not found: ID does not exist"
Jul 09 09:36:35.197393 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.197359    2549 scope.go:117] "RemoveContainer" containerID="bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5"
Jul 09 09:36:35.197562 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.197548    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5"} err="failed to get container status \"bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5\": rpc error: code = NotFound desc = could not find container \"bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5\": container with ID starting with bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5 not found: ID does not exist"
Jul 09 09:36:35.197562 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.197562    2549 scope.go:117] "RemoveContainer" containerID="588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50"
Jul 09 09:36:35.197755 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.197737    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50"} err="failed to get container status \"588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50\": rpc error: code = NotFound desc = could not find container \"588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50\": container with ID starting with 588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50 not found: ID does not exist"
Jul 09 09:36:35.197814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.197757    2549 scope.go:117] "RemoveContainer" containerID="fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda"
Jul 09 09:36:35.198030 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.198012    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda"} err="failed to get container status \"fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda\": rpc error: code = NotFound desc = could not find container \"fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda\": container with ID starting with fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda not found: ID does not exist"
Jul 09 09:36:35.198112 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.198041    2549 scope.go:117] "RemoveContainer" containerID="51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a"
Jul 09 09:36:35.198281 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.198261    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a"} err="failed to get container status \"51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a\": rpc error: code = NotFound desc = could not find container \"51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a\": container with ID starting with 51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a not found: ID does not exist"
Jul 09 09:36:35.198281 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.198280    2549 scope.go:117] "RemoveContainer" containerID="d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769"
Jul 09 09:36:35.198507 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.198490    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769"} err="failed to get container status \"d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769\": rpc error: code = NotFound desc = could not find container \"d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769\": container with ID starting with d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769 not found: ID does not exist"
Jul 09 09:36:35.198507 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.198509    2549 scope.go:117] "RemoveContainer" containerID="a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5"
Jul 09 09:36:35.198778 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.198754    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5"} err="failed to get container status \"a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5\": rpc error: code = NotFound desc = could not find container \"a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5\": container with ID starting with a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5 not found: ID does not exist"
Jul 09 09:36:35.198778 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.198779    2549 scope.go:117] "RemoveContainer" containerID="2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c"
Jul 09 09:36:35.199095 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.199076    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c"} err="failed to get container status \"2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c\": rpc error: code = NotFound desc = could not find container \"2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c\": container with ID starting with 2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c not found: ID does not exist"
Jul 09 09:36:35.199168 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.199106    2549 scope.go:117] "RemoveContainer" containerID="bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5"
Jul 09 09:36:35.199349 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.199332    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5"} err="failed to get container status \"bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5\": rpc error: code = NotFound desc = could not find container \"bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5\": container with ID starting with bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5 not found: ID does not exist"
Jul 09 09:36:35.199398 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.199350    2549 scope.go:117] "RemoveContainer" containerID="588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50"
Jul 09 09:36:35.199553 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.199538    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50"} err="failed to get container status \"588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50\": rpc error: code = NotFound desc = could not find container \"588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50\": container with ID starting with 588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50 not found: ID does not exist"
Jul 09 09:36:35.199599 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.199553    2549 scope.go:117] "RemoveContainer" containerID="fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda"
Jul 09 09:36:35.199755 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.199739    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda"} err="failed to get container status \"fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda\": rpc error: code = NotFound desc = could not find container \"fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda\": container with ID starting with fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda not found: ID does not exist"
Jul 09 09:36:35.199755 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.199755    2549 scope.go:117] "RemoveContainer" containerID="51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a"
Jul 09 09:36:35.199936 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.199920    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a"} err="failed to get container status \"51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a\": rpc error: code = NotFound desc = could not find container \"51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a\": container with ID starting with 51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a not found: ID does not exist"
Jul 09 09:36:35.199999 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.199937    2549 scope.go:117] "RemoveContainer" containerID="d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769"
Jul 09 09:36:35.200147 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.200126    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769"} err="failed to get container status \"d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769\": rpc error: code = NotFound desc = could not find container \"d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769\": container with ID starting with d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769 not found: ID does not exist"
Jul 09 09:36:35.200191 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.200148    2549 scope.go:117] "RemoveContainer" containerID="a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5"
Jul 09 09:36:35.200318 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.200303    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5"} err="failed to get container status \"a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5\": rpc error: code = NotFound desc = could not find container \"a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5\": container with ID starting with a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5 not found: ID does not exist"
Jul 09 09:36:35.200351 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.200318    2549 scope.go:117] "RemoveContainer" containerID="2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c"
Jul 09 09:36:35.200516 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.200498    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c"} err="failed to get container status \"2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c\": rpc error: code = NotFound desc = could not find container \"2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c\": container with ID starting with 2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c not found: ID does not exist"
Jul 09 09:36:35.200553 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.200517    2549 scope.go:117] "RemoveContainer" containerID="bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5"
Jul 09 09:36:35.200710 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.200695    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5"} err="failed to get container status \"bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5\": rpc error: code = NotFound desc = could not find container \"bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5\": container with ID starting with bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5 not found: ID does not exist"
Jul 09 09:36:35.200746 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.200711    2549 scope.go:117] "RemoveContainer" containerID="588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50"
Jul 09 09:36:35.200912 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.200895    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50"} err="failed to get container status \"588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50\": rpc error: code = NotFound desc = could not find container \"588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50\": container with ID starting with 588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50 not found: ID does not exist"
Jul 09 09:36:35.200953 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.200913    2549 scope.go:117] "RemoveContainer" containerID="fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda"
Jul 09 09:36:35.201085 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.201067    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda"} err="failed to get container status \"fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda\": rpc error: code = NotFound desc = could not find container \"fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda\": container with ID starting with fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda not found: ID does not exist"
Jul 09 09:36:35.201127 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.201085    2549 scope.go:117] "RemoveContainer" containerID="51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a"
Jul 09 09:36:35.201240 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.201222    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a"} err="failed to get container status \"51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a\": rpc error: code = NotFound desc = could not find container \"51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a\": container with ID starting with 51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a not found: ID does not exist"
Jul 09 09:36:35.201277 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.201241    2549 scope.go:117] "RemoveContainer" containerID="d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769"
Jul 09 09:36:35.201442 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.201428    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769"} err="failed to get container status \"d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769\": rpc error: code = NotFound desc = could not find container \"d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769\": container with ID starting with d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769 not found: ID does not exist"
Jul 09 09:36:35.201442 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.201442    2549 scope.go:117] "RemoveContainer" containerID="a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5"
Jul 09 09:36:35.201643 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.201627    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5"} err="failed to get container status \"a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5\": rpc error: code = NotFound desc = could not find container \"a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5\": container with ID starting with a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5 not found: ID does not exist"
Jul 09 09:36:35.201687 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.201643    2549 scope.go:117] "RemoveContainer" containerID="2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c"
Jul 09 09:36:35.201859 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.201841    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c"} err="failed to get container status \"2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c\": rpc error: code = NotFound desc = could not find container \"2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c\": container with ID starting with 2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c not found: ID does not exist"
Jul 09 09:36:35.201919 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.201859    2549 scope.go:117] "RemoveContainer" containerID="bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5"
Jul 09 09:36:35.202093 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.202077    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5"} err="failed to get container status \"bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5\": rpc error: code = NotFound desc = could not find container \"bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5\": container with ID starting with bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5 not found: ID does not exist"
Jul 09 09:36:35.202134 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.202093    2549 scope.go:117] "RemoveContainer" containerID="588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50"
Jul 09 09:36:35.202284 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.202268    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50"} err="failed to get container status \"588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50\": rpc error: code = NotFound desc = could not find container \"588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50\": container with ID starting with 588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50 not found: ID does not exist"
Jul 09 09:36:35.202337 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.202287    2549 scope.go:117] "RemoveContainer" containerID="fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda"
Jul 09 09:36:35.202507 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.202490    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda"} err="failed to get container status \"fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda\": rpc error: code = NotFound desc = could not find container \"fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda\": container with ID starting with fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda not found: ID does not exist"
Jul 09 09:36:35.202557 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.202507    2549 scope.go:117] "RemoveContainer" containerID="51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a"
Jul 09 09:36:35.202746 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.202729    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a"} err="failed to get container status \"51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a\": rpc error: code = NotFound desc = could not find container \"51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a\": container with ID starting with 51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a not found: ID does not exist"
Jul 09 09:36:35.202786 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.202748    2549 scope.go:117] "RemoveContainer" containerID="d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769"
Jul 09 09:36:35.202985 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.202969    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769"} err="failed to get container status \"d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769\": rpc error: code = NotFound desc = could not find container \"d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769\": container with ID starting with d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769 not found: ID does not exist"
Jul 09 09:36:35.203035 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.202986    2549 scope.go:117] "RemoveContainer" containerID="a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5"
Jul 09 09:36:35.203150 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.203136    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5"} err="failed to get container status \"a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5\": rpc error: code = NotFound desc = could not find container \"a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5\": container with ID starting with a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5 not found: ID does not exist"
Jul 09 09:36:35.203187 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.203150    2549 scope.go:117] "RemoveContainer" containerID="2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c"
Jul 09 09:36:35.203352 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.203336    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c"} err="failed to get container status \"2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c\": rpc error: code = NotFound desc = could not find container \"2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c\": container with ID starting with 2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c not found: ID does not exist"
Jul 09 09:36:35.203400 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.203352    2549 scope.go:117] "RemoveContainer" containerID="bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5"
Jul 09 09:36:35.203539 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.203522    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5"} err="failed to get container status \"bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5\": rpc error: code = NotFound desc = could not find container \"bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5\": container with ID starting with bf4155d161b7ca4dfcd9fe0751bae8b911d45ded179674de199701c8282b46d5 not found: ID does not exist"
Jul 09 09:36:35.203580 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.203539    2549 scope.go:117] "RemoveContainer" containerID="588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50"
Jul 09 09:36:35.203756 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.203738    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50"} err="failed to get container status \"588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50\": rpc error: code = NotFound desc = could not find container \"588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50\": container with ID starting with 588471134dd60f356c24fded184cbfb26caf83f869d84d9a34c71e451a984a50 not found: ID does not exist"
Jul 09 09:36:35.203802 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.203756    2549 scope.go:117] "RemoveContainer" containerID="fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda"
Jul 09 09:36:35.203974 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.203960    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda"} err="failed to get container status \"fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda\": rpc error: code = NotFound desc = could not find container \"fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda\": container with ID starting with fc6c7e84fe1dcc02d3267f9ec9449afa9e9f781af93642ee75c0ea27457d3eda not found: ID does not exist"
Jul 09 09:36:35.203974 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.203974    2549 scope.go:117] "RemoveContainer" containerID="51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a"
Jul 09 09:36:35.204164 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.204147    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a"} err="failed to get container status \"51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a\": rpc error: code = NotFound desc = could not find container \"51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a\": container with ID starting with 51ba70b8ffb0a9d79539a4c0352ee3e20321ed76c87157a4a759a5a5f66a822a not found: ID does not exist"
Jul 09 09:36:35.204164 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.204164    2549 scope.go:117] "RemoveContainer" containerID="d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769"
Jul 09 09:36:35.204356 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.204335    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769"} err="failed to get container status \"d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769\": rpc error: code = NotFound desc = could not find container \"d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769\": container with ID starting with d86ae9d916efee1670489e88ca204230a03dceded850c8953069bf4e666d6769 not found: ID does not exist"
Jul 09 09:36:35.204356 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.204355    2549 scope.go:117] "RemoveContainer" containerID="a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5"
Jul 09 09:36:35.204549 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.204535    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5"} err="failed to get container status \"a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5\": rpc error: code = NotFound desc = could not find container \"a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5\": container with ID starting with a8caaed326dfece1efef8520f691e6d7797c359fd09a69a2cc1da54419e3bdf5 not found: ID does not exist"
Jul 09 09:36:35.204598 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.204550    2549 scope.go:117] "RemoveContainer" containerID="2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c"
Jul 09 09:36:35.204716 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.204702    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c"} err="failed to get container status \"2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c\": rpc error: code = NotFound desc = could not find container \"2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c\": container with ID starting with 2e97238f71397a83c8f531f9a97d6f9c1f0c9624178f1ca6482ce3912bf2716c not found: ID does not exist"
Jul 09 09:36:35.208732 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.208713    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 09:36:35.209025 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.209012    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerName="kube-rbac-proxy-thanos"
Jul 09 09:36:35.209071 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.209027    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerName="kube-rbac-proxy-thanos"
Jul 09 09:36:35.209071 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.209035    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerName="config-reloader"
Jul 09 09:36:35.209071 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.209041    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerName="config-reloader"
Jul 09 09:36:35.209071 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.209050    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerName="kube-rbac-proxy-web"
Jul 09 09:36:35.209071 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.209055    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerName="kube-rbac-proxy-web"
Jul 09 09:36:35.209071 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.209061    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerName="init-config-reloader"
Jul 09 09:36:35.209071 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.209065    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerName="init-config-reloader"
Jul 09 09:36:35.209071 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.209070    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerName="thanos-sidecar"
Jul 09 09:36:35.209290 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.209076    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerName="thanos-sidecar"
Jul 09 09:36:35.209290 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.209087    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerName="kube-rbac-proxy"
Jul 09 09:36:35.209290 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.209093    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerName="kube-rbac-proxy"
Jul 09 09:36:35.209290 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.209100    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerName="prometheus"
Jul 09 09:36:35.209290 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.209105    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerName="prometheus"
Jul 09 09:36:35.209290 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.209156    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerName="prometheus"
Jul 09 09:36:35.209290 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.209165    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerName="kube-rbac-proxy-web"
Jul 09 09:36:35.209290 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.209171    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerName="kube-rbac-proxy-thanos"
Jul 09 09:36:35.209290 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.209177    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerName="kube-rbac-proxy"
Jul 09 09:36:35.209290 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.209183    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerName="config-reloader"
Jul 09 09:36:35.209290 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.209189    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="e611bfba-e12e-416c-bf0a-42055b9ed56e" containerName="thanos-sidecar"
Jul 09 09:36:35.212656 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.212641    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.215757 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.215733    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 09 09:36:35.215846 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.215742    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 09 09:36:35.215846 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.215797    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 09 09:36:35.215846 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.215760    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 09 09:36:35.216097 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.216081    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6j816k2kt5o32\""
Jul 09 09:36:35.216157 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.216143    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-kzbcv\""
Jul 09 09:36:35.216232 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.216217    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 09 09:36:35.216485 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.216464    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 09 09:36:35.216532 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.216464    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 09 09:36:35.216995 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.216980    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 09 09:36:35.217187 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.217171    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 09 09:36:35.217268 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.217241    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 09 09:36:35.217447 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.217431    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 09 09:36:35.217493 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.217475    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 09 09:36:35.226646 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.223530    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.226646 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.223641    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.226646 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.223679    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.226646 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.223749    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.226646 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.223789    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-config-out\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.226646 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.223824    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.226646 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.223897    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.226646 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.224014    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-config\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.226646 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.224067    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-web-config\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.226646 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.224263    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.226646 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.224622    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.226646 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.224667    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbfhk\" (UniqueName: \"kubernetes.io/projected/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-kube-api-access-fbfhk\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.226646 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.224727    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.226646 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.224795    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.226646 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.224879    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.226646 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.224951    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.227443 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.225036    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.227443 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.225061    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.227443 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.225118    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.227443 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.225152    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.227838 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.227820    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 09 09:36:35.228529 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.228506    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 09 09:36:35.230837 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.230814    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 09:36:35.326380 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.326349    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.326528 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.326405    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.326528 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.326431    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.326528 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.326464    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.326528 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.326493    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-config-out\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.326528 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.326519    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.326787 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.326538    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.326787 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.326562    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-config\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.326787 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.326588    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-web-config\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.326787 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.326611    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.326787 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.326644    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.326787 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.326668    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbfhk\" (UniqueName: \"kubernetes.io/projected/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-kube-api-access-fbfhk\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.326787 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.326695    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.326787 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.326730    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.326787 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.326764    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.327262 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.326792    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.327262 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.326859    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.327262 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.326908    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.327262 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.326944    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.327262 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.326972    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.327493 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.327262    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.327493 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.327338    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.327670 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.327643    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.330057 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.329218    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.330057 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.329238    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-config-out\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.330057 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.329720    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.330057 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.329944    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.331008 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.330314    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.331008 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.330553    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.331008 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.330953    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.331396 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.331374    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.331476 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.331406    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.331476 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.331405    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.332047 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.332007    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.332436 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.332416    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.333066 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.333047    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.333533 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.333513    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-config\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.333837 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.333820    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-web-config\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.334063 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.334041    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.339051 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.339019    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbfhk\" (UniqueName: \"kubernetes.io/projected/a57c8eee-2c09-4e5d-aa20-4894697ed0b8-kube-api-access-fbfhk\") pod \"prometheus-k8s-0\" (UID: \"a57c8eee-2c09-4e5d-aa20-4894697ed0b8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.524072 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.524044    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:35.676762 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:35.676607    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 09:36:35.679714 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:36:35.679684    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda57c8eee_2c09_4e5d_aa20_4894697ed0b8.slice/crio-47f07ee37da20a5c2e4d5feb86ed9578f50c7702316b00eaaffbf3488010370f WatchSource:0}: Error finding container 47f07ee37da20a5c2e4d5feb86ed9578f50c7702316b00eaaffbf3488010370f: Status 404 returned error can't find the container with id 47f07ee37da20a5c2e4d5feb86ed9578f50c7702316b00eaaffbf3488010370f
Jul 09 09:36:36.154416 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:36.154383    2549 generic.go:356] "Generic (PLEG): container finished" podID="a57c8eee-2c09-4e5d-aa20-4894697ed0b8" containerID="83ee69c782a3a640598a69df692df57afdbabb1e138dd28ca5776c1fc8b51c4f" exitCode=0
Jul 09 09:36:36.154905 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:36.154483    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a57c8eee-2c09-4e5d-aa20-4894697ed0b8","Type":"ContainerDied","Data":"83ee69c782a3a640598a69df692df57afdbabb1e138dd28ca5776c1fc8b51c4f"}
Jul 09 09:36:36.154905 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:36.154579    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a57c8eee-2c09-4e5d-aa20-4894697ed0b8","Type":"ContainerStarted","Data":"47f07ee37da20a5c2e4d5feb86ed9578f50c7702316b00eaaffbf3488010370f"}
Jul 09 09:36:36.385687 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:36.385653    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e611bfba-e12e-416c-bf0a-42055b9ed56e" path="/var/lib/kubelet/pods/e611bfba-e12e-416c-bf0a-42055b9ed56e/volumes"
Jul 09 09:36:37.162650 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:37.162615    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf" event={"ID":"cf4837df-9a6d-40ee-9c81-c22cee47a344","Type":"ContainerStarted","Data":"0b003f79f55ae647d3fc59c44a21c157486ed35e3b086668d24f1473e9838510"}
Jul 09 09:36:37.163104 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:37.162708    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf" event={"ID":"cf4837df-9a6d-40ee-9c81-c22cee47a344","Type":"ContainerStarted","Data":"6a12b23c544d471df7fb28e28d161691f289e1c1811186b6f623d4c4ee812e4f"}
Jul 09 09:36:37.163104 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:37.162737    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf" event={"ID":"cf4837df-9a6d-40ee-9c81-c22cee47a344","Type":"ContainerStarted","Data":"c364a8250ab92ad5f41d98532c6714c63d461e53e742402b5aa6b28396331f0c"}
Jul 09 09:36:37.165741 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:37.165719    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a57c8eee-2c09-4e5d-aa20-4894697ed0b8","Type":"ContainerStarted","Data":"918e65ddda848f4ee3089b6c5287c5adbf267172005203a8d0011137fe54322b"}
Jul 09 09:36:37.165893 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:37.165780    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a57c8eee-2c09-4e5d-aa20-4894697ed0b8","Type":"ContainerStarted","Data":"26e0150b102e604244580c9d6757231a7c151b5e32809316119db9038cf28f65"}
Jul 09 09:36:37.165893 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:37.165818    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a57c8eee-2c09-4e5d-aa20-4894697ed0b8","Type":"ContainerStarted","Data":"647c745d2134f6951dd29ff6874f0fe047508e4f54ef4bd80b498a1216b71aca"}
Jul 09 09:36:37.165893 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:37.165859    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a57c8eee-2c09-4e5d-aa20-4894697ed0b8","Type":"ContainerStarted","Data":"4b21bf261fbf0c6e7a920714a2c552cae97b2cc45d27e09082c2ea15a8c62fc4"}
Jul 09 09:36:37.166061 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:37.165915    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a57c8eee-2c09-4e5d-aa20-4894697ed0b8","Type":"ContainerStarted","Data":"67d8016b5cfdaa7bc02c8e988fd99985ccc992f86267884924fb4afe07cb1d72"}
Jul 09 09:36:37.166061 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:37.165960    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a57c8eee-2c09-4e5d-aa20-4894697ed0b8","Type":"ContainerStarted","Data":"dd5222bdd643aa655220fa0f8564a4680b655c40f2aacba5fced028b848a131a"}
Jul 09 09:36:37.192171 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:37.192127    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-77f97ff54d-d5rnf" podStartSLOduration=1.272732826 podStartE2EDuration="3.192111711s" totalImagesPullingTime="1.919378885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:36:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:36:37.190129281 +0000 UTC m=+155.361287850" watchObservedRunningTime="2026-07-09 09:36:37.192111711 +0000 UTC m=+155.363270267"
Jul 09 09:36:37.223618 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:37.223578    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.200152671 podStartE2EDuration="2.223564096s" totalImagesPullingTime="0s" totalInitContainerRuntime="23.411425ms" isStatefulPod=false podCreationTimestamp="2026-07-09 09:36:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:36:37.222193157 +0000 UTC m=+155.393351730" watchObservedRunningTime="2026-07-09 09:36:37.223564096 +0000 UTC m=+155.394722652"
Jul 09 09:36:40.524403 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:40.524359    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:36:40.602198 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:40.602170    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6bb9b9c769-qvc5m"]
Jul 09 09:36:40.605288 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:40.605272    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bb9b9c769-qvc5m"
Jul 09 09:36:40.621196 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:40.621169    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bb9b9c769-qvc5m"]
Jul 09 09:36:40.670226 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:40.670202    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e46e2eb-e134-40d6-b82f-141c40f92810-service-ca\") pod \"console-6bb9b9c769-qvc5m\" (UID: \"0e46e2eb-e134-40d6-b82f-141c40f92810\") " pod="openshift-console/console-6bb9b9c769-qvc5m"
Jul 09 09:36:40.670281 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:40.670248    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e46e2eb-e134-40d6-b82f-141c40f92810-console-serving-cert\") pod \"console-6bb9b9c769-qvc5m\" (UID: \"0e46e2eb-e134-40d6-b82f-141c40f92810\") " pod="openshift-console/console-6bb9b9c769-qvc5m"
Jul 09 09:36:40.670320 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:40.670308    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e46e2eb-e134-40d6-b82f-141c40f92810-console-config\") pod \"console-6bb9b9c769-qvc5m\" (UID: \"0e46e2eb-e134-40d6-b82f-141c40f92810\") " pod="openshift-console/console-6bb9b9c769-qvc5m"
Jul 09 09:36:40.670370 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:40.670358    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e46e2eb-e134-40d6-b82f-141c40f92810-oauth-serving-cert\") pod \"console-6bb9b9c769-qvc5m\" (UID: \"0e46e2eb-e134-40d6-b82f-141c40f92810\") " pod="openshift-console/console-6bb9b9c769-qvc5m"
Jul 09 09:36:40.670404 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:40.670378    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e46e2eb-e134-40d6-b82f-141c40f92810-trusted-ca-bundle\") pod \"console-6bb9b9c769-qvc5m\" (UID: \"0e46e2eb-e134-40d6-b82f-141c40f92810\") " pod="openshift-console/console-6bb9b9c769-qvc5m"
Jul 09 09:36:40.670404 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:40.670393    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xvkd\" (UniqueName: \"kubernetes.io/projected/0e46e2eb-e134-40d6-b82f-141c40f92810-kube-api-access-5xvkd\") pod \"console-6bb9b9c769-qvc5m\" (UID: \"0e46e2eb-e134-40d6-b82f-141c40f92810\") " pod="openshift-console/console-6bb9b9c769-qvc5m"
Jul 09 09:36:40.670465 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:40.670443    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e46e2eb-e134-40d6-b82f-141c40f92810-console-oauth-config\") pod \"console-6bb9b9c769-qvc5m\" (UID: \"0e46e2eb-e134-40d6-b82f-141c40f92810\") " pod="openshift-console/console-6bb9b9c769-qvc5m"
Jul 09 09:36:40.771405 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:40.771376    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e46e2eb-e134-40d6-b82f-141c40f92810-console-serving-cert\") pod \"console-6bb9b9c769-qvc5m\" (UID: \"0e46e2eb-e134-40d6-b82f-141c40f92810\") " pod="openshift-console/console-6bb9b9c769-qvc5m"
Jul 09 09:36:40.771527 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:40.771434    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e46e2eb-e134-40d6-b82f-141c40f92810-console-config\") pod \"console-6bb9b9c769-qvc5m\" (UID: \"0e46e2eb-e134-40d6-b82f-141c40f92810\") " pod="openshift-console/console-6bb9b9c769-qvc5m"
Jul 09 09:36:40.771527 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:40.771481    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e46e2eb-e134-40d6-b82f-141c40f92810-oauth-serving-cert\") pod \"console-6bb9b9c769-qvc5m\" (UID: \"0e46e2eb-e134-40d6-b82f-141c40f92810\") " pod="openshift-console/console-6bb9b9c769-qvc5m"
Jul 09 09:36:40.771527 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:40.771505    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e46e2eb-e134-40d6-b82f-141c40f92810-trusted-ca-bundle\") pod \"console-6bb9b9c769-qvc5m\" (UID: \"0e46e2eb-e134-40d6-b82f-141c40f92810\") " pod="openshift-console/console-6bb9b9c769-qvc5m"
Jul 09 09:36:40.771683 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:40.771547    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xvkd\" (UniqueName: \"kubernetes.io/projected/0e46e2eb-e134-40d6-b82f-141c40f92810-kube-api-access-5xvkd\") pod \"console-6bb9b9c769-qvc5m\" (UID: \"0e46e2eb-e134-40d6-b82f-141c40f92810\") " pod="openshift-console/console-6bb9b9c769-qvc5m"
Jul 09 09:36:40.771683 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:40.771589    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e46e2eb-e134-40d6-b82f-141c40f92810-console-oauth-config\") pod \"console-6bb9b9c769-qvc5m\" (UID: \"0e46e2eb-e134-40d6-b82f-141c40f92810\") " pod="openshift-console/console-6bb9b9c769-qvc5m"
Jul 09 09:36:40.771683 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:40.771615    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e46e2eb-e134-40d6-b82f-141c40f92810-service-ca\") pod \"console-6bb9b9c769-qvc5m\" (UID: \"0e46e2eb-e134-40d6-b82f-141c40f92810\") " pod="openshift-console/console-6bb9b9c769-qvc5m"
Jul 09 09:36:40.772240 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:40.772218    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e46e2eb-e134-40d6-b82f-141c40f92810-service-ca\") pod \"console-6bb9b9c769-qvc5m\" (UID: \"0e46e2eb-e134-40d6-b82f-141c40f92810\") " pod="openshift-console/console-6bb9b9c769-qvc5m"
Jul 09 09:36:40.772366 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:40.772251    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e46e2eb-e134-40d6-b82f-141c40f92810-oauth-serving-cert\") pod \"console-6bb9b9c769-qvc5m\" (UID: \"0e46e2eb-e134-40d6-b82f-141c40f92810\") " pod="openshift-console/console-6bb9b9c769-qvc5m"
Jul 09 09:36:40.772433 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:40.772381    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e46e2eb-e134-40d6-b82f-141c40f92810-console-config\") pod \"console-6bb9b9c769-qvc5m\" (UID: \"0e46e2eb-e134-40d6-b82f-141c40f92810\") " pod="openshift-console/console-6bb9b9c769-qvc5m"
Jul 09 09:36:40.772542 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:40.772522    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e46e2eb-e134-40d6-b82f-141c40f92810-trusted-ca-bundle\") pod \"console-6bb9b9c769-qvc5m\" (UID: \"0e46e2eb-e134-40d6-b82f-141c40f92810\") " pod="openshift-console/console-6bb9b9c769-qvc5m"
Jul 09 09:36:40.773914 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:40.773893    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e46e2eb-e134-40d6-b82f-141c40f92810-console-serving-cert\") pod \"console-6bb9b9c769-qvc5m\" (UID: \"0e46e2eb-e134-40d6-b82f-141c40f92810\") " pod="openshift-console/console-6bb9b9c769-qvc5m"
Jul 09 09:36:40.773997 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:40.773919    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e46e2eb-e134-40d6-b82f-141c40f92810-console-oauth-config\") pod \"console-6bb9b9c769-qvc5m\" (UID: \"0e46e2eb-e134-40d6-b82f-141c40f92810\") " pod="openshift-console/console-6bb9b9c769-qvc5m"
Jul 09 09:36:40.783057 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:40.783007    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xvkd\" (UniqueName: \"kubernetes.io/projected/0e46e2eb-e134-40d6-b82f-141c40f92810-kube-api-access-5xvkd\") pod \"console-6bb9b9c769-qvc5m\" (UID: \"0e46e2eb-e134-40d6-b82f-141c40f92810\") " pod="openshift-console/console-6bb9b9c769-qvc5m"
Jul 09 09:36:40.916481 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:40.916454    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bb9b9c769-qvc5m"
Jul 09 09:36:41.048348 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:41.048217    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bb9b9c769-qvc5m"]
Jul 09 09:36:41.050930 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:36:41.050906    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e46e2eb_e134_40d6_b82f_141c40f92810.slice/crio-2bfec7ac37724c867533413eedeecd74d7003d009e228c71f4f0bdee5736c824 WatchSource:0}: Error finding container 2bfec7ac37724c867533413eedeecd74d7003d009e228c71f4f0bdee5736c824: Status 404 returned error can't find the container with id 2bfec7ac37724c867533413eedeecd74d7003d009e228c71f4f0bdee5736c824
Jul 09 09:36:41.193971 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:41.193929    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bb9b9c769-qvc5m" event={"ID":"0e46e2eb-e134-40d6-b82f-141c40f92810","Type":"ContainerStarted","Data":"0175854ca8edfd27c74ce686f0d1dae2dc3243732db9243b6fbbfd604ca7dd17"}
Jul 09 09:36:41.194122 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:41.194021    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bb9b9c769-qvc5m" event={"ID":"0e46e2eb-e134-40d6-b82f-141c40f92810","Type":"ContainerStarted","Data":"2bfec7ac37724c867533413eedeecd74d7003d009e228c71f4f0bdee5736c824"}
Jul 09 09:36:41.214671 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:41.214622    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6bb9b9c769-qvc5m" podStartSLOduration=1.214605992 podStartE2EDuration="1.214605992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:36:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:36:41.212580095 +0000 UTC m=+159.383738652" watchObservedRunningTime="2026-07-09 09:36:41.214605992 +0000 UTC m=+159.385764548"
Jul 09 09:36:50.916852 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:50.916816    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6bb9b9c769-qvc5m"
Jul 09 09:36:50.917245 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:50.916893    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6bb9b9c769-qvc5m"
Jul 09 09:36:50.921733 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:50.921710    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6bb9b9c769-qvc5m"
Jul 09 09:36:51.238785 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:51.238761    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6bb9b9c769-qvc5m"
Jul 09 09:36:51.287480 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:36:51.287450    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c445df65c-kmk8b"]
Jul 09 09:37:16.307784 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:16.307718    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7c445df65c-kmk8b" podUID="1f7e57cf-4c3b-49ee-9fee-3849eeb462ef" containerName="console" containerID="cri-o://fa6e0dc90cc051b787346f06926fa223deb435e6fce60d1227d92c3ca9563e01" gracePeriod=15
Jul 09 09:37:16.548986 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:16.548964    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c445df65c-kmk8b"
Jul 09 09:37:16.585393 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:16.585333    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pdb4w\" (UniqueName: \"kubernetes.io/projected/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-kube-api-access-pdb4w\") pod \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\" (UID: \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\") "
Jul 09 09:37:16.585393 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:16.585380    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-console-oauth-config\") pod \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\" (UID: \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\") "
Jul 09 09:37:16.585547 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:16.585408    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-console-serving-cert\") pod \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\" (UID: \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\") "
Jul 09 09:37:16.585547 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:16.585473    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-oauth-serving-cert\") pod \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\" (UID: \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\") "
Jul 09 09:37:16.585547 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:16.585507    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-service-ca\") pod \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\" (UID: \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\") "
Jul 09 09:37:16.585691 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:16.585564    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-trusted-ca-bundle\") pod \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\" (UID: \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\") "
Jul 09 09:37:16.585691 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:16.585600    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-console-config\") pod \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\" (UID: \"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef\") "
Jul 09 09:37:16.585916 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:16.585888    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-service-ca" (OuterVolumeSpecName: "service-ca") pod "1f7e57cf-4c3b-49ee-9fee-3849eeb462ef" (UID: "1f7e57cf-4c3b-49ee-9fee-3849eeb462ef"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:37:16.585997 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:16.585882    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1f7e57cf-4c3b-49ee-9fee-3849eeb462ef" (UID: "1f7e57cf-4c3b-49ee-9fee-3849eeb462ef"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:37:16.586121 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:16.586085    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1f7e57cf-4c3b-49ee-9fee-3849eeb462ef" (UID: "1f7e57cf-4c3b-49ee-9fee-3849eeb462ef"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:37:16.586209 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:16.586187    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-console-config" (OuterVolumeSpecName: "console-config") pod "1f7e57cf-4c3b-49ee-9fee-3849eeb462ef" (UID: "1f7e57cf-4c3b-49ee-9fee-3849eeb462ef"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:37:16.587574 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:16.587548    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1f7e57cf-4c3b-49ee-9fee-3849eeb462ef" (UID: "1f7e57cf-4c3b-49ee-9fee-3849eeb462ef"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:37:16.587739 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:16.587721    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1f7e57cf-4c3b-49ee-9fee-3849eeb462ef" (UID: "1f7e57cf-4c3b-49ee-9fee-3849eeb462ef"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:37:16.587945 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:16.587923    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-kube-api-access-pdb4w" (OuterVolumeSpecName: "kube-api-access-pdb4w") pod "1f7e57cf-4c3b-49ee-9fee-3849eeb462ef" (UID: "1f7e57cf-4c3b-49ee-9fee-3849eeb462ef"). InnerVolumeSpecName "kube-api-access-pdb4w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:37:16.686689 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:16.686665    2549 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-trusted-ca-bundle\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:37:16.686689 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:16.686685    2549 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-console-config\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:37:16.686826 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:16.686696    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdb4w\" (UniqueName: \"kubernetes.io/projected/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-kube-api-access-pdb4w\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:37:16.686826 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:16.686705    2549 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-console-oauth-config\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:37:16.686826 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:16.686713    2549 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-console-serving-cert\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:37:16.686826 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:16.686723    2549 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-oauth-serving-cert\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:37:16.686826 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:16.686731    2549 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef-service-ca\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:37:17.339166 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:17.339129    2549 generic.go:356] "Generic (PLEG): container finished" podID="1f7e57cf-4c3b-49ee-9fee-3849eeb462ef" containerID="fa6e0dc90cc051b787346f06926fa223deb435e6fce60d1227d92c3ca9563e01" exitCode=0
Jul 09 09:37:17.339560 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:17.339191    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c445df65c-kmk8b"
Jul 09 09:37:17.339560 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:17.339207    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c445df65c-kmk8b" event={"ID":"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef","Type":"ContainerDied","Data":"fa6e0dc90cc051b787346f06926fa223deb435e6fce60d1227d92c3ca9563e01"}
Jul 09 09:37:17.339560 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:17.339270    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c445df65c-kmk8b" event={"ID":"1f7e57cf-4c3b-49ee-9fee-3849eeb462ef","Type":"ContainerDied","Data":"ecdff7bf2b46728b05f563e2ba72dd17efe29eb5edb6e37e1002cb55d081ad34"}
Jul 09 09:37:17.339560 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:17.339289    2549 scope.go:117] "RemoveContainer" containerID="fa6e0dc90cc051b787346f06926fa223deb435e6fce60d1227d92c3ca9563e01"
Jul 09 09:37:17.348280 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:17.348261    2549 scope.go:117] "RemoveContainer" containerID="fa6e0dc90cc051b787346f06926fa223deb435e6fce60d1227d92c3ca9563e01"
Jul 09 09:37:17.348524 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:37:17.348506    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa6e0dc90cc051b787346f06926fa223deb435e6fce60d1227d92c3ca9563e01\": container with ID starting with fa6e0dc90cc051b787346f06926fa223deb435e6fce60d1227d92c3ca9563e01 not found: ID does not exist" containerID="fa6e0dc90cc051b787346f06926fa223deb435e6fce60d1227d92c3ca9563e01"
Jul 09 09:37:17.348565 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:17.348533    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa6e0dc90cc051b787346f06926fa223deb435e6fce60d1227d92c3ca9563e01"} err="failed to get container status \"fa6e0dc90cc051b787346f06926fa223deb435e6fce60d1227d92c3ca9563e01\": rpc error: code = NotFound desc = could not find container \"fa6e0dc90cc051b787346f06926fa223deb435e6fce60d1227d92c3ca9563e01\": container with ID starting with fa6e0dc90cc051b787346f06926fa223deb435e6fce60d1227d92c3ca9563e01 not found: ID does not exist"
Jul 09 09:37:17.362052 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:17.362031    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c445df65c-kmk8b"]
Jul 09 09:37:17.365982 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:17.365963    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c445df65c-kmk8b"]
Jul 09 09:37:18.384417 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:18.384386    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f7e57cf-4c3b-49ee-9fee-3849eeb462ef" path="/var/lib/kubelet/pods/1f7e57cf-4c3b-49ee-9fee-3849eeb462ef/volumes"
Jul 09 09:37:35.525035 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:35.525004    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:37:35.539567 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:35.539545    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:37:36.427553 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:36.427528    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 09:37:47.355415 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:47.355386    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b7968f45-rtl4p"]
Jul 09 09:37:47.355825 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:47.355713    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f7e57cf-4c3b-49ee-9fee-3849eeb462ef" containerName="console"
Jul 09 09:37:47.355825 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:47.355723    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f7e57cf-4c3b-49ee-9fee-3849eeb462ef" containerName="console"
Jul 09 09:37:47.355825 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:47.355793    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f7e57cf-4c3b-49ee-9fee-3849eeb462ef" containerName="console"
Jul 09 09:37:47.360188 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:47.360169    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7968f45-rtl4p"
Jul 09 09:37:47.372925 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:47.372897    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b7968f45-rtl4p"]
Jul 09 09:37:47.427499 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:47.427469    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7b06680b-d594-40f8-a490-5cc4ba36e37b-console-oauth-config\") pod \"console-6b7968f45-rtl4p\" (UID: \"7b06680b-d594-40f8-a490-5cc4ba36e37b\") " pod="openshift-console/console-6b7968f45-rtl4p"
Jul 09 09:37:47.427499 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:47.427498    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7b06680b-d594-40f8-a490-5cc4ba36e37b-service-ca\") pod \"console-6b7968f45-rtl4p\" (UID: \"7b06680b-d594-40f8-a490-5cc4ba36e37b\") " pod="openshift-console/console-6b7968f45-rtl4p"
Jul 09 09:37:47.427702 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:47.427651    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7b06680b-d594-40f8-a490-5cc4ba36e37b-console-config\") pod \"console-6b7968f45-rtl4p\" (UID: \"7b06680b-d594-40f8-a490-5cc4ba36e37b\") " pod="openshift-console/console-6b7968f45-rtl4p"
Jul 09 09:37:47.427760 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:47.427728    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt6n2\" (UniqueName: \"kubernetes.io/projected/7b06680b-d594-40f8-a490-5cc4ba36e37b-kube-api-access-jt6n2\") pod \"console-6b7968f45-rtl4p\" (UID: \"7b06680b-d594-40f8-a490-5cc4ba36e37b\") " pod="openshift-console/console-6b7968f45-rtl4p"
Jul 09 09:37:47.427819 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:47.427801    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b06680b-d594-40f8-a490-5cc4ba36e37b-console-serving-cert\") pod \"console-6b7968f45-rtl4p\" (UID: \"7b06680b-d594-40f8-a490-5cc4ba36e37b\") " pod="openshift-console/console-6b7968f45-rtl4p"
Jul 09 09:37:47.427898 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:47.427828    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b06680b-d594-40f8-a490-5cc4ba36e37b-trusted-ca-bundle\") pod \"console-6b7968f45-rtl4p\" (UID: \"7b06680b-d594-40f8-a490-5cc4ba36e37b\") " pod="openshift-console/console-6b7968f45-rtl4p"
Jul 09 09:37:47.428080 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:47.428047    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7b06680b-d594-40f8-a490-5cc4ba36e37b-oauth-serving-cert\") pod \"console-6b7968f45-rtl4p\" (UID: \"7b06680b-d594-40f8-a490-5cc4ba36e37b\") " pod="openshift-console/console-6b7968f45-rtl4p"
Jul 09 09:37:47.529191 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:47.529165    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7b06680b-d594-40f8-a490-5cc4ba36e37b-oauth-serving-cert\") pod \"console-6b7968f45-rtl4p\" (UID: \"7b06680b-d594-40f8-a490-5cc4ba36e37b\") " pod="openshift-console/console-6b7968f45-rtl4p"
Jul 09 09:37:47.529313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:47.529196    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7b06680b-d594-40f8-a490-5cc4ba36e37b-console-oauth-config\") pod \"console-6b7968f45-rtl4p\" (UID: \"7b06680b-d594-40f8-a490-5cc4ba36e37b\") " pod="openshift-console/console-6b7968f45-rtl4p"
Jul 09 09:37:47.529313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:47.529210    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7b06680b-d594-40f8-a490-5cc4ba36e37b-service-ca\") pod \"console-6b7968f45-rtl4p\" (UID: \"7b06680b-d594-40f8-a490-5cc4ba36e37b\") " pod="openshift-console/console-6b7968f45-rtl4p"
Jul 09 09:37:47.529313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:47.529233    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7b06680b-d594-40f8-a490-5cc4ba36e37b-console-config\") pod \"console-6b7968f45-rtl4p\" (UID: \"7b06680b-d594-40f8-a490-5cc4ba36e37b\") " pod="openshift-console/console-6b7968f45-rtl4p"
Jul 09 09:37:47.529313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:47.529270    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt6n2\" (UniqueName: \"kubernetes.io/projected/7b06680b-d594-40f8-a490-5cc4ba36e37b-kube-api-access-jt6n2\") pod \"console-6b7968f45-rtl4p\" (UID: \"7b06680b-d594-40f8-a490-5cc4ba36e37b\") " pod="openshift-console/console-6b7968f45-rtl4p"
Jul 09 09:37:47.529313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:47.529304    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b06680b-d594-40f8-a490-5cc4ba36e37b-console-serving-cert\") pod \"console-6b7968f45-rtl4p\" (UID: \"7b06680b-d594-40f8-a490-5cc4ba36e37b\") " pod="openshift-console/console-6b7968f45-rtl4p"
Jul 09 09:37:47.529538 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:47.529329    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b06680b-d594-40f8-a490-5cc4ba36e37b-trusted-ca-bundle\") pod \"console-6b7968f45-rtl4p\" (UID: \"7b06680b-d594-40f8-a490-5cc4ba36e37b\") " pod="openshift-console/console-6b7968f45-rtl4p"
Jul 09 09:37:47.529975 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:47.529947    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7b06680b-d594-40f8-a490-5cc4ba36e37b-oauth-serving-cert\") pod \"console-6b7968f45-rtl4p\" (UID: \"7b06680b-d594-40f8-a490-5cc4ba36e37b\") " pod="openshift-console/console-6b7968f45-rtl4p"
Jul 09 09:37:47.530092 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:47.530007    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7b06680b-d594-40f8-a490-5cc4ba36e37b-service-ca\") pod \"console-6b7968f45-rtl4p\" (UID: \"7b06680b-d594-40f8-a490-5cc4ba36e37b\") " pod="openshift-console/console-6b7968f45-rtl4p"
Jul 09 09:37:47.530092 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:47.530013    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7b06680b-d594-40f8-a490-5cc4ba36e37b-console-config\") pod \"console-6b7968f45-rtl4p\" (UID: \"7b06680b-d594-40f8-a490-5cc4ba36e37b\") " pod="openshift-console/console-6b7968f45-rtl4p"
Jul 09 09:37:47.530211 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:47.530130    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b06680b-d594-40f8-a490-5cc4ba36e37b-trusted-ca-bundle\") pod \"console-6b7968f45-rtl4p\" (UID: \"7b06680b-d594-40f8-a490-5cc4ba36e37b\") " pod="openshift-console/console-6b7968f45-rtl4p"
Jul 09 09:37:47.531755 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:47.531725    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7b06680b-d594-40f8-a490-5cc4ba36e37b-console-oauth-config\") pod \"console-6b7968f45-rtl4p\" (UID: \"7b06680b-d594-40f8-a490-5cc4ba36e37b\") " pod="openshift-console/console-6b7968f45-rtl4p"
Jul 09 09:37:47.531755 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:47.531751    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b06680b-d594-40f8-a490-5cc4ba36e37b-console-serving-cert\") pod \"console-6b7968f45-rtl4p\" (UID: \"7b06680b-d594-40f8-a490-5cc4ba36e37b\") " pod="openshift-console/console-6b7968f45-rtl4p"
Jul 09 09:37:47.537811 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:47.537792    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt6n2\" (UniqueName: \"kubernetes.io/projected/7b06680b-d594-40f8-a490-5cc4ba36e37b-kube-api-access-jt6n2\") pod \"console-6b7968f45-rtl4p\" (UID: \"7b06680b-d594-40f8-a490-5cc4ba36e37b\") " pod="openshift-console/console-6b7968f45-rtl4p"
Jul 09 09:37:47.669129 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:47.669066    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7968f45-rtl4p"
Jul 09 09:37:47.797666 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:47.797640    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b7968f45-rtl4p"]
Jul 09 09:37:47.799738 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:37:47.799709    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b06680b_d594_40f8_a490_5cc4ba36e37b.slice/crio-08229479a40f31a32f06498faaba650fb4fca217ab7e38fa135312192a633319 WatchSource:0}: Error finding container 08229479a40f31a32f06498faaba650fb4fca217ab7e38fa135312192a633319: Status 404 returned error can't find the container with id 08229479a40f31a32f06498faaba650fb4fca217ab7e38fa135312192a633319
Jul 09 09:37:48.460274 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:48.460241    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7968f45-rtl4p" event={"ID":"7b06680b-d594-40f8-a490-5cc4ba36e37b","Type":"ContainerStarted","Data":"06974d8547018eeb9e60009a3e75b01b25cc3ed3e53dbfadfdf30890eded4516"}
Jul 09 09:37:48.460640 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:48.460294    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7968f45-rtl4p" event={"ID":"7b06680b-d594-40f8-a490-5cc4ba36e37b","Type":"ContainerStarted","Data":"08229479a40f31a32f06498faaba650fb4fca217ab7e38fa135312192a633319"}
Jul 09 09:37:48.494824 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:48.494780    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b7968f45-rtl4p" podStartSLOduration=1.4947657699999999 podStartE2EDuration="1.49476577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:37:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:37:48.494412696 +0000 UTC m=+226.665571252" watchObservedRunningTime="2026-07-09 09:37:48.49476577 +0000 UTC m=+226.665924326"
Jul 09 09:37:57.669825 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:57.669780    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b7968f45-rtl4p"
Jul 09 09:37:57.669825 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:57.669839    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b7968f45-rtl4p"
Jul 09 09:37:57.674545 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:57.674525    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b7968f45-rtl4p"
Jul 09 09:37:58.505188 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:58.505164    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b7968f45-rtl4p"
Jul 09 09:37:58.553387 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:37:58.553352    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bb9b9c769-qvc5m"]
Jul 09 09:38:23.580522 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:23.580464    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6bb9b9c769-qvc5m" podUID="0e46e2eb-e134-40d6-b82f-141c40f92810" containerName="console" containerID="cri-o://0175854ca8edfd27c74ce686f0d1dae2dc3243732db9243b6fbbfd604ca7dd17" gracePeriod=15
Jul 09 09:38:23.816809 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:23.816784    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bb9b9c769-qvc5m"
Jul 09 09:38:23.827915 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:23.827887    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e46e2eb-e134-40d6-b82f-141c40f92810-trusted-ca-bundle\") pod \"0e46e2eb-e134-40d6-b82f-141c40f92810\" (UID: \"0e46e2eb-e134-40d6-b82f-141c40f92810\") "
Jul 09 09:38:23.828032 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:23.827927    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xvkd\" (UniqueName: \"kubernetes.io/projected/0e46e2eb-e134-40d6-b82f-141c40f92810-kube-api-access-5xvkd\") pod \"0e46e2eb-e134-40d6-b82f-141c40f92810\" (UID: \"0e46e2eb-e134-40d6-b82f-141c40f92810\") "
Jul 09 09:38:23.828032 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:23.827993    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e46e2eb-e134-40d6-b82f-141c40f92810-console-serving-cert\") pod \"0e46e2eb-e134-40d6-b82f-141c40f92810\" (UID: \"0e46e2eb-e134-40d6-b82f-141c40f92810\") "
Jul 09 09:38:23.828032 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:23.828019    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e46e2eb-e134-40d6-b82f-141c40f92810-console-oauth-config\") pod \"0e46e2eb-e134-40d6-b82f-141c40f92810\" (UID: \"0e46e2eb-e134-40d6-b82f-141c40f92810\") "
Jul 09 09:38:23.828185 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:23.828154    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e46e2eb-e134-40d6-b82f-141c40f92810-console-config\") pod \"0e46e2eb-e134-40d6-b82f-141c40f92810\" (UID: \"0e46e2eb-e134-40d6-b82f-141c40f92810\") "
Jul 09 09:38:23.828243 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:23.828184    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e46e2eb-e134-40d6-b82f-141c40f92810-oauth-serving-cert\") pod \"0e46e2eb-e134-40d6-b82f-141c40f92810\" (UID: \"0e46e2eb-e134-40d6-b82f-141c40f92810\") "
Jul 09 09:38:23.828243 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:23.828213    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e46e2eb-e134-40d6-b82f-141c40f92810-service-ca\") pod \"0e46e2eb-e134-40d6-b82f-141c40f92810\" (UID: \"0e46e2eb-e134-40d6-b82f-141c40f92810\") "
Jul 09 09:38:23.828343 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:23.828286    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e46e2eb-e134-40d6-b82f-141c40f92810-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0e46e2eb-e134-40d6-b82f-141c40f92810" (UID: "0e46e2eb-e134-40d6-b82f-141c40f92810"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:38:23.828589 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:23.828560    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e46e2eb-e134-40d6-b82f-141c40f92810-console-config" (OuterVolumeSpecName: "console-config") pod "0e46e2eb-e134-40d6-b82f-141c40f92810" (UID: "0e46e2eb-e134-40d6-b82f-141c40f92810"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:38:23.828589 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:23.828566    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e46e2eb-e134-40d6-b82f-141c40f92810-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0e46e2eb-e134-40d6-b82f-141c40f92810" (UID: "0e46e2eb-e134-40d6-b82f-141c40f92810"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:38:23.828735 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:23.828648    2549 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e46e2eb-e134-40d6-b82f-141c40f92810-trusted-ca-bundle\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:38:23.828735 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:23.828700    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e46e2eb-e134-40d6-b82f-141c40f92810-service-ca" (OuterVolumeSpecName: "service-ca") pod "0e46e2eb-e134-40d6-b82f-141c40f92810" (UID: "0e46e2eb-e134-40d6-b82f-141c40f92810"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:38:23.830205 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:23.830181    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e46e2eb-e134-40d6-b82f-141c40f92810-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0e46e2eb-e134-40d6-b82f-141c40f92810" (UID: "0e46e2eb-e134-40d6-b82f-141c40f92810"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:38:23.830291 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:23.830249    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e46e2eb-e134-40d6-b82f-141c40f92810-kube-api-access-5xvkd" (OuterVolumeSpecName: "kube-api-access-5xvkd") pod "0e46e2eb-e134-40d6-b82f-141c40f92810" (UID: "0e46e2eb-e134-40d6-b82f-141c40f92810"). InnerVolumeSpecName "kube-api-access-5xvkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:38:23.830467 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:23.830445    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e46e2eb-e134-40d6-b82f-141c40f92810-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0e46e2eb-e134-40d6-b82f-141c40f92810" (UID: "0e46e2eb-e134-40d6-b82f-141c40f92810"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:38:23.929657 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:23.929592    2549 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e46e2eb-e134-40d6-b82f-141c40f92810-console-config\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:38:23.929657 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:23.929618    2549 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e46e2eb-e134-40d6-b82f-141c40f92810-oauth-serving-cert\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:38:23.929657 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:23.929628    2549 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e46e2eb-e134-40d6-b82f-141c40f92810-service-ca\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:38:23.929657 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:23.929637    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xvkd\" (UniqueName: \"kubernetes.io/projected/0e46e2eb-e134-40d6-b82f-141c40f92810-kube-api-access-5xvkd\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:38:23.929657 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:23.929646    2549 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e46e2eb-e134-40d6-b82f-141c40f92810-console-serving-cert\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:38:23.929657 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:23.929654    2549 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e46e2eb-e134-40d6-b82f-141c40f92810-console-oauth-config\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:38:24.615575 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:24.615540    2549 generic.go:356] "Generic (PLEG): container finished" podID="0e46e2eb-e134-40d6-b82f-141c40f92810" containerID="0175854ca8edfd27c74ce686f0d1dae2dc3243732db9243b6fbbfd604ca7dd17" exitCode=0
Jul 09 09:38:24.615992 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:24.615591    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bb9b9c769-qvc5m"
Jul 09 09:38:24.615992 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:24.615602    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bb9b9c769-qvc5m" event={"ID":"0e46e2eb-e134-40d6-b82f-141c40f92810","Type":"ContainerDied","Data":"0175854ca8edfd27c74ce686f0d1dae2dc3243732db9243b6fbbfd604ca7dd17"}
Jul 09 09:38:24.615992 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:24.615641    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bb9b9c769-qvc5m" event={"ID":"0e46e2eb-e134-40d6-b82f-141c40f92810","Type":"ContainerDied","Data":"2bfec7ac37724c867533413eedeecd74d7003d009e228c71f4f0bdee5736c824"}
Jul 09 09:38:24.615992 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:24.615663    2549 scope.go:117] "RemoveContainer" containerID="0175854ca8edfd27c74ce686f0d1dae2dc3243732db9243b6fbbfd604ca7dd17"
Jul 09 09:38:24.624293 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:24.624277    2549 scope.go:117] "RemoveContainer" containerID="0175854ca8edfd27c74ce686f0d1dae2dc3243732db9243b6fbbfd604ca7dd17"
Jul 09 09:38:24.625007 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:38:24.624989    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0175854ca8edfd27c74ce686f0d1dae2dc3243732db9243b6fbbfd604ca7dd17\": container with ID starting with 0175854ca8edfd27c74ce686f0d1dae2dc3243732db9243b6fbbfd604ca7dd17 not found: ID does not exist" containerID="0175854ca8edfd27c74ce686f0d1dae2dc3243732db9243b6fbbfd604ca7dd17"
Jul 09 09:38:24.625092 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:24.625020    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0175854ca8edfd27c74ce686f0d1dae2dc3243732db9243b6fbbfd604ca7dd17"} err="failed to get container status \"0175854ca8edfd27c74ce686f0d1dae2dc3243732db9243b6fbbfd604ca7dd17\": rpc error: code = NotFound desc = could not find container \"0175854ca8edfd27c74ce686f0d1dae2dc3243732db9243b6fbbfd604ca7dd17\": container with ID starting with 0175854ca8edfd27c74ce686f0d1dae2dc3243732db9243b6fbbfd604ca7dd17 not found: ID does not exist"
Jul 09 09:38:24.632362 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:24.632342    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bb9b9c769-qvc5m"]
Jul 09 09:38:24.638007 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:24.637986    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6bb9b9c769-qvc5m"]
Jul 09 09:38:26.385231 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:38:26.385196    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e46e2eb-e134-40d6-b82f-141c40f92810" path="/var/lib/kubelet/pods/0e46e2eb-e134-40d6-b82f-141c40f92810/volumes"
Jul 09 09:39:02.260989 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:39:02.260962    2549 kubelet.go:1648] "Image garbage collection succeeded"
Jul 09 09:41:51.115003 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:51.114969    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8bcd4cfd5-7szgp"]
Jul 09 09:41:51.115415 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:51.115286    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e46e2eb-e134-40d6-b82f-141c40f92810" containerName="console"
Jul 09 09:41:51.115415 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:51.115297    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e46e2eb-e134-40d6-b82f-141c40f92810" containerName="console"
Jul 09 09:41:51.115415 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:51.115362    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e46e2eb-e134-40d6-b82f-141c40f92810" containerName="console"
Jul 09 09:41:51.118315 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:51.118295    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8bcd4cfd5-7szgp"
Jul 09 09:41:51.130928 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:51.130906    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8bcd4cfd5-7szgp"]
Jul 09 09:41:51.158249 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:51.158225    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e57c6981-978d-44fc-9f55-c20077bce0bc-console-serving-cert\") pod \"console-8bcd4cfd5-7szgp\" (UID: \"e57c6981-978d-44fc-9f55-c20077bce0bc\") " pod="openshift-console/console-8bcd4cfd5-7szgp"
Jul 09 09:41:51.158359 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:51.158257    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e57c6981-978d-44fc-9f55-c20077bce0bc-console-config\") pod \"console-8bcd4cfd5-7szgp\" (UID: \"e57c6981-978d-44fc-9f55-c20077bce0bc\") " pod="openshift-console/console-8bcd4cfd5-7szgp"
Jul 09 09:41:51.158359 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:51.158284    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e57c6981-978d-44fc-9f55-c20077bce0bc-trusted-ca-bundle\") pod \"console-8bcd4cfd5-7szgp\" (UID: \"e57c6981-978d-44fc-9f55-c20077bce0bc\") " pod="openshift-console/console-8bcd4cfd5-7szgp"
Jul 09 09:41:51.158469 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:51.158388    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e57c6981-978d-44fc-9f55-c20077bce0bc-oauth-serving-cert\") pod \"console-8bcd4cfd5-7szgp\" (UID: \"e57c6981-978d-44fc-9f55-c20077bce0bc\") " pod="openshift-console/console-8bcd4cfd5-7szgp"
Jul 09 09:41:51.158506 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:51.158478    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e57c6981-978d-44fc-9f55-c20077bce0bc-console-oauth-config\") pod \"console-8bcd4cfd5-7szgp\" (UID: \"e57c6981-978d-44fc-9f55-c20077bce0bc\") " pod="openshift-console/console-8bcd4cfd5-7szgp"
Jul 09 09:41:51.158546 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:51.158516    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e57c6981-978d-44fc-9f55-c20077bce0bc-service-ca\") pod \"console-8bcd4cfd5-7szgp\" (UID: \"e57c6981-978d-44fc-9f55-c20077bce0bc\") " pod="openshift-console/console-8bcd4cfd5-7szgp"
Jul 09 09:41:51.158581 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:51.158548    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv2zg\" (UniqueName: \"kubernetes.io/projected/e57c6981-978d-44fc-9f55-c20077bce0bc-kube-api-access-dv2zg\") pod \"console-8bcd4cfd5-7szgp\" (UID: \"e57c6981-978d-44fc-9f55-c20077bce0bc\") " pod="openshift-console/console-8bcd4cfd5-7szgp"
Jul 09 09:41:51.259185 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:51.259149    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv2zg\" (UniqueName: \"kubernetes.io/projected/e57c6981-978d-44fc-9f55-c20077bce0bc-kube-api-access-dv2zg\") pod \"console-8bcd4cfd5-7szgp\" (UID: \"e57c6981-978d-44fc-9f55-c20077bce0bc\") " pod="openshift-console/console-8bcd4cfd5-7szgp"
Jul 09 09:41:51.259302 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:51.259212    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e57c6981-978d-44fc-9f55-c20077bce0bc-console-serving-cert\") pod \"console-8bcd4cfd5-7szgp\" (UID: \"e57c6981-978d-44fc-9f55-c20077bce0bc\") " pod="openshift-console/console-8bcd4cfd5-7szgp"
Jul 09 09:41:51.259302 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:51.259237    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e57c6981-978d-44fc-9f55-c20077bce0bc-console-config\") pod \"console-8bcd4cfd5-7szgp\" (UID: \"e57c6981-978d-44fc-9f55-c20077bce0bc\") " pod="openshift-console/console-8bcd4cfd5-7szgp"
Jul 09 09:41:51.259302 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:51.259265    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e57c6981-978d-44fc-9f55-c20077bce0bc-trusted-ca-bundle\") pod \"console-8bcd4cfd5-7szgp\" (UID: \"e57c6981-978d-44fc-9f55-c20077bce0bc\") " pod="openshift-console/console-8bcd4cfd5-7szgp"
Jul 09 09:41:51.259463 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:51.259311    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e57c6981-978d-44fc-9f55-c20077bce0bc-oauth-serving-cert\") pod \"console-8bcd4cfd5-7szgp\" (UID: \"e57c6981-978d-44fc-9f55-c20077bce0bc\") " pod="openshift-console/console-8bcd4cfd5-7szgp"
Jul 09 09:41:51.259463 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:51.259374    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e57c6981-978d-44fc-9f55-c20077bce0bc-console-oauth-config\") pod \"console-8bcd4cfd5-7szgp\" (UID: \"e57c6981-978d-44fc-9f55-c20077bce0bc\") " pod="openshift-console/console-8bcd4cfd5-7szgp"
Jul 09 09:41:51.259463 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:51.259408    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e57c6981-978d-44fc-9f55-c20077bce0bc-service-ca\") pod \"console-8bcd4cfd5-7szgp\" (UID: \"e57c6981-978d-44fc-9f55-c20077bce0bc\") " pod="openshift-console/console-8bcd4cfd5-7szgp"
Jul 09 09:41:51.260053 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:51.260030    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e57c6981-978d-44fc-9f55-c20077bce0bc-console-config\") pod \"console-8bcd4cfd5-7szgp\" (UID: \"e57c6981-978d-44fc-9f55-c20077bce0bc\") " pod="openshift-console/console-8bcd4cfd5-7szgp"
Jul 09 09:41:51.260151 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:51.260115    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e57c6981-978d-44fc-9f55-c20077bce0bc-service-ca\") pod \"console-8bcd4cfd5-7szgp\" (UID: \"e57c6981-978d-44fc-9f55-c20077bce0bc\") " pod="openshift-console/console-8bcd4cfd5-7szgp"
Jul 09 09:41:51.260151 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:51.260115    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e57c6981-978d-44fc-9f55-c20077bce0bc-oauth-serving-cert\") pod \"console-8bcd4cfd5-7szgp\" (UID: \"e57c6981-978d-44fc-9f55-c20077bce0bc\") " pod="openshift-console/console-8bcd4cfd5-7szgp"
Jul 09 09:41:51.260262 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:51.260159    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e57c6981-978d-44fc-9f55-c20077bce0bc-trusted-ca-bundle\") pod \"console-8bcd4cfd5-7szgp\" (UID: \"e57c6981-978d-44fc-9f55-c20077bce0bc\") " pod="openshift-console/console-8bcd4cfd5-7szgp"
Jul 09 09:41:51.261640 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:51.261616    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e57c6981-978d-44fc-9f55-c20077bce0bc-console-oauth-config\") pod \"console-8bcd4cfd5-7szgp\" (UID: \"e57c6981-978d-44fc-9f55-c20077bce0bc\") " pod="openshift-console/console-8bcd4cfd5-7szgp"
Jul 09 09:41:51.261716 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:51.261699    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e57c6981-978d-44fc-9f55-c20077bce0bc-console-serving-cert\") pod \"console-8bcd4cfd5-7szgp\" (UID: \"e57c6981-978d-44fc-9f55-c20077bce0bc\") " pod="openshift-console/console-8bcd4cfd5-7szgp"
Jul 09 09:41:51.266947 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:51.266925    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv2zg\" (UniqueName: \"kubernetes.io/projected/e57c6981-978d-44fc-9f55-c20077bce0bc-kube-api-access-dv2zg\") pod \"console-8bcd4cfd5-7szgp\" (UID: \"e57c6981-978d-44fc-9f55-c20077bce0bc\") " pod="openshift-console/console-8bcd4cfd5-7szgp"
Jul 09 09:41:51.426955 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:51.426855    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8bcd4cfd5-7szgp"
Jul 09 09:41:51.552293 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:51.552271    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8bcd4cfd5-7szgp"]
Jul 09 09:41:51.554711 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:41:51.554685    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode57c6981_978d_44fc_9f55_c20077bce0bc.slice/crio-7b81b44a005a30df826132beea97e232152594196901d3c8dd86eb2c5cb17fd2 WatchSource:0}: Error finding container 7b81b44a005a30df826132beea97e232152594196901d3c8dd86eb2c5cb17fd2: Status 404 returned error can't find the container with id 7b81b44a005a30df826132beea97e232152594196901d3c8dd86eb2c5cb17fd2
Jul 09 09:41:52.435258 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:52.435221    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8bcd4cfd5-7szgp" event={"ID":"e57c6981-978d-44fc-9f55-c20077bce0bc","Type":"ContainerStarted","Data":"8e04425f29104c86e991e4f0e53e151c70b779486bc9bf8e4ac19b338be948ba"}
Jul 09 09:41:52.435660 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:52.435305    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8bcd4cfd5-7szgp" event={"ID":"e57c6981-978d-44fc-9f55-c20077bce0bc","Type":"ContainerStarted","Data":"7b81b44a005a30df826132beea97e232152594196901d3c8dd86eb2c5cb17fd2"}
Jul 09 09:41:52.459981 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:41:52.459923    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8bcd4cfd5-7szgp" podStartSLOduration=1.459905781 podStartE2EDuration="1.459905781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:41:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:41:52.459788005 +0000 UTC m=+470.630946562" watchObservedRunningTime="2026-07-09 09:41:52.459905781 +0000 UTC m=+470.631064329"
Jul 09 09:42:00.965691 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:00.965655    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-dmpbt"]
Jul 09 09:42:00.969094 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:00.969078    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-dmpbt"
Jul 09 09:42:00.971431 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:00.971403    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Jul 09 09:42:00.972437 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:00.972417    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 09 09:42:00.972437 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:00.972431    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-w9nzs\""
Jul 09 09:42:00.972542 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:00.972417    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 09 09:42:00.975897 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:00.975859    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-dmpbt"]
Jul 09 09:42:01.037076 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:01.037040    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dttpr\" (UniqueName: \"kubernetes.io/projected/2e836b72-ca73-496d-a127-2631e76ed06e-kube-api-access-dttpr\") pod \"s3-init-dmpbt\" (UID: \"2e836b72-ca73-496d-a127-2631e76ed06e\") " pod="kserve/s3-init-dmpbt"
Jul 09 09:42:01.037231 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:01.037160    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e836b72-ca73-496d-a127-2631e76ed06e-model-cache\") pod \"s3-init-dmpbt\" (UID: \"2e836b72-ca73-496d-a127-2631e76ed06e\") " pod="kserve/s3-init-dmpbt"
Jul 09 09:42:01.138450 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:01.138400    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e836b72-ca73-496d-a127-2631e76ed06e-model-cache\") pod \"s3-init-dmpbt\" (UID: \"2e836b72-ca73-496d-a127-2631e76ed06e\") " pod="kserve/s3-init-dmpbt"
Jul 09 09:42:01.138612 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:01.138490    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dttpr\" (UniqueName: \"kubernetes.io/projected/2e836b72-ca73-496d-a127-2631e76ed06e-kube-api-access-dttpr\") pod \"s3-init-dmpbt\" (UID: \"2e836b72-ca73-496d-a127-2631e76ed06e\") " pod="kserve/s3-init-dmpbt"
Jul 09 09:42:01.138818 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:01.138797    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e836b72-ca73-496d-a127-2631e76ed06e-model-cache\") pod \"s3-init-dmpbt\" (UID: \"2e836b72-ca73-496d-a127-2631e76ed06e\") " pod="kserve/s3-init-dmpbt"
Jul 09 09:42:01.147267 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:01.147239    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dttpr\" (UniqueName: \"kubernetes.io/projected/2e836b72-ca73-496d-a127-2631e76ed06e-kube-api-access-dttpr\") pod \"s3-init-dmpbt\" (UID: \"2e836b72-ca73-496d-a127-2631e76ed06e\") " pod="kserve/s3-init-dmpbt"
Jul 09 09:42:01.289250 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:01.289218    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-dmpbt"
Jul 09 09:42:01.419104 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:01.419083    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-dmpbt"]
Jul 09 09:42:01.421920 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:42:01.421894    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e836b72_ca73_496d_a127_2631e76ed06e.slice/crio-671927c81a200bf115e3cf6bc606fab20550ea7c02a9cf974871fcd86a85846c WatchSource:0}: Error finding container 671927c81a200bf115e3cf6bc606fab20550ea7c02a9cf974871fcd86a85846c: Status 404 returned error can't find the container with id 671927c81a200bf115e3cf6bc606fab20550ea7c02a9cf974871fcd86a85846c
Jul 09 09:42:01.424442 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:01.424420    2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 09:42:01.428016 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:01.427995    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8bcd4cfd5-7szgp"
Jul 09 09:42:01.428124 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:01.428058    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8bcd4cfd5-7szgp"
Jul 09 09:42:01.433146 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:01.433126    2549 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8bcd4cfd5-7szgp"
Jul 09 09:42:01.471248 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:01.471218    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-dmpbt" event={"ID":"2e836b72-ca73-496d-a127-2631e76ed06e","Type":"ContainerStarted","Data":"671927c81a200bf115e3cf6bc606fab20550ea7c02a9cf974871fcd86a85846c"}
Jul 09 09:42:01.474967 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:01.474942    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8bcd4cfd5-7szgp"
Jul 09 09:42:01.515857 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:01.515820    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b7968f45-rtl4p"]
Jul 09 09:42:07.501813 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:07.501765    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-dmpbt" event={"ID":"2e836b72-ca73-496d-a127-2631e76ed06e","Type":"ContainerStarted","Data":"69aaf1f3b438e53467e939f16e7339ad1e9eeb7dbc1a46b6fc197b39a83e153c"}
Jul 09 09:42:08.506965 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:08.506937    2549 generic.go:356] "Generic (PLEG): container finished" podID="2e836b72-ca73-496d-a127-2631e76ed06e" containerID="69aaf1f3b438e53467e939f16e7339ad1e9eeb7dbc1a46b6fc197b39a83e153c" exitCode=0
Jul 09 09:42:08.507348 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:08.506992    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-dmpbt" event={"ID":"2e836b72-ca73-496d-a127-2631e76ed06e","Type":"ContainerDied","Data":"69aaf1f3b438e53467e939f16e7339ad1e9eeb7dbc1a46b6fc197b39a83e153c"}
Jul 09 09:42:12.994961 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:12.994938    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Jul 09 09:42:13.534496 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:13.534463    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-dmpbt" event={"ID":"2e836b72-ca73-496d-a127-2631e76ed06e","Type":"ContainerStarted","Data":"d2170337bf0a014389621f2e3c7543ba5fff954747037e27eda60729f9d7d8b6"}
Jul 09 09:42:13.553068 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:13.553022    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-dmpbt" podStartSLOduration=2.566385543 podStartE2EDuration="13.553007393s" totalImagesPullingTime="9.767820852s" totalInitContainerRuntime="1.218800998s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:42:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:42:13.551942114 +0000 UTC m=+491.723100672" watchObservedRunningTime="2026-07-09 09:42:13.553007393 +0000 UTC m=+491.724165949"
Jul 09 09:42:26.536175 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:26.536139    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6b7968f45-rtl4p" podUID="7b06680b-d594-40f8-a490-5cc4ba36e37b" containerName="console" containerID="cri-o://06974d8547018eeb9e60009a3e75b01b25cc3ed3e53dbfadfdf30890eded4516" gracePeriod=15
Jul 09 09:42:27.281107 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.281086    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7968f45-rtl4p"
Jul 09 09:42:27.374160 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.374130    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jt6n2\" (UniqueName: \"kubernetes.io/projected/7b06680b-d594-40f8-a490-5cc4ba36e37b-kube-api-access-jt6n2\") pod \"7b06680b-d594-40f8-a490-5cc4ba36e37b\" (UID: \"7b06680b-d594-40f8-a490-5cc4ba36e37b\") "
Jul 09 09:42:27.374160 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.374162    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7b06680b-d594-40f8-a490-5cc4ba36e37b-oauth-serving-cert\") pod \"7b06680b-d594-40f8-a490-5cc4ba36e37b\" (UID: \"7b06680b-d594-40f8-a490-5cc4ba36e37b\") "
Jul 09 09:42:27.374363 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.374196    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7b06680b-d594-40f8-a490-5cc4ba36e37b-console-oauth-config\") pod \"7b06680b-d594-40f8-a490-5cc4ba36e37b\" (UID: \"7b06680b-d594-40f8-a490-5cc4ba36e37b\") "
Jul 09 09:42:27.374363 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.374216    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b06680b-d594-40f8-a490-5cc4ba36e37b-console-serving-cert\") pod \"7b06680b-d594-40f8-a490-5cc4ba36e37b\" (UID: \"7b06680b-d594-40f8-a490-5cc4ba36e37b\") "
Jul 09 09:42:27.374363 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.374263    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b06680b-d594-40f8-a490-5cc4ba36e37b-trusted-ca-bundle\") pod \"7b06680b-d594-40f8-a490-5cc4ba36e37b\" (UID: \"7b06680b-d594-40f8-a490-5cc4ba36e37b\") "
Jul 09 09:42:27.374363 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.374305    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7b06680b-d594-40f8-a490-5cc4ba36e37b-console-config\") pod \"7b06680b-d594-40f8-a490-5cc4ba36e37b\" (UID: \"7b06680b-d594-40f8-a490-5cc4ba36e37b\") "
Jul 09 09:42:27.374363 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.374344    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7b06680b-d594-40f8-a490-5cc4ba36e37b-service-ca\") pod \"7b06680b-d594-40f8-a490-5cc4ba36e37b\" (UID: \"7b06680b-d594-40f8-a490-5cc4ba36e37b\") "
Jul 09 09:42:27.374662 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.374572    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b06680b-d594-40f8-a490-5cc4ba36e37b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7b06680b-d594-40f8-a490-5cc4ba36e37b" (UID: "7b06680b-d594-40f8-a490-5cc4ba36e37b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:42:27.374662 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.374655    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b06680b-d594-40f8-a490-5cc4ba36e37b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7b06680b-d594-40f8-a490-5cc4ba36e37b" (UID: "7b06680b-d594-40f8-a490-5cc4ba36e37b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:42:27.374786 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.374752    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b06680b-d594-40f8-a490-5cc4ba36e37b-console-config" (OuterVolumeSpecName: "console-config") pod "7b06680b-d594-40f8-a490-5cc4ba36e37b" (UID: "7b06680b-d594-40f8-a490-5cc4ba36e37b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:42:27.374860 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.374833    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b06680b-d594-40f8-a490-5cc4ba36e37b-service-ca" (OuterVolumeSpecName: "service-ca") pod "7b06680b-d594-40f8-a490-5cc4ba36e37b" (UID: "7b06680b-d594-40f8-a490-5cc4ba36e37b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:42:27.376429 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.376411    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b06680b-d594-40f8-a490-5cc4ba36e37b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7b06680b-d594-40f8-a490-5cc4ba36e37b" (UID: "7b06680b-d594-40f8-a490-5cc4ba36e37b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:42:27.376681 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.376661    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b06680b-d594-40f8-a490-5cc4ba36e37b-kube-api-access-jt6n2" (OuterVolumeSpecName: "kube-api-access-jt6n2") pod "7b06680b-d594-40f8-a490-5cc4ba36e37b" (UID: "7b06680b-d594-40f8-a490-5cc4ba36e37b"). InnerVolumeSpecName "kube-api-access-jt6n2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:42:27.376740 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.376672    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b06680b-d594-40f8-a490-5cc4ba36e37b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7b06680b-d594-40f8-a490-5cc4ba36e37b" (UID: "7b06680b-d594-40f8-a490-5cc4ba36e37b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:42:27.475028 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.474969    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jt6n2\" (UniqueName: \"kubernetes.io/projected/7b06680b-d594-40f8-a490-5cc4ba36e37b-kube-api-access-jt6n2\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:42:27.475028 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.474997    2549 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7b06680b-d594-40f8-a490-5cc4ba36e37b-oauth-serving-cert\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:42:27.475028 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.475008    2549 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7b06680b-d594-40f8-a490-5cc4ba36e37b-console-oauth-config\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:42:27.475028 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.475017    2549 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b06680b-d594-40f8-a490-5cc4ba36e37b-console-serving-cert\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:42:27.475028 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.475026    2549 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b06680b-d594-40f8-a490-5cc4ba36e37b-trusted-ca-bundle\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:42:27.475028 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.475034    2549 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7b06680b-d594-40f8-a490-5cc4ba36e37b-console-config\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:42:27.475267 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.475043    2549 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7b06680b-d594-40f8-a490-5cc4ba36e37b-service-ca\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:42:27.591414 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.591387    2549 generic.go:356] "Generic (PLEG): container finished" podID="7b06680b-d594-40f8-a490-5cc4ba36e37b" containerID="06974d8547018eeb9e60009a3e75b01b25cc3ed3e53dbfadfdf30890eded4516" exitCode=0
Jul 09 09:42:27.591699 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.591432    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7968f45-rtl4p"
Jul 09 09:42:27.591699 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.591444    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7968f45-rtl4p" event={"ID":"7b06680b-d594-40f8-a490-5cc4ba36e37b","Type":"ContainerDied","Data":"06974d8547018eeb9e60009a3e75b01b25cc3ed3e53dbfadfdf30890eded4516"}
Jul 09 09:42:27.591699 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.591492    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7968f45-rtl4p" event={"ID":"7b06680b-d594-40f8-a490-5cc4ba36e37b","Type":"ContainerDied","Data":"08229479a40f31a32f06498faaba650fb4fca217ab7e38fa135312192a633319"}
Jul 09 09:42:27.591699 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.591517    2549 scope.go:117] "RemoveContainer" containerID="06974d8547018eeb9e60009a3e75b01b25cc3ed3e53dbfadfdf30890eded4516"
Jul 09 09:42:27.599556 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.599540    2549 scope.go:117] "RemoveContainer" containerID="06974d8547018eeb9e60009a3e75b01b25cc3ed3e53dbfadfdf30890eded4516"
Jul 09 09:42:27.599787 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:42:27.599770    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06974d8547018eeb9e60009a3e75b01b25cc3ed3e53dbfadfdf30890eded4516\": container with ID starting with 06974d8547018eeb9e60009a3e75b01b25cc3ed3e53dbfadfdf30890eded4516 not found: ID does not exist" containerID="06974d8547018eeb9e60009a3e75b01b25cc3ed3e53dbfadfdf30890eded4516"
Jul 09 09:42:27.599844 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.599799    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06974d8547018eeb9e60009a3e75b01b25cc3ed3e53dbfadfdf30890eded4516"} err="failed to get container status \"06974d8547018eeb9e60009a3e75b01b25cc3ed3e53dbfadfdf30890eded4516\": rpc error: code = NotFound desc = could not find container \"06974d8547018eeb9e60009a3e75b01b25cc3ed3e53dbfadfdf30890eded4516\": container with ID starting with 06974d8547018eeb9e60009a3e75b01b25cc3ed3e53dbfadfdf30890eded4516 not found: ID does not exist"
Jul 09 09:42:27.612939 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.612917    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b7968f45-rtl4p"]
Jul 09 09:42:27.617277 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:27.617254    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b7968f45-rtl4p"]
Jul 09 09:42:28.384736 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:42:28.384706    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b06680b-d594-40f8-a490-5cc4ba36e37b" path="/var/lib/kubelet/pods/7b06680b-d594-40f8-a490-5cc4ba36e37b/volumes"
Jul 09 09:43:18.802803 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:18.802772    2549 generic.go:356] "Generic (PLEG): container finished" podID="2e836b72-ca73-496d-a127-2631e76ed06e" containerID="d2170337bf0a014389621f2e3c7543ba5fff954747037e27eda60729f9d7d8b6" exitCode=0
Jul 09 09:43:18.803224 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:18.802854    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-dmpbt" event={"ID":"2e836b72-ca73-496d-a127-2631e76ed06e","Type":"ContainerDied","Data":"d2170337bf0a014389621f2e3c7543ba5fff954747037e27eda60729f9d7d8b6"}
Jul 09 09:43:19.933649 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:19.933628    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-dmpbt"
Jul 09 09:43:20.102408 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:20.102323    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e836b72-ca73-496d-a127-2631e76ed06e-model-cache\") pod \"2e836b72-ca73-496d-a127-2631e76ed06e\" (UID: \"2e836b72-ca73-496d-a127-2631e76ed06e\") "
Jul 09 09:43:20.102408 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:20.102382    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dttpr\" (UniqueName: \"kubernetes.io/projected/2e836b72-ca73-496d-a127-2631e76ed06e-kube-api-access-dttpr\") pod \"2e836b72-ca73-496d-a127-2631e76ed06e\" (UID: \"2e836b72-ca73-496d-a127-2631e76ed06e\") "
Jul 09 09:43:20.104558 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:20.104525    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e836b72-ca73-496d-a127-2631e76ed06e-kube-api-access-dttpr" (OuterVolumeSpecName: "kube-api-access-dttpr") pod "2e836b72-ca73-496d-a127-2631e76ed06e" (UID: "2e836b72-ca73-496d-a127-2631e76ed06e"). InnerVolumeSpecName "kube-api-access-dttpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:43:20.120708 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:20.120677    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e836b72-ca73-496d-a127-2631e76ed06e-model-cache" (OuterVolumeSpecName: "model-cache") pod "2e836b72-ca73-496d-a127-2631e76ed06e" (UID: "2e836b72-ca73-496d-a127-2631e76ed06e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:43:20.202970 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:20.202922    2549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e836b72-ca73-496d-a127-2631e76ed06e-model-cache\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:43:20.202970 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:20.202966    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dttpr\" (UniqueName: \"kubernetes.io/projected/2e836b72-ca73-496d-a127-2631e76ed06e-kube-api-access-dttpr\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:43:20.811690 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:20.811657    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-dmpbt" event={"ID":"2e836b72-ca73-496d-a127-2631e76ed06e","Type":"ContainerDied","Data":"671927c81a200bf115e3cf6bc606fab20550ea7c02a9cf974871fcd86a85846c"}
Jul 09 09:43:20.811690 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:20.811676    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-dmpbt"
Jul 09 09:43:20.811690 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:20.811697    2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="671927c81a200bf115e3cf6bc606fab20550ea7c02a9cf974871fcd86a85846c"
Jul 09 09:43:21.607992 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:21.607959    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-tcp4w"]
Jul 09 09:43:21.608355 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:21.608274    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e836b72-ca73-496d-a127-2631e76ed06e" containerName="download-hf-model"
Jul 09 09:43:21.608355 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:21.608285    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e836b72-ca73-496d-a127-2631e76ed06e" containerName="download-hf-model"
Jul 09 09:43:21.608355 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:21.608302    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e836b72-ca73-496d-a127-2631e76ed06e" containerName="s3-init"
Jul 09 09:43:21.608355 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:21.608307    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e836b72-ca73-496d-a127-2631e76ed06e" containerName="s3-init"
Jul 09 09:43:21.608355 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:21.608316    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b06680b-d594-40f8-a490-5cc4ba36e37b" containerName="console"
Jul 09 09:43:21.608355 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:21.608321    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b06680b-d594-40f8-a490-5cc4ba36e37b" containerName="console"
Jul 09 09:43:21.608537 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:21.608378    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e836b72-ca73-496d-a127-2631e76ed06e" containerName="s3-init"
Jul 09 09:43:21.608537 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:21.608387    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b06680b-d594-40f8-a490-5cc4ba36e37b" containerName="console"
Jul 09 09:43:21.611398 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:21.611382    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-tcp4w"
Jul 09 09:43:21.614346 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:21.614307    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom-artifact\""
Jul 09 09:43:21.614470 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:21.614309    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-w9nzs\""
Jul 09 09:43:21.614470 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:21.614430    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 09 09:43:21.615318 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:21.615300    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 09 09:43:21.620157 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:21.620137    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-tcp4w"]
Jul 09 09:43:21.715745 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:21.715716    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5a1b7c1b-cd17-48f7-8a80-981c4fc1286a-data\") pod \"seaweedfs-tls-custom-6644bf5d6c-tcp4w\" (UID: \"5a1b7c1b-cd17-48f7-8a80-981c4fc1286a\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-tcp4w"
Jul 09 09:43:21.715911 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:21.715759    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxj8f\" (UniqueName: \"kubernetes.io/projected/5a1b7c1b-cd17-48f7-8a80-981c4fc1286a-kube-api-access-wxj8f\") pod \"seaweedfs-tls-custom-6644bf5d6c-tcp4w\" (UID: \"5a1b7c1b-cd17-48f7-8a80-981c4fc1286a\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-tcp4w"
Jul 09 09:43:21.817024 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:21.816991    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5a1b7c1b-cd17-48f7-8a80-981c4fc1286a-data\") pod \"seaweedfs-tls-custom-6644bf5d6c-tcp4w\" (UID: \"5a1b7c1b-cd17-48f7-8a80-981c4fc1286a\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-tcp4w"
Jul 09 09:43:21.817181 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:21.817034    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxj8f\" (UniqueName: \"kubernetes.io/projected/5a1b7c1b-cd17-48f7-8a80-981c4fc1286a-kube-api-access-wxj8f\") pod \"seaweedfs-tls-custom-6644bf5d6c-tcp4w\" (UID: \"5a1b7c1b-cd17-48f7-8a80-981c4fc1286a\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-tcp4w"
Jul 09 09:43:21.817437 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:21.817421    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5a1b7c1b-cd17-48f7-8a80-981c4fc1286a-data\") pod \"seaweedfs-tls-custom-6644bf5d6c-tcp4w\" (UID: \"5a1b7c1b-cd17-48f7-8a80-981c4fc1286a\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-tcp4w"
Jul 09 09:43:21.826043 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:21.826016    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxj8f\" (UniqueName: \"kubernetes.io/projected/5a1b7c1b-cd17-48f7-8a80-981c4fc1286a-kube-api-access-wxj8f\") pod \"seaweedfs-tls-custom-6644bf5d6c-tcp4w\" (UID: \"5a1b7c1b-cd17-48f7-8a80-981c4fc1286a\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-tcp4w"
Jul 09 09:43:21.921136 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:21.921061    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-tcp4w"
Jul 09 09:43:22.044854 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:22.044826    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-tcp4w"]
Jul 09 09:43:22.047270 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:43:22.047247    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a1b7c1b_cd17_48f7_8a80_981c4fc1286a.slice/crio-1cc55f438f293f297068fed1cdb3456b277a6a23fa6758be7c19b7916fe6628e WatchSource:0}: Error finding container 1cc55f438f293f297068fed1cdb3456b277a6a23fa6758be7c19b7916fe6628e: Status 404 returned error can't find the container with id 1cc55f438f293f297068fed1cdb3456b277a6a23fa6758be7c19b7916fe6628e
Jul 09 09:43:22.821662 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:22.821622    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-tcp4w" event={"ID":"5a1b7c1b-cd17-48f7-8a80-981c4fc1286a","Type":"ContainerStarted","Data":"1cc55f438f293f297068fed1cdb3456b277a6a23fa6758be7c19b7916fe6628e"}
Jul 09 09:43:25.835203 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:25.835173    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-tcp4w" event={"ID":"5a1b7c1b-cd17-48f7-8a80-981c4fc1286a","Type":"ContainerStarted","Data":"0e9b1308fa46c4ddf05661b32b9d2c18419299486385487c07e46149a7df1809"}
Jul 09 09:43:25.852641 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:25.852585    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-tcp4w" podStartSLOduration=2.013131479 podStartE2EDuration="4.852568421s" totalImagesPullingTime="2.839436942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:43:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:43:25.852007685 +0000 UTC m=+564.023166252" watchObservedRunningTime="2026-07-09 09:43:25.852568421 +0000 UTC m=+564.023726977"
Jul 09 09:43:26.580668 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:26.580638    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-tcp4w"]
Jul 09 09:43:27.845703 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:27.845656    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-tcp4w" podUID="5a1b7c1b-cd17-48f7-8a80-981c4fc1286a" containerName="seaweedfs-tls-custom" containerID="cri-o://0e9b1308fa46c4ddf05661b32b9d2c18419299486385487c07e46149a7df1809" gracePeriod=30
Jul 09 09:43:29.079126 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.079098    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-tcp4w"
Jul 09 09:43:29.178574 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.178503    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5a1b7c1b-cd17-48f7-8a80-981c4fc1286a-data\") pod \"5a1b7c1b-cd17-48f7-8a80-981c4fc1286a\" (UID: \"5a1b7c1b-cd17-48f7-8a80-981c4fc1286a\") "
Jul 09 09:43:29.178574 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.178567    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxj8f\" (UniqueName: \"kubernetes.io/projected/5a1b7c1b-cd17-48f7-8a80-981c4fc1286a-kube-api-access-wxj8f\") pod \"5a1b7c1b-cd17-48f7-8a80-981c4fc1286a\" (UID: \"5a1b7c1b-cd17-48f7-8a80-981c4fc1286a\") "
Jul 09 09:43:29.179769 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.179742    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a1b7c1b-cd17-48f7-8a80-981c4fc1286a-data" (OuterVolumeSpecName: "data") pod "5a1b7c1b-cd17-48f7-8a80-981c4fc1286a" (UID: "5a1b7c1b-cd17-48f7-8a80-981c4fc1286a"). InnerVolumeSpecName "data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:43:29.180438 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.180417    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a1b7c1b-cd17-48f7-8a80-981c4fc1286a-kube-api-access-wxj8f" (OuterVolumeSpecName: "kube-api-access-wxj8f") pod "5a1b7c1b-cd17-48f7-8a80-981c4fc1286a" (UID: "5a1b7c1b-cd17-48f7-8a80-981c4fc1286a"). InnerVolumeSpecName "kube-api-access-wxj8f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:43:29.279120 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.279098    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxj8f\" (UniqueName: \"kubernetes.io/projected/5a1b7c1b-cd17-48f7-8a80-981c4fc1286a-kube-api-access-wxj8f\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:43:29.279120 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.279118    2549 reconciler_common.go:299] "Volume detached for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5a1b7c1b-cd17-48f7-8a80-981c4fc1286a-data\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:43:29.854809 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.854771    2549 generic.go:356] "Generic (PLEG): container finished" podID="5a1b7c1b-cd17-48f7-8a80-981c4fc1286a" containerID="0e9b1308fa46c4ddf05661b32b9d2c18419299486385487c07e46149a7df1809" exitCode=0
Jul 09 09:43:29.855026 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.854831    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-tcp4w"
Jul 09 09:43:29.855026 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.854861    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-tcp4w" event={"ID":"5a1b7c1b-cd17-48f7-8a80-981c4fc1286a","Type":"ContainerDied","Data":"0e9b1308fa46c4ddf05661b32b9d2c18419299486385487c07e46149a7df1809"}
Jul 09 09:43:29.855026 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.854960    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-tcp4w" event={"ID":"5a1b7c1b-cd17-48f7-8a80-981c4fc1286a","Type":"ContainerDied","Data":"1cc55f438f293f297068fed1cdb3456b277a6a23fa6758be7c19b7916fe6628e"}
Jul 09 09:43:29.855026 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.854981    2549 scope.go:117] "RemoveContainer" containerID="0e9b1308fa46c4ddf05661b32b9d2c18419299486385487c07e46149a7df1809"
Jul 09 09:43:29.864311 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.864295    2549 scope.go:117] "RemoveContainer" containerID="0e9b1308fa46c4ddf05661b32b9d2c18419299486385487c07e46149a7df1809"
Jul 09 09:43:29.864559 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:43:29.864541    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e9b1308fa46c4ddf05661b32b9d2c18419299486385487c07e46149a7df1809\": container with ID starting with 0e9b1308fa46c4ddf05661b32b9d2c18419299486385487c07e46149a7df1809 not found: ID does not exist" containerID="0e9b1308fa46c4ddf05661b32b9d2c18419299486385487c07e46149a7df1809"
Jul 09 09:43:29.864595 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.864567    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e9b1308fa46c4ddf05661b32b9d2c18419299486385487c07e46149a7df1809"} err="failed to get container status \"0e9b1308fa46c4ddf05661b32b9d2c18419299486385487c07e46149a7df1809\": rpc error: code = NotFound desc = could not find container \"0e9b1308fa46c4ddf05661b32b9d2c18419299486385487c07e46149a7df1809\": container with ID starting with 0e9b1308fa46c4ddf05661b32b9d2c18419299486385487c07e46149a7df1809 not found: ID does not exist"
Jul 09 09:43:29.875101 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.875080    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-tcp4w"]
Jul 09 09:43:29.877160 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.877141    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-tcp4w"]
Jul 09 09:43:29.905541 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.905523    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-tls-custom-8b44d767d-79m62"]
Jul 09 09:43:29.905890 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.905859    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a1b7c1b-cd17-48f7-8a80-981c4fc1286a" containerName="seaweedfs-tls-custom"
Jul 09 09:43:29.905961 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.905892    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a1b7c1b-cd17-48f7-8a80-981c4fc1286a" containerName="seaweedfs-tls-custom"
Jul 09 09:43:29.906020 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.906001    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a1b7c1b-cd17-48f7-8a80-981c4fc1286a" containerName="seaweedfs-tls-custom"
Jul 09 09:43:29.910364 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.910348    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-8b44d767d-79m62"
Jul 09 09:43:29.913135 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.913117    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 09 09:43:29.913242 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.913195    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 09 09:43:29.913305 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.913251    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom-artifact\""
Jul 09 09:43:29.913305 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.913270    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-w9nzs\""
Jul 09 09:43:29.913408 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.913304    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom\""
Jul 09 09:43:29.917875 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.917841    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-8b44d767d-79m62"]
Jul 09 09:43:29.985837 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.985814    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"seaweedfs-tls-custom\" (UniqueName: \"kubernetes.io/projected/f5106f13-399a-4cc9-aaa3-dda3c3a2dda2-seaweedfs-tls-custom\") pod \"seaweedfs-tls-custom-8b44d767d-79m62\" (UID: \"f5106f13-399a-4cc9-aaa3-dda3c3a2dda2\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-79m62"
Jul 09 09:43:29.985951 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.985842    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f5106f13-399a-4cc9-aaa3-dda3c3a2dda2-data\") pod \"seaweedfs-tls-custom-8b44d767d-79m62\" (UID: \"f5106f13-399a-4cc9-aaa3-dda3c3a2dda2\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-79m62"
Jul 09 09:43:29.985951 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:29.985941    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6mdf\" (UniqueName: \"kubernetes.io/projected/f5106f13-399a-4cc9-aaa3-dda3c3a2dda2-kube-api-access-s6mdf\") pod \"seaweedfs-tls-custom-8b44d767d-79m62\" (UID: \"f5106f13-399a-4cc9-aaa3-dda3c3a2dda2\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-79m62"
Jul 09 09:43:30.087044 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:30.087019    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"seaweedfs-tls-custom\" (UniqueName: \"kubernetes.io/projected/f5106f13-399a-4cc9-aaa3-dda3c3a2dda2-seaweedfs-tls-custom\") pod \"seaweedfs-tls-custom-8b44d767d-79m62\" (UID: \"f5106f13-399a-4cc9-aaa3-dda3c3a2dda2\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-79m62"
Jul 09 09:43:30.087342 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:30.087050    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f5106f13-399a-4cc9-aaa3-dda3c3a2dda2-data\") pod \"seaweedfs-tls-custom-8b44d767d-79m62\" (UID: \"f5106f13-399a-4cc9-aaa3-dda3c3a2dda2\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-79m62"
Jul 09 09:43:30.087342 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:30.087087    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6mdf\" (UniqueName: \"kubernetes.io/projected/f5106f13-399a-4cc9-aaa3-dda3c3a2dda2-kube-api-access-s6mdf\") pod \"seaweedfs-tls-custom-8b44d767d-79m62\" (UID: \"f5106f13-399a-4cc9-aaa3-dda3c3a2dda2\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-79m62"
Jul 09 09:43:30.087457 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:30.087440    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f5106f13-399a-4cc9-aaa3-dda3c3a2dda2-data\") pod \"seaweedfs-tls-custom-8b44d767d-79m62\" (UID: \"f5106f13-399a-4cc9-aaa3-dda3c3a2dda2\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-79m62"
Jul 09 09:43:30.089354 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:30.089337    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"seaweedfs-tls-custom\" (UniqueName: \"kubernetes.io/projected/f5106f13-399a-4cc9-aaa3-dda3c3a2dda2-seaweedfs-tls-custom\") pod \"seaweedfs-tls-custom-8b44d767d-79m62\" (UID: \"f5106f13-399a-4cc9-aaa3-dda3c3a2dda2\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-79m62"
Jul 09 09:43:30.096686 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:30.096663    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6mdf\" (UniqueName: \"kubernetes.io/projected/f5106f13-399a-4cc9-aaa3-dda3c3a2dda2-kube-api-access-s6mdf\") pod \"seaweedfs-tls-custom-8b44d767d-79m62\" (UID: \"f5106f13-399a-4cc9-aaa3-dda3c3a2dda2\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-79m62"
Jul 09 09:43:30.220109 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:30.220053    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-8b44d767d-79m62"
Jul 09 09:43:30.343142 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:30.343096    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-8b44d767d-79m62"]
Jul 09 09:43:30.345709 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:43:30.345682    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5106f13_399a_4cc9_aaa3_dda3c3a2dda2.slice/crio-3ef4be4c18d4e9c5dd0fc3a3676b6a72625c75e3fd8b98a02a459506a583b7d9 WatchSource:0}: Error finding container 3ef4be4c18d4e9c5dd0fc3a3676b6a72625c75e3fd8b98a02a459506a583b7d9: Status 404 returned error can't find the container with id 3ef4be4c18d4e9c5dd0fc3a3676b6a72625c75e3fd8b98a02a459506a583b7d9
Jul 09 09:43:30.384887 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:30.384841    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a1b7c1b-cd17-48f7-8a80-981c4fc1286a" path="/var/lib/kubelet/pods/5a1b7c1b-cd17-48f7-8a80-981c4fc1286a/volumes"
Jul 09 09:43:30.860553 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:30.860522    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-8b44d767d-79m62" event={"ID":"f5106f13-399a-4cc9-aaa3-dda3c3a2dda2","Type":"ContainerStarted","Data":"3964cbd78aad814d3675db10c722d03a6bd3052aea53f7c87a466e2386fd7d12"}
Jul 09 09:43:30.860685 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:30.860566    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-8b44d767d-79m62" event={"ID":"f5106f13-399a-4cc9-aaa3-dda3c3a2dda2","Type":"ContainerStarted","Data":"3ef4be4c18d4e9c5dd0fc3a3676b6a72625c75e3fd8b98a02a459506a583b7d9"}
Jul 09 09:43:30.879375 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:30.879330    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-tls-custom-8b44d767d-79m62" podStartSLOduration=1.5829054359999999 podStartE2EDuration="1.879315677s" totalImagesPullingTime="296.410241ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:43:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:43:30.878048216 +0000 UTC m=+569.049206773" watchObservedRunningTime="2026-07-09 09:43:30.879315677 +0000 UTC m=+569.050474232"
Jul 09 09:43:31.196212 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:31.196181    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-tls-init-custom-fqtpr"]
Jul 09 09:43:31.199669 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:31.199647    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-custom-fqtpr"
Jul 09 09:43:31.207951 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:31.207928    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-tls-init-custom-fqtpr"]
Jul 09 09:43:31.299586 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:31.299544    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3defb281-5ccb-49ea-9f28-9a16562662cc-model-cache\") pod \"s3-tls-init-custom-fqtpr\" (UID: \"3defb281-5ccb-49ea-9f28-9a16562662cc\") " pod="kserve/s3-tls-init-custom-fqtpr"
Jul 09 09:43:31.299729 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:31.299666    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nnd4\" (UniqueName: \"kubernetes.io/projected/3defb281-5ccb-49ea-9f28-9a16562662cc-kube-api-access-9nnd4\") pod \"s3-tls-init-custom-fqtpr\" (UID: \"3defb281-5ccb-49ea-9f28-9a16562662cc\") " pod="kserve/s3-tls-init-custom-fqtpr"
Jul 09 09:43:31.401099 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:31.401056    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nnd4\" (UniqueName: \"kubernetes.io/projected/3defb281-5ccb-49ea-9f28-9a16562662cc-kube-api-access-9nnd4\") pod \"s3-tls-init-custom-fqtpr\" (UID: \"3defb281-5ccb-49ea-9f28-9a16562662cc\") " pod="kserve/s3-tls-init-custom-fqtpr"
Jul 09 09:43:31.401259 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:31.401123    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3defb281-5ccb-49ea-9f28-9a16562662cc-model-cache\") pod \"s3-tls-init-custom-fqtpr\" (UID: \"3defb281-5ccb-49ea-9f28-9a16562662cc\") " pod="kserve/s3-tls-init-custom-fqtpr"
Jul 09 09:43:31.401430 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:31.401412    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3defb281-5ccb-49ea-9f28-9a16562662cc-model-cache\") pod \"s3-tls-init-custom-fqtpr\" (UID: \"3defb281-5ccb-49ea-9f28-9a16562662cc\") " pod="kserve/s3-tls-init-custom-fqtpr"
Jul 09 09:43:31.414006 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:31.413986    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nnd4\" (UniqueName: \"kubernetes.io/projected/3defb281-5ccb-49ea-9f28-9a16562662cc-kube-api-access-9nnd4\") pod \"s3-tls-init-custom-fqtpr\" (UID: \"3defb281-5ccb-49ea-9f28-9a16562662cc\") " pod="kserve/s3-tls-init-custom-fqtpr"
Jul 09 09:43:31.521776 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:31.521754    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-custom-fqtpr"
Jul 09 09:43:31.652818 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:31.652780    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-tls-init-custom-fqtpr"]
Jul 09 09:43:31.656774 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:43:31.656741    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3defb281_5ccb_49ea_9f28_9a16562662cc.slice/crio-a9d0aac4e2607c682e748f7a28f54f31fd21ebb9f72cc229f9129d0e63e29a57 WatchSource:0}: Error finding container a9d0aac4e2607c682e748f7a28f54f31fd21ebb9f72cc229f9129d0e63e29a57: Status 404 returned error can't find the container with id a9d0aac4e2607c682e748f7a28f54f31fd21ebb9f72cc229f9129d0e63e29a57
Jul 09 09:43:31.866056 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:31.865973    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-fqtpr" event={"ID":"3defb281-5ccb-49ea-9f28-9a16562662cc","Type":"ContainerStarted","Data":"f96248a409a4ec7eab686f2804f6f3529f8fb6a7aa95ae899efb312b0886424f"}
Jul 09 09:43:31.866056 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:31.866010    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-fqtpr" event={"ID":"3defb281-5ccb-49ea-9f28-9a16562662cc","Type":"ContainerStarted","Data":"a9d0aac4e2607c682e748f7a28f54f31fd21ebb9f72cc229f9129d0e63e29a57"}
Jul 09 09:43:33.877369 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:33.877333    2549 generic.go:356] "Generic (PLEG): container finished" podID="3defb281-5ccb-49ea-9f28-9a16562662cc" containerID="f96248a409a4ec7eab686f2804f6f3529f8fb6a7aa95ae899efb312b0886424f" exitCode=0
Jul 09 09:43:33.877765 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:33.877405    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-fqtpr" event={"ID":"3defb281-5ccb-49ea-9f28-9a16562662cc","Type":"ContainerDied","Data":"f96248a409a4ec7eab686f2804f6f3529f8fb6a7aa95ae899efb312b0886424f"}
Jul 09 09:43:34.883185 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:34.883157    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-fqtpr" event={"ID":"3defb281-5ccb-49ea-9f28-9a16562662cc","Type":"ContainerStarted","Data":"11551541a96b94488fecd83d9b6dcef21a1dad8c33a6d43677856d07ac184f88"}
Jul 09 09:43:34.900682 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:34.900639    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-tls-init-custom-fqtpr" podStartSLOduration=2.592039496 podStartE2EDuration="3.900625073s" totalImagesPullingTime="0s" totalInitContainerRuntime="1.308585577s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:43:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:43:34.899381024 +0000 UTC m=+573.070539580" watchObservedRunningTime="2026-07-09 09:43:34.900625073 +0000 UTC m=+573.071783662"
Jul 09 09:43:41.914623 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:41.914592    2549 generic.go:356] "Generic (PLEG): container finished" podID="3defb281-5ccb-49ea-9f28-9a16562662cc" containerID="11551541a96b94488fecd83d9b6dcef21a1dad8c33a6d43677856d07ac184f88" exitCode=0
Jul 09 09:43:41.914984 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:41.914668    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-fqtpr" event={"ID":"3defb281-5ccb-49ea-9f28-9a16562662cc","Type":"ContainerDied","Data":"11551541a96b94488fecd83d9b6dcef21a1dad8c33a6d43677856d07ac184f88"}
Jul 09 09:43:43.055834 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:43.055808    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-custom-fqtpr"
Jul 09 09:43:43.100147 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:43.100123    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3defb281-5ccb-49ea-9f28-9a16562662cc-model-cache\") pod \"3defb281-5ccb-49ea-9f28-9a16562662cc\" (UID: \"3defb281-5ccb-49ea-9f28-9a16562662cc\") "
Jul 09 09:43:43.100277 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:43.100171    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9nnd4\" (UniqueName: \"kubernetes.io/projected/3defb281-5ccb-49ea-9f28-9a16562662cc-kube-api-access-9nnd4\") pod \"3defb281-5ccb-49ea-9f28-9a16562662cc\" (UID: \"3defb281-5ccb-49ea-9f28-9a16562662cc\") "
Jul 09 09:43:43.102173 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:43.102148    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3defb281-5ccb-49ea-9f28-9a16562662cc-kube-api-access-9nnd4" (OuterVolumeSpecName: "kube-api-access-9nnd4") pod "3defb281-5ccb-49ea-9f28-9a16562662cc" (UID: "3defb281-5ccb-49ea-9f28-9a16562662cc"). InnerVolumeSpecName "kube-api-access-9nnd4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:43:43.183089 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:43.183033    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3defb281-5ccb-49ea-9f28-9a16562662cc-model-cache" (OuterVolumeSpecName: "model-cache") pod "3defb281-5ccb-49ea-9f28-9a16562662cc" (UID: "3defb281-5ccb-49ea-9f28-9a16562662cc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:43:43.201217 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:43.201199    2549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3defb281-5ccb-49ea-9f28-9a16562662cc-model-cache\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:43:43.201289 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:43.201219    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9nnd4\" (UniqueName: \"kubernetes.io/projected/3defb281-5ccb-49ea-9f28-9a16562662cc-kube-api-access-9nnd4\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:43:43.925424 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:43.925394    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-custom-fqtpr"
Jul 09 09:43:43.925424 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:43.925415    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-fqtpr" event={"ID":"3defb281-5ccb-49ea-9f28-9a16562662cc","Type":"ContainerDied","Data":"a9d0aac4e2607c682e748f7a28f54f31fd21ebb9f72cc229f9129d0e63e29a57"}
Jul 09 09:43:43.925619 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:43.925453    2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a9d0aac4e2607c682e748f7a28f54f31fd21ebb9f72cc229f9129d0e63e29a57"
Jul 09 09:43:44.558354 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:44.558320    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-tls-serving-67b7477c6d-vfc9f"]
Jul 09 09:43:44.558961 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:44.558636    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3defb281-5ccb-49ea-9f28-9a16562662cc" containerName="download-hf-model"
Jul 09 09:43:44.558961 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:44.558647    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="3defb281-5ccb-49ea-9f28-9a16562662cc" containerName="download-hf-model"
Jul 09 09:43:44.558961 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:44.558671    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3defb281-5ccb-49ea-9f28-9a16562662cc" containerName="s3-tls-init-custom"
Jul 09 09:43:44.558961 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:44.558676    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="3defb281-5ccb-49ea-9f28-9a16562662cc" containerName="s3-tls-init-custom"
Jul 09 09:43:44.558961 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:44.558727    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="3defb281-5ccb-49ea-9f28-9a16562662cc" containerName="s3-tls-init-custom"
Jul 09 09:43:44.564041 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:44.564025    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-serving-67b7477c6d-vfc9f"
Jul 09 09:43:44.566166 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:44.566141    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-serving\""
Jul 09 09:43:44.566288 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:44.566191    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-serving-artifact\""
Jul 09 09:43:44.569238 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:44.569216    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-serving-67b7477c6d-vfc9f"]
Jul 09 09:43:44.610756 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:44.610721    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7d6bd83e-4062-43b0-a9ef-94c2d72d31dc-data\") pod \"seaweedfs-tls-serving-67b7477c6d-vfc9f\" (UID: \"7d6bd83e-4062-43b0-a9ef-94c2d72d31dc\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-vfc9f"
Jul 09 09:43:44.610888 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:44.610773    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frcm6\" (UniqueName: \"kubernetes.io/projected/7d6bd83e-4062-43b0-a9ef-94c2d72d31dc-kube-api-access-frcm6\") pod \"seaweedfs-tls-serving-67b7477c6d-vfc9f\" (UID: \"7d6bd83e-4062-43b0-a9ef-94c2d72d31dc\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-vfc9f"
Jul 09 09:43:44.610888 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:44.610810    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"seaweedfs-tls-serving\" (UniqueName: \"kubernetes.io/projected/7d6bd83e-4062-43b0-a9ef-94c2d72d31dc-seaweedfs-tls-serving\") pod \"seaweedfs-tls-serving-67b7477c6d-vfc9f\" (UID: \"7d6bd83e-4062-43b0-a9ef-94c2d72d31dc\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-vfc9f"
Jul 09 09:43:44.712130 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:44.712107    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7d6bd83e-4062-43b0-a9ef-94c2d72d31dc-data\") pod \"seaweedfs-tls-serving-67b7477c6d-vfc9f\" (UID: \"7d6bd83e-4062-43b0-a9ef-94c2d72d31dc\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-vfc9f"
Jul 09 09:43:44.712234 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:44.712147    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frcm6\" (UniqueName: \"kubernetes.io/projected/7d6bd83e-4062-43b0-a9ef-94c2d72d31dc-kube-api-access-frcm6\") pod \"seaweedfs-tls-serving-67b7477c6d-vfc9f\" (UID: \"7d6bd83e-4062-43b0-a9ef-94c2d72d31dc\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-vfc9f"
Jul 09 09:43:44.712234 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:44.712172    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"seaweedfs-tls-serving\" (UniqueName: \"kubernetes.io/projected/7d6bd83e-4062-43b0-a9ef-94c2d72d31dc-seaweedfs-tls-serving\") pod \"seaweedfs-tls-serving-67b7477c6d-vfc9f\" (UID: \"7d6bd83e-4062-43b0-a9ef-94c2d72d31dc\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-vfc9f"
Jul 09 09:43:44.712324 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:43:44.712287    2549 projected.go:266] Couldn't get secret kserve/seaweedfs-tls-serving: secret "seaweedfs-tls-serving" not found
Jul 09 09:43:44.712324 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:43:44.712297    2549 projected.go:196] Error preparing data for projected volume seaweedfs-tls-serving for pod kserve/seaweedfs-tls-serving-67b7477c6d-vfc9f: secret "seaweedfs-tls-serving" not found
Jul 09 09:43:44.712408 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:43:44.712342    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d6bd83e-4062-43b0-a9ef-94c2d72d31dc-seaweedfs-tls-serving podName:7d6bd83e-4062-43b0-a9ef-94c2d72d31dc nodeName:}" failed. No retries permitted until 2026-07-09 09:43:45.212328094 +0000 UTC m=+583.383486628 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "seaweedfs-tls-serving" (UniqueName: "kubernetes.io/projected/7d6bd83e-4062-43b0-a9ef-94c2d72d31dc-seaweedfs-tls-serving") pod "seaweedfs-tls-serving-67b7477c6d-vfc9f" (UID: "7d6bd83e-4062-43b0-a9ef-94c2d72d31dc") : secret "seaweedfs-tls-serving" not found
Jul 09 09:43:44.712460 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:44.712427    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7d6bd83e-4062-43b0-a9ef-94c2d72d31dc-data\") pod \"seaweedfs-tls-serving-67b7477c6d-vfc9f\" (UID: \"7d6bd83e-4062-43b0-a9ef-94c2d72d31dc\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-vfc9f"
Jul 09 09:43:44.720942 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:44.720920    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frcm6\" (UniqueName: \"kubernetes.io/projected/7d6bd83e-4062-43b0-a9ef-94c2d72d31dc-kube-api-access-frcm6\") pod \"seaweedfs-tls-serving-67b7477c6d-vfc9f\" (UID: \"7d6bd83e-4062-43b0-a9ef-94c2d72d31dc\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-vfc9f"
Jul 09 09:43:45.216072 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:45.216043    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"seaweedfs-tls-serving\" (UniqueName: \"kubernetes.io/projected/7d6bd83e-4062-43b0-a9ef-94c2d72d31dc-seaweedfs-tls-serving\") pod \"seaweedfs-tls-serving-67b7477c6d-vfc9f\" (UID: \"7d6bd83e-4062-43b0-a9ef-94c2d72d31dc\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-vfc9f"
Jul 09 09:43:45.218450 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:45.218424    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"seaweedfs-tls-serving\" (UniqueName: \"kubernetes.io/projected/7d6bd83e-4062-43b0-a9ef-94c2d72d31dc-seaweedfs-tls-serving\") pod \"seaweedfs-tls-serving-67b7477c6d-vfc9f\" (UID: \"7d6bd83e-4062-43b0-a9ef-94c2d72d31dc\") " pod="kserve/seaweedfs-tls-serving-67b7477c6d-vfc9f"
Jul 09 09:43:45.474953 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:45.474881    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-serving-67b7477c6d-vfc9f"
Jul 09 09:43:45.612096 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:45.612064    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-serving-67b7477c6d-vfc9f"]
Jul 09 09:43:45.615821 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:43:45.615794    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d6bd83e_4062_43b0_a9ef_94c2d72d31dc.slice/crio-50711be45a2f53487f210c5b197d16871d37de4077e2431313150e93fcc5043a WatchSource:0}: Error finding container 50711be45a2f53487f210c5b197d16871d37de4077e2431313150e93fcc5043a: Status 404 returned error can't find the container with id 50711be45a2f53487f210c5b197d16871d37de4077e2431313150e93fcc5043a
Jul 09 09:43:45.935934 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:45.935907    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-serving-67b7477c6d-vfc9f" event={"ID":"7d6bd83e-4062-43b0-a9ef-94c2d72d31dc","Type":"ContainerStarted","Data":"0f06aca0c921752f7760227c7e60c81dfc895a2b3eebf6ce9c7dbca1488b3934"}
Jul 09 09:43:45.936037 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:45.935959    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-serving-67b7477c6d-vfc9f" event={"ID":"7d6bd83e-4062-43b0-a9ef-94c2d72d31dc","Type":"ContainerStarted","Data":"50711be45a2f53487f210c5b197d16871d37de4077e2431313150e93fcc5043a"}
Jul 09 09:43:45.971684 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:45.971632    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-tls-serving-67b7477c6d-vfc9f" podStartSLOduration=1.7081215730000001 podStartE2EDuration="1.971615689s" totalImagesPullingTime="263.494116ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:43:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:43:45.969985023 +0000 UTC m=+584.141143580" watchObservedRunningTime="2026-07-09 09:43:45.971615689 +0000 UTC m=+584.142774245"
Jul 09 09:43:46.539886 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:46.539835    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-tls-init-serving-wzg5j"]
Jul 09 09:43:46.543238 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:46.543223    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-serving-wzg5j"
Jul 09 09:43:46.551223 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:46.551191    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-tls-init-serving-wzg5j"]
Jul 09 09:43:46.627025 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:46.626996    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02e38491-5d3e-4457-9df1-eda978e82010-model-cache\") pod \"s3-tls-init-serving-wzg5j\" (UID: \"02e38491-5d3e-4457-9df1-eda978e82010\") " pod="kserve/s3-tls-init-serving-wzg5j"
Jul 09 09:43:46.627379 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:46.627050    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8h6n\" (UniqueName: \"kubernetes.io/projected/02e38491-5d3e-4457-9df1-eda978e82010-kube-api-access-v8h6n\") pod \"s3-tls-init-serving-wzg5j\" (UID: \"02e38491-5d3e-4457-9df1-eda978e82010\") " pod="kserve/s3-tls-init-serving-wzg5j"
Jul 09 09:43:46.728074 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:46.728037    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8h6n\" (UniqueName: \"kubernetes.io/projected/02e38491-5d3e-4457-9df1-eda978e82010-kube-api-access-v8h6n\") pod \"s3-tls-init-serving-wzg5j\" (UID: \"02e38491-5d3e-4457-9df1-eda978e82010\") " pod="kserve/s3-tls-init-serving-wzg5j"
Jul 09 09:43:46.728223 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:46.728120    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02e38491-5d3e-4457-9df1-eda978e82010-model-cache\") pod \"s3-tls-init-serving-wzg5j\" (UID: \"02e38491-5d3e-4457-9df1-eda978e82010\") " pod="kserve/s3-tls-init-serving-wzg5j"
Jul 09 09:43:46.728439 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:46.728418    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02e38491-5d3e-4457-9df1-eda978e82010-model-cache\") pod \"s3-tls-init-serving-wzg5j\" (UID: \"02e38491-5d3e-4457-9df1-eda978e82010\") " pod="kserve/s3-tls-init-serving-wzg5j"
Jul 09 09:43:46.737481 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:46.737463    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8h6n\" (UniqueName: \"kubernetes.io/projected/02e38491-5d3e-4457-9df1-eda978e82010-kube-api-access-v8h6n\") pod \"s3-tls-init-serving-wzg5j\" (UID: \"02e38491-5d3e-4457-9df1-eda978e82010\") " pod="kserve/s3-tls-init-serving-wzg5j"
Jul 09 09:43:46.870959 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:46.870895    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-serving-wzg5j"
Jul 09 09:43:46.998069 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:46.998044    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-tls-init-serving-wzg5j"]
Jul 09 09:43:47.000076 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:43:47.000051    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02e38491_5d3e_4457_9df1_eda978e82010.slice/crio-4a3a992fd333eb85ec777243717f07795a1ed34e97f07dc4d5fce4d731185869 WatchSource:0}: Error finding container 4a3a992fd333eb85ec777243717f07795a1ed34e97f07dc4d5fce4d731185869: Status 404 returned error can't find the container with id 4a3a992fd333eb85ec777243717f07795a1ed34e97f07dc4d5fce4d731185869
Jul 09 09:43:47.946913 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:47.946859    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-wzg5j" event={"ID":"02e38491-5d3e-4457-9df1-eda978e82010","Type":"ContainerStarted","Data":"538341fe6b5726c593cbf51d8e2ef5989fbceb67281798a59f1a53b613dcf994"}
Jul 09 09:43:47.947332 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:47.946925    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-wzg5j" event={"ID":"02e38491-5d3e-4457-9df1-eda978e82010","Type":"ContainerStarted","Data":"4a3a992fd333eb85ec777243717f07795a1ed34e97f07dc4d5fce4d731185869"}
Jul 09 09:43:48.951770 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:48.951746    2549 generic.go:356] "Generic (PLEG): container finished" podID="02e38491-5d3e-4457-9df1-eda978e82010" containerID="538341fe6b5726c593cbf51d8e2ef5989fbceb67281798a59f1a53b613dcf994" exitCode=0
Jul 09 09:43:48.952077 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:48.951819    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-wzg5j" event={"ID":"02e38491-5d3e-4457-9df1-eda978e82010","Type":"ContainerDied","Data":"538341fe6b5726c593cbf51d8e2ef5989fbceb67281798a59f1a53b613dcf994"}
Jul 09 09:43:49.958088 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:49.958053    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-wzg5j" event={"ID":"02e38491-5d3e-4457-9df1-eda978e82010","Type":"ContainerStarted","Data":"cb59831aec5fe3d2e0f92e3a874ef091baa49cb38913d5a12760befd95fc2ffd"}
Jul 09 09:43:49.977035 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:49.976988    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-tls-init-serving-wzg5j" podStartSLOduration=2.101978891 podStartE2EDuration="3.976973466s" totalImagesPullingTime="0s" totalInitContainerRuntime="1.874994575s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:43:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:43:49.975705671 +0000 UTC m=+588.146864229" watchObservedRunningTime="2026-07-09 09:43:49.976973466 +0000 UTC m=+588.148132021"
Jul 09 09:43:55.987134 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:55.987101    2549 generic.go:356] "Generic (PLEG): container finished" podID="02e38491-5d3e-4457-9df1-eda978e82010" containerID="cb59831aec5fe3d2e0f92e3a874ef091baa49cb38913d5a12760befd95fc2ffd" exitCode=0
Jul 09 09:43:55.987472 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:55.987153    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-wzg5j" event={"ID":"02e38491-5d3e-4457-9df1-eda978e82010","Type":"ContainerDied","Data":"cb59831aec5fe3d2e0f92e3a874ef091baa49cb38913d5a12760befd95fc2ffd"}
Jul 09 09:43:57.130135 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:57.130112    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-serving-wzg5j"
Jul 09 09:43:57.223234 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:57.223204    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8h6n\" (UniqueName: \"kubernetes.io/projected/02e38491-5d3e-4457-9df1-eda978e82010-kube-api-access-v8h6n\") pod \"02e38491-5d3e-4457-9df1-eda978e82010\" (UID: \"02e38491-5d3e-4457-9df1-eda978e82010\") "
Jul 09 09:43:57.223396 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:57.223295    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02e38491-5d3e-4457-9df1-eda978e82010-model-cache\") pod \"02e38491-5d3e-4457-9df1-eda978e82010\" (UID: \"02e38491-5d3e-4457-9df1-eda978e82010\") "
Jul 09 09:43:57.225215 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:57.225191    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02e38491-5d3e-4457-9df1-eda978e82010-kube-api-access-v8h6n" (OuterVolumeSpecName: "kube-api-access-v8h6n") pod "02e38491-5d3e-4457-9df1-eda978e82010" (UID: "02e38491-5d3e-4457-9df1-eda978e82010"). InnerVolumeSpecName "kube-api-access-v8h6n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:43:57.248124 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:57.248071    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02e38491-5d3e-4457-9df1-eda978e82010-model-cache" (OuterVolumeSpecName: "model-cache") pod "02e38491-5d3e-4457-9df1-eda978e82010" (UID: "02e38491-5d3e-4457-9df1-eda978e82010"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:43:57.324257 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:57.324233    2549 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02e38491-5d3e-4457-9df1-eda978e82010-model-cache\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:43:57.324257 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:57.324257    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8h6n\" (UniqueName: \"kubernetes.io/projected/02e38491-5d3e-4457-9df1-eda978e82010-kube-api-access-v8h6n\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:43:57.997970 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:57.997946    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-serving-wzg5j"
Jul 09 09:43:57.998141 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:57.997975    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-wzg5j" event={"ID":"02e38491-5d3e-4457-9df1-eda978e82010","Type":"ContainerDied","Data":"4a3a992fd333eb85ec777243717f07795a1ed34e97f07dc4d5fce4d731185869"}
Jul 09 09:43:57.998141 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:43:57.998005    2549 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4a3a992fd333eb85ec777243717f07795a1ed34e97f07dc4d5fce4d731185869"
Jul 09 09:44:10.506548 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:10.506517    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"]
Jul 09 09:44:10.507012 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:10.506889    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02e38491-5d3e-4457-9df1-eda978e82010" containerName="download-hf-model"
Jul 09 09:44:10.507012 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:10.506903    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="02e38491-5d3e-4457-9df1-eda978e82010" containerName="download-hf-model"
Jul 09 09:44:10.507012 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:10.506945    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02e38491-5d3e-4457-9df1-eda978e82010" containerName="s3-tls-init-serving"
Jul 09 09:44:10.507012 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:10.506954    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="02e38491-5d3e-4457-9df1-eda978e82010" containerName="s3-tls-init-serving"
Jul 09 09:44:10.507209 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:10.507034    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="02e38491-5d3e-4457-9df1-eda978e82010" containerName="s3-tls-init-serving"
Jul 09 09:44:10.510422 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:10.510406    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"
Jul 09 09:44:10.513227 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:10.513190    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 09 09:44:10.513227 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:10.513202    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 09:44:10.513227 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:10.513214    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-predictor-serving-cert\""
Jul 09 09:44:10.513476 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:10.513228    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-2mhmw\""
Jul 09 09:44:10.513476 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:10.513246    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 09:44:10.513476 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:10.513271    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-kube-rbac-proxy-sar-config\""
Jul 09 09:44:10.513649 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:10.513631    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 09 09:44:10.521969 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:10.521943    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"]
Jul 09 09:44:10.635836 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:10.635805    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6be1933a-7790-4515-9be3-86df8ae542f4-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46\" (UID: \"6be1933a-7790-4515-9be3-86df8ae542f4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"
Jul 09 09:44:10.635836 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:10.635836    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6be1933a-7790-4515-9be3-86df8ae542f4-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46\" (UID: \"6be1933a-7790-4515-9be3-86df8ae542f4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"
Jul 09 09:44:10.636062 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:10.635895    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6be1933a-7790-4515-9be3-86df8ae542f4-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46\" (UID: \"6be1933a-7790-4515-9be3-86df8ae542f4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"
Jul 09 09:44:10.636062 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:10.635963    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6be1933a-7790-4515-9be3-86df8ae542f4-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46\" (UID: \"6be1933a-7790-4515-9be3-86df8ae542f4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"
Jul 09 09:44:10.636062 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:10.636030    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxxr7\" (UniqueName: \"kubernetes.io/projected/6be1933a-7790-4515-9be3-86df8ae542f4-kube-api-access-pxxr7\") pod \"isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46\" (UID: \"6be1933a-7790-4515-9be3-86df8ae542f4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"
Jul 09 09:44:10.737431 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:10.737398    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6be1933a-7790-4515-9be3-86df8ae542f4-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46\" (UID: \"6be1933a-7790-4515-9be3-86df8ae542f4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"
Jul 09 09:44:10.737573 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:10.737455    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxxr7\" (UniqueName: \"kubernetes.io/projected/6be1933a-7790-4515-9be3-86df8ae542f4-kube-api-access-pxxr7\") pod \"isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46\" (UID: \"6be1933a-7790-4515-9be3-86df8ae542f4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"
Jul 09 09:44:10.737573 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:10.737500    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6be1933a-7790-4515-9be3-86df8ae542f4-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46\" (UID: \"6be1933a-7790-4515-9be3-86df8ae542f4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"
Jul 09 09:44:10.737573 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:10.737517    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6be1933a-7790-4515-9be3-86df8ae542f4-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46\" (UID: \"6be1933a-7790-4515-9be3-86df8ae542f4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"
Jul 09 09:44:10.737573 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:10.737542    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6be1933a-7790-4515-9be3-86df8ae542f4-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46\" (UID: \"6be1933a-7790-4515-9be3-86df8ae542f4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"
Jul 09 09:44:10.737737 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:44:10.737642    2549 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-serving-cert: secret "isvc-sklearn-batcher-predictor-serving-cert" not found
Jul 09 09:44:10.737782 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:44:10.737746    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6be1933a-7790-4515-9be3-86df8ae542f4-proxy-tls podName:6be1933a-7790-4515-9be3-86df8ae542f4 nodeName:}" failed. No retries permitted until 2026-07-09 09:44:11.237722759 +0000 UTC m=+609.408881310 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/6be1933a-7790-4515-9be3-86df8ae542f4-proxy-tls") pod "isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" (UID: "6be1933a-7790-4515-9be3-86df8ae542f4") : secret "isvc-sklearn-batcher-predictor-serving-cert" not found
Jul 09 09:44:10.737930 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:10.737915    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6be1933a-7790-4515-9be3-86df8ae542f4-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46\" (UID: \"6be1933a-7790-4515-9be3-86df8ae542f4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"
Jul 09 09:44:10.738198 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:10.738180    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6be1933a-7790-4515-9be3-86df8ae542f4-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46\" (UID: \"6be1933a-7790-4515-9be3-86df8ae542f4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"
Jul 09 09:44:10.738198 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:10.738191    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6be1933a-7790-4515-9be3-86df8ae542f4-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46\" (UID: \"6be1933a-7790-4515-9be3-86df8ae542f4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"
Jul 09 09:44:10.746220 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:10.746193    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxxr7\" (UniqueName: \"kubernetes.io/projected/6be1933a-7790-4515-9be3-86df8ae542f4-kube-api-access-pxxr7\") pod \"isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46\" (UID: \"6be1933a-7790-4515-9be3-86df8ae542f4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"
Jul 09 09:44:11.241430 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:11.241392    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6be1933a-7790-4515-9be3-86df8ae542f4-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46\" (UID: \"6be1933a-7790-4515-9be3-86df8ae542f4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"
Jul 09 09:44:11.243781 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:11.243761    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6be1933a-7790-4515-9be3-86df8ae542f4-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46\" (UID: \"6be1933a-7790-4515-9be3-86df8ae542f4\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"
Jul 09 09:44:11.421352 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:11.421322    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"
Jul 09 09:44:11.556263 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:11.556240    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"]
Jul 09 09:44:11.558463 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:44:11.558434    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6be1933a_7790_4515_9be3_86df8ae542f4.slice/crio-ece6f368f16320e0e7dc8a9afff49805d2665c4654d7c4e0d482ae433500dd0e WatchSource:0}: Error finding container ece6f368f16320e0e7dc8a9afff49805d2665c4654d7c4e0d482ae433500dd0e: Status 404 returned error can't find the container with id ece6f368f16320e0e7dc8a9afff49805d2665c4654d7c4e0d482ae433500dd0e
Jul 09 09:44:12.060142 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:12.060098    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" event={"ID":"6be1933a-7790-4515-9be3-86df8ae542f4","Type":"ContainerStarted","Data":"3cec45a4881afa1ff84de71e637942de15d912174f6f32a1fc76aa71ef9fbd00"}
Jul 09 09:44:12.060309 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:12.060187    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" event={"ID":"6be1933a-7790-4515-9be3-86df8ae542f4","Type":"ContainerStarted","Data":"ece6f368f16320e0e7dc8a9afff49805d2665c4654d7c4e0d482ae433500dd0e"}
Jul 09 09:44:16.080522 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:16.080490    2549 generic.go:356] "Generic (PLEG): container finished" podID="6be1933a-7790-4515-9be3-86df8ae542f4" containerID="3cec45a4881afa1ff84de71e637942de15d912174f6f32a1fc76aa71ef9fbd00" exitCode=0
Jul 09 09:44:16.080998 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:16.080568    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" event={"ID":"6be1933a-7790-4515-9be3-86df8ae542f4","Type":"ContainerDied","Data":"3cec45a4881afa1ff84de71e637942de15d912174f6f32a1fc76aa71ef9fbd00"}
Jul 09 09:44:31.175141 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:31.175064    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" event={"ID":"6be1933a-7790-4515-9be3-86df8ae542f4","Type":"ContainerStarted","Data":"c5bbadea48966b1636268d9ecd59fb3e13c454fb7b1d5cc6d13d703d9fbfe6e9"}
Jul 09 09:44:33.188007 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:33.187975    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" event={"ID":"6be1933a-7790-4515-9be3-86df8ae542f4","Type":"ContainerStarted","Data":"ce0c24c944163ed64227b483e4b2b6265bc4fd6fccb192ceeba6ea1cc47bc743"}
Jul 09 09:44:35.200459 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:35.200426    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" event={"ID":"6be1933a-7790-4515-9be3-86df8ae542f4","Type":"ContainerStarted","Data":"055fbface2b7c77a993d9bd8aae972f8a782e1afc8a7435f960ec4480a07c0fc"}
Jul 09 09:44:35.200837 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:35.200642    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"
Jul 09 09:44:35.200837 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:35.200669    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"
Jul 09 09:44:35.201681 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:35.201653    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Jul 09 09:44:35.223266 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:35.223220    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podStartSLOduration=3.179096033 podStartE2EDuration="25.223208199s" totalImagesPullingTime="18.418689069s" totalInitContainerRuntime="3.625423097s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:44:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:44:35.22069593 +0000 UTC m=+633.391854495" watchObservedRunningTime="2026-07-09 09:44:35.223208199 +0000 UTC m=+633.394366755"
Jul 09 09:44:36.205477 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:36.205430    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Jul 09 09:44:36.205477 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:36.205458    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"
Jul 09 09:44:36.206599 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:36.206570    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:44:37.210115 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:37.210076    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Jul 09 09:44:37.210571 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:37.210471    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:44:37.213921 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:37.213899    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"
Jul 09 09:44:38.219960 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:38.219920    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Jul 09 09:44:38.220336 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:38.220212    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:44:48.219797 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:48.219753    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Jul 09 09:44:48.220249 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:48.220163    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:44:58.220238 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:58.220193    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Jul 09 09:44:58.220754 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:44:58.220728    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:45:08.219966 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:08.219925    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Jul 09 09:45:08.220504 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:08.220320    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:45:18.220640 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:18.220591    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Jul 09 09:45:18.221137 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:18.221031    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:45:28.219767 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:28.219723    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Jul 09 09:45:28.220234 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:28.220211    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:45:38.220717 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:38.220667    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Jul 09 09:45:38.221234 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:38.221140    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:45:48.221063 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:48.221032    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"
Jul 09 09:45:48.221673 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:48.221447    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"
Jul 09 09:45:55.799149 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:55.798972    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"]
Jul 09 09:45:55.800254 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:55.800215    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="kserve-container" containerID="cri-o://c5bbadea48966b1636268d9ecd59fb3e13c454fb7b1d5cc6d13d703d9fbfe6e9" gracePeriod=30
Jul 09 09:45:55.800921 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:55.800477    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="agent" containerID="cri-o://055fbface2b7c77a993d9bd8aae972f8a782e1afc8a7435f960ec4480a07c0fc" gracePeriod=30
Jul 09 09:45:55.800921 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:55.800777    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="kube-rbac-proxy" containerID="cri-o://ce0c24c944163ed64227b483e4b2b6265bc4fd6fccb192ceeba6ea1cc47bc743" gracePeriod=30
Jul 09 09:45:55.829835 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:55.829807    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"]
Jul 09 09:45:55.833137 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:55.833122    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"
Jul 09 09:45:55.835477 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:55.835457    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-custom-kube-rbac-proxy-sar-config\""
Jul 09 09:45:55.835571 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:55.835512    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-custom-predictor-serving-cert\""
Jul 09 09:45:55.849929 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:55.849908    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"]
Jul 09 09:45:55.939183 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:55.939154    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2c3f2b62-e10b-42fe-99d8-2163cb678e39-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59\" (UID: \"2c3f2b62-e10b-42fe-99d8-2163cb678e39\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"
Jul 09 09:45:55.939343 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:55.939270    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2c3f2b62-e10b-42fe-99d8-2163cb678e39-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59\" (UID: \"2c3f2b62-e10b-42fe-99d8-2163cb678e39\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"
Jul 09 09:45:55.939343 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:55.939310    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tt5w\" (UniqueName: \"kubernetes.io/projected/2c3f2b62-e10b-42fe-99d8-2163cb678e39-kube-api-access-7tt5w\") pod \"isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59\" (UID: \"2c3f2b62-e10b-42fe-99d8-2163cb678e39\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"
Jul 09 09:45:55.939430 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:55.939344    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2c3f2b62-e10b-42fe-99d8-2163cb678e39-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59\" (UID: \"2c3f2b62-e10b-42fe-99d8-2163cb678e39\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"
Jul 09 09:45:55.939430 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:55.939414    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2c3f2b62-e10b-42fe-99d8-2163cb678e39-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59\" (UID: \"2c3f2b62-e10b-42fe-99d8-2163cb678e39\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"
Jul 09 09:45:56.040061 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:56.040031    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2c3f2b62-e10b-42fe-99d8-2163cb678e39-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59\" (UID: \"2c3f2b62-e10b-42fe-99d8-2163cb678e39\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"
Jul 09 09:45:56.040061 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:56.040066    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tt5w\" (UniqueName: \"kubernetes.io/projected/2c3f2b62-e10b-42fe-99d8-2163cb678e39-kube-api-access-7tt5w\") pod \"isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59\" (UID: \"2c3f2b62-e10b-42fe-99d8-2163cb678e39\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"
Jul 09 09:45:56.040281 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:56.040091    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2c3f2b62-e10b-42fe-99d8-2163cb678e39-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59\" (UID: \"2c3f2b62-e10b-42fe-99d8-2163cb678e39\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"
Jul 09 09:45:56.040281 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:56.040123    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2c3f2b62-e10b-42fe-99d8-2163cb678e39-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59\" (UID: \"2c3f2b62-e10b-42fe-99d8-2163cb678e39\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"
Jul 09 09:45:56.040281 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:56.040157    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2c3f2b62-e10b-42fe-99d8-2163cb678e39-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59\" (UID: \"2c3f2b62-e10b-42fe-99d8-2163cb678e39\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"
Jul 09 09:45:56.040648 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:56.040626    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2c3f2b62-e10b-42fe-99d8-2163cb678e39-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59\" (UID: \"2c3f2b62-e10b-42fe-99d8-2163cb678e39\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"
Jul 09 09:45:56.040921 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:56.040901    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2c3f2b62-e10b-42fe-99d8-2163cb678e39-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59\" (UID: \"2c3f2b62-e10b-42fe-99d8-2163cb678e39\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"
Jul 09 09:45:56.040921 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:56.040908    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2c3f2b62-e10b-42fe-99d8-2163cb678e39-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59\" (UID: \"2c3f2b62-e10b-42fe-99d8-2163cb678e39\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"
Jul 09 09:45:56.042393 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:56.042372    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2c3f2b62-e10b-42fe-99d8-2163cb678e39-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59\" (UID: \"2c3f2b62-e10b-42fe-99d8-2163cb678e39\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"
Jul 09 09:45:56.053220 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:56.053173    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tt5w\" (UniqueName: \"kubernetes.io/projected/2c3f2b62-e10b-42fe-99d8-2163cb678e39-kube-api-access-7tt5w\") pod \"isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59\" (UID: \"2c3f2b62-e10b-42fe-99d8-2163cb678e39\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"
Jul 09 09:45:56.143374 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:56.143345    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"
Jul 09 09:45:56.277551 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:56.277524    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"]
Jul 09 09:45:56.279490 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:45:56.279466    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c3f2b62_e10b_42fe_99d8_2163cb678e39.slice/crio-e924c3ce4b1dec6e5fbc17bdb3f444043d2cfab9ae75a5195e50643b561fd504 WatchSource:0}: Error finding container e924c3ce4b1dec6e5fbc17bdb3f444043d2cfab9ae75a5195e50643b561fd504: Status 404 returned error can't find the container with id e924c3ce4b1dec6e5fbc17bdb3f444043d2cfab9ae75a5195e50643b561fd504
Jul 09 09:45:56.573363 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:56.573270    2549 generic.go:356] "Generic (PLEG): container finished" podID="6be1933a-7790-4515-9be3-86df8ae542f4" containerID="ce0c24c944163ed64227b483e4b2b6265bc4fd6fccb192ceeba6ea1cc47bc743" exitCode=2
Jul 09 09:45:56.573363 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:56.573329    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" event={"ID":"6be1933a-7790-4515-9be3-86df8ae542f4","Type":"ContainerDied","Data":"ce0c24c944163ed64227b483e4b2b6265bc4fd6fccb192ceeba6ea1cc47bc743"}
Jul 09 09:45:56.574857 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:56.574832    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" event={"ID":"2c3f2b62-e10b-42fe-99d8-2163cb678e39","Type":"ContainerStarted","Data":"9864b363a0234d558d0167183c14658228523db25981cb829f6e6a6e963025e7"}
Jul 09 09:45:56.574995 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:56.574902    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" event={"ID":"2c3f2b62-e10b-42fe-99d8-2163cb678e39","Type":"ContainerStarted","Data":"e924c3ce4b1dec6e5fbc17bdb3f444043d2cfab9ae75a5195e50643b561fd504"}
Jul 09 09:45:57.210992 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:57.210954    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.33:8643/healthz\": dial tcp 10.134.0.33:8643: connect: connection refused"
Jul 09 09:45:58.220341 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:58.220304    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Jul 09 09:45:58.221905 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:45:58.221861    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:46:00.594551 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:00.594518    2549 generic.go:356] "Generic (PLEG): container finished" podID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerID="9864b363a0234d558d0167183c14658228523db25981cb829f6e6a6e963025e7" exitCode=0
Jul 09 09:46:00.595056 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:00.594596    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" event={"ID":"2c3f2b62-e10b-42fe-99d8-2163cb678e39","Type":"ContainerDied","Data":"9864b363a0234d558d0167183c14658228523db25981cb829f6e6a6e963025e7"}
Jul 09 09:46:00.597369 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:00.597345    2549 generic.go:356] "Generic (PLEG): container finished" podID="6be1933a-7790-4515-9be3-86df8ae542f4" containerID="c5bbadea48966b1636268d9ecd59fb3e13c454fb7b1d5cc6d13d703d9fbfe6e9" exitCode=0
Jul 09 09:46:00.597453 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:00.597410    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" event={"ID":"6be1933a-7790-4515-9be3-86df8ae542f4","Type":"ContainerDied","Data":"c5bbadea48966b1636268d9ecd59fb3e13c454fb7b1d5cc6d13d703d9fbfe6e9"}
Jul 09 09:46:01.604235 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:01.604202    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" event={"ID":"2c3f2b62-e10b-42fe-99d8-2163cb678e39","Type":"ContainerStarted","Data":"e49e33380ecfd8725d2369eb3f53c5ced6b26d20d4126e443610909438b588d5"}
Jul 09 09:46:01.604749 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:01.604293    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" event={"ID":"2c3f2b62-e10b-42fe-99d8-2163cb678e39","Type":"ContainerStarted","Data":"8133717aa948924d4878b70eaf58293af432699b86cb2eba06aaea5d9d531346"}
Jul 09 09:46:01.604749 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:01.604319    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" event={"ID":"2c3f2b62-e10b-42fe-99d8-2163cb678e39","Type":"ContainerStarted","Data":"782b744435889c5f5586b97a9ad92285d7475f932bef4a9ab5e6bf689cf9b36d"}
Jul 09 09:46:01.604749 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:01.604600    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"
Jul 09 09:46:01.604749 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:01.604640    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"
Jul 09 09:46:01.605990 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:01.605965    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:5000: connect: connection refused"
Jul 09 09:46:01.630932 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:01.630883    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podStartSLOduration=3.12298032 podStartE2EDuration="6.630850643s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.507870323s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:45:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:46:01.627953546 +0000 UTC m=+719.799112102" watchObservedRunningTime="2026-07-09 09:46:01.630850643 +0000 UTC m=+719.802009232"
Jul 09 09:46:02.211113 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:02.211078    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.33:8643/healthz\": dial tcp 10.134.0.33:8643: connect: connection refused"
Jul 09 09:46:02.609229 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:02.609195    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:5000: connect: connection refused"
Jul 09 09:46:02.609679 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:02.609293    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"
Jul 09 09:46:02.610372 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:02.610352    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:46:03.613797 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:03.613759    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:5000: connect: connection refused"
Jul 09 09:46:03.614274 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:03.614100    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:46:07.210541 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:07.210499    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.33:8643/healthz\": dial tcp 10.134.0.33:8643: connect: connection refused"
Jul 09 09:46:07.211004 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:07.210649    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"
Jul 09 09:46:08.220594 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:08.220550    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Jul 09 09:46:08.222166 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:08.222136    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:46:08.617853 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:08.617828    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"
Jul 09 09:46:08.618467 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:08.618432    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:5000: connect: connection refused"
Jul 09 09:46:08.618726 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:08.618703    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:46:12.211033 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:12.210994    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.33:8643/healthz\": dial tcp 10.134.0.33:8643: connect: connection refused"
Jul 09 09:46:17.210852 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:17.210814    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.33:8643/healthz\": dial tcp 10.134.0.33:8643: connect: connection refused"
Jul 09 09:46:18.219990 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:18.219946    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Jul 09 09:46:18.220373 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:18.220093    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"
Jul 09 09:46:18.222544 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:18.222521    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:46:18.222639 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:18.222628    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"
Jul 09 09:46:18.618599 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:18.618555    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:5000: connect: connection refused"
Jul 09 09:46:18.619002 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:18.618979    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:46:22.210922 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:22.210884    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.33:8643/healthz\": dial tcp 10.134.0.33:8643: connect: connection refused"
Jul 09 09:46:25.949319 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:25.949295    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"
Jul 09 09:46:26.093499 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.093428    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6be1933a-7790-4515-9be3-86df8ae542f4-kube-rbac-proxy-sar-config\") pod \"6be1933a-7790-4515-9be3-86df8ae542f4\" (UID: \"6be1933a-7790-4515-9be3-86df8ae542f4\") "
Jul 09 09:46:26.093499 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.093480    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6be1933a-7790-4515-9be3-86df8ae542f4-cabundle-cert\") pod \"6be1933a-7790-4515-9be3-86df8ae542f4\" (UID: \"6be1933a-7790-4515-9be3-86df8ae542f4\") "
Jul 09 09:46:26.093670 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.093526    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxxr7\" (UniqueName: \"kubernetes.io/projected/6be1933a-7790-4515-9be3-86df8ae542f4-kube-api-access-pxxr7\") pod \"6be1933a-7790-4515-9be3-86df8ae542f4\" (UID: \"6be1933a-7790-4515-9be3-86df8ae542f4\") "
Jul 09 09:46:26.093670 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.093548    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6be1933a-7790-4515-9be3-86df8ae542f4-kserve-provision-location\") pod \"6be1933a-7790-4515-9be3-86df8ae542f4\" (UID: \"6be1933a-7790-4515-9be3-86df8ae542f4\") "
Jul 09 09:46:26.093670 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.093616    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6be1933a-7790-4515-9be3-86df8ae542f4-proxy-tls\") pod \"6be1933a-7790-4515-9be3-86df8ae542f4\" (UID: \"6be1933a-7790-4515-9be3-86df8ae542f4\") "
Jul 09 09:46:26.093919 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.093894    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6be1933a-7790-4515-9be3-86df8ae542f4-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "6be1933a-7790-4515-9be3-86df8ae542f4" (UID: "6be1933a-7790-4515-9be3-86df8ae542f4"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:46:26.093998 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.093902    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6be1933a-7790-4515-9be3-86df8ae542f4-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "6be1933a-7790-4515-9be3-86df8ae542f4" (UID: "6be1933a-7790-4515-9be3-86df8ae542f4"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:46:26.093998 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.093901    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6be1933a-7790-4515-9be3-86df8ae542f4-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "6be1933a-7790-4515-9be3-86df8ae542f4" (UID: "6be1933a-7790-4515-9be3-86df8ae542f4"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:46:26.095782 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.095758    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6be1933a-7790-4515-9be3-86df8ae542f4-kube-api-access-pxxr7" (OuterVolumeSpecName: "kube-api-access-pxxr7") pod "6be1933a-7790-4515-9be3-86df8ae542f4" (UID: "6be1933a-7790-4515-9be3-86df8ae542f4"). InnerVolumeSpecName "kube-api-access-pxxr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:46:26.095894 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.095759    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6be1933a-7790-4515-9be3-86df8ae542f4-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "6be1933a-7790-4515-9be3-86df8ae542f4" (UID: "6be1933a-7790-4515-9be3-86df8ae542f4"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:46:26.195187 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.195157    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxxr7\" (UniqueName: \"kubernetes.io/projected/6be1933a-7790-4515-9be3-86df8ae542f4-kube-api-access-pxxr7\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:46:26.195187 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.195186    2549 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6be1933a-7790-4515-9be3-86df8ae542f4-kserve-provision-location\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:46:26.195360 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.195198    2549 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6be1933a-7790-4515-9be3-86df8ae542f4-proxy-tls\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:46:26.195360 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.195210    2549 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6be1933a-7790-4515-9be3-86df8ae542f4-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:46:26.195360 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.195223    2549 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6be1933a-7790-4515-9be3-86df8ae542f4-cabundle-cert\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:46:26.721847 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.721767    2549 generic.go:356] "Generic (PLEG): container finished" podID="6be1933a-7790-4515-9be3-86df8ae542f4" containerID="055fbface2b7c77a993d9bd8aae972f8a782e1afc8a7435f960ec4480a07c0fc" exitCode=0
Jul 09 09:46:26.721847 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.721811    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"
Jul 09 09:46:26.721847 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.721837    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" event={"ID":"6be1933a-7790-4515-9be3-86df8ae542f4","Type":"ContainerDied","Data":"055fbface2b7c77a993d9bd8aae972f8a782e1afc8a7435f960ec4480a07c0fc"}
Jul 09 09:46:26.722146 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.721930    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46" event={"ID":"6be1933a-7790-4515-9be3-86df8ae542f4","Type":"ContainerDied","Data":"ece6f368f16320e0e7dc8a9afff49805d2665c4654d7c4e0d482ae433500dd0e"}
Jul 09 09:46:26.722146 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.721974    2549 scope.go:117] "RemoveContainer" containerID="055fbface2b7c77a993d9bd8aae972f8a782e1afc8a7435f960ec4480a07c0fc"
Jul 09 09:46:26.729886 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.729858    2549 scope.go:117] "RemoveContainer" containerID="ce0c24c944163ed64227b483e4b2b6265bc4fd6fccb192ceeba6ea1cc47bc743"
Jul 09 09:46:26.740176 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.740157    2549 scope.go:117] "RemoveContainer" containerID="c5bbadea48966b1636268d9ecd59fb3e13c454fb7b1d5cc6d13d703d9fbfe6e9"
Jul 09 09:46:26.744748 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.744704    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"]
Jul 09 09:46:26.748703 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.748682    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-54dcb69b4c-mgf46"]
Jul 09 09:46:26.749579 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.749565    2549 scope.go:117] "RemoveContainer" containerID="3cec45a4881afa1ff84de71e637942de15d912174f6f32a1fc76aa71ef9fbd00"
Jul 09 09:46:26.756412 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.756394    2549 scope.go:117] "RemoveContainer" containerID="055fbface2b7c77a993d9bd8aae972f8a782e1afc8a7435f960ec4480a07c0fc"
Jul 09 09:46:26.756634 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:46:26.756619    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"055fbface2b7c77a993d9bd8aae972f8a782e1afc8a7435f960ec4480a07c0fc\": container with ID starting with 055fbface2b7c77a993d9bd8aae972f8a782e1afc8a7435f960ec4480a07c0fc not found: ID does not exist" containerID="055fbface2b7c77a993d9bd8aae972f8a782e1afc8a7435f960ec4480a07c0fc"
Jul 09 09:46:26.756689 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.756642    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"055fbface2b7c77a993d9bd8aae972f8a782e1afc8a7435f960ec4480a07c0fc"} err="failed to get container status \"055fbface2b7c77a993d9bd8aae972f8a782e1afc8a7435f960ec4480a07c0fc\": rpc error: code = NotFound desc = could not find container \"055fbface2b7c77a993d9bd8aae972f8a782e1afc8a7435f960ec4480a07c0fc\": container with ID starting with 055fbface2b7c77a993d9bd8aae972f8a782e1afc8a7435f960ec4480a07c0fc not found: ID does not exist"
Jul 09 09:46:26.756689 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.756657    2549 scope.go:117] "RemoveContainer" containerID="ce0c24c944163ed64227b483e4b2b6265bc4fd6fccb192ceeba6ea1cc47bc743"
Jul 09 09:46:26.756913 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:46:26.756894    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce0c24c944163ed64227b483e4b2b6265bc4fd6fccb192ceeba6ea1cc47bc743\": container with ID starting with ce0c24c944163ed64227b483e4b2b6265bc4fd6fccb192ceeba6ea1cc47bc743 not found: ID does not exist" containerID="ce0c24c944163ed64227b483e4b2b6265bc4fd6fccb192ceeba6ea1cc47bc743"
Jul 09 09:46:26.756991 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.756923    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce0c24c944163ed64227b483e4b2b6265bc4fd6fccb192ceeba6ea1cc47bc743"} err="failed to get container status \"ce0c24c944163ed64227b483e4b2b6265bc4fd6fccb192ceeba6ea1cc47bc743\": rpc error: code = NotFound desc = could not find container \"ce0c24c944163ed64227b483e4b2b6265bc4fd6fccb192ceeba6ea1cc47bc743\": container with ID starting with ce0c24c944163ed64227b483e4b2b6265bc4fd6fccb192ceeba6ea1cc47bc743 not found: ID does not exist"
Jul 09 09:46:26.756991 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.756948    2549 scope.go:117] "RemoveContainer" containerID="c5bbadea48966b1636268d9ecd59fb3e13c454fb7b1d5cc6d13d703d9fbfe6e9"
Jul 09 09:46:26.757187 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:46:26.757170    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5bbadea48966b1636268d9ecd59fb3e13c454fb7b1d5cc6d13d703d9fbfe6e9\": container with ID starting with c5bbadea48966b1636268d9ecd59fb3e13c454fb7b1d5cc6d13d703d9fbfe6e9 not found: ID does not exist" containerID="c5bbadea48966b1636268d9ecd59fb3e13c454fb7b1d5cc6d13d703d9fbfe6e9"
Jul 09 09:46:26.757226 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.757193    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5bbadea48966b1636268d9ecd59fb3e13c454fb7b1d5cc6d13d703d9fbfe6e9"} err="failed to get container status \"c5bbadea48966b1636268d9ecd59fb3e13c454fb7b1d5cc6d13d703d9fbfe6e9\": rpc error: code = NotFound desc = could not find container \"c5bbadea48966b1636268d9ecd59fb3e13c454fb7b1d5cc6d13d703d9fbfe6e9\": container with ID starting with c5bbadea48966b1636268d9ecd59fb3e13c454fb7b1d5cc6d13d703d9fbfe6e9 not found: ID does not exist"
Jul 09 09:46:26.757226 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.757206    2549 scope.go:117] "RemoveContainer" containerID="3cec45a4881afa1ff84de71e637942de15d912174f6f32a1fc76aa71ef9fbd00"
Jul 09 09:46:26.757413 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:46:26.757396    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cec45a4881afa1ff84de71e637942de15d912174f6f32a1fc76aa71ef9fbd00\": container with ID starting with 3cec45a4881afa1ff84de71e637942de15d912174f6f32a1fc76aa71ef9fbd00 not found: ID does not exist" containerID="3cec45a4881afa1ff84de71e637942de15d912174f6f32a1fc76aa71ef9fbd00"
Jul 09 09:46:26.757468 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:26.757420    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cec45a4881afa1ff84de71e637942de15d912174f6f32a1fc76aa71ef9fbd00"} err="failed to get container status \"3cec45a4881afa1ff84de71e637942de15d912174f6f32a1fc76aa71ef9fbd00\": rpc error: code = NotFound desc = could not find container \"3cec45a4881afa1ff84de71e637942de15d912174f6f32a1fc76aa71ef9fbd00\": container with ID starting with 3cec45a4881afa1ff84de71e637942de15d912174f6f32a1fc76aa71ef9fbd00 not found: ID does not exist"
Jul 09 09:46:28.385260 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:28.385229    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" path="/var/lib/kubelet/pods/6be1933a-7790-4515-9be3-86df8ae542f4/volumes"
Jul 09 09:46:28.620854 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:28.620790    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:5000: connect: connection refused"
Jul 09 09:46:28.621568 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:28.621524    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:46:38.618298 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:38.618256    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:5000: connect: connection refused"
Jul 09 09:46:38.618829 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:38.618664    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:46:48.618363 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:48.618320    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:5000: connect: connection refused"
Jul 09 09:46:48.618856 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:48.618774    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:46:58.618493 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:58.618398    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:5000: connect: connection refused"
Jul 09 09:46:58.618914 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:46:58.618891    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:47:08.618919 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:08.618884    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:5000: connect: connection refused"
Jul 09 09:47:08.619344 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:08.619322    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:47:18.619417 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:18.619387    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"
Jul 09 09:47:18.619893 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:18.619585    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"
Jul 09 09:47:30.806635 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:30.806551    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"]
Jul 09 09:47:30.807108 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:30.806892    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="kserve-container" containerID="cri-o://782b744435889c5f5586b97a9ad92285d7475f932bef4a9ab5e6bf689cf9b36d" gracePeriod=30
Jul 09 09:47:30.807108 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:30.806941    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="kube-rbac-proxy" containerID="cri-o://8133717aa948924d4878b70eaf58293af432699b86cb2eba06aaea5d9d531346" gracePeriod=30
Jul 09 09:47:30.807108 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:30.806909    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="agent" containerID="cri-o://e49e33380ecfd8725d2369eb3f53c5ced6b26d20d4126e443610909438b588d5" gracePeriod=30
Jul 09 09:47:30.900696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:30.900666    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf"]
Jul 09 09:47:30.901063 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:30.901051    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="agent"
Jul 09 09:47:30.901117 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:30.901065    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="agent"
Jul 09 09:47:30.901117 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:30.901084    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="storage-initializer"
Jul 09 09:47:30.901117 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:30.901089    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="storage-initializer"
Jul 09 09:47:30.901117 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:30.901099    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="kube-rbac-proxy"
Jul 09 09:47:30.901117 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:30.901105    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="kube-rbac-proxy"
Jul 09 09:47:30.901117 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:30.901112    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="kserve-container"
Jul 09 09:47:30.901117 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:30.901117    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="kserve-container"
Jul 09 09:47:30.901330 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:30.901177    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="agent"
Jul 09 09:47:30.901330 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:30.901188    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="kserve-container"
Jul 09 09:47:30.901330 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:30.901194    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="6be1933a-7790-4515-9be3-86df8ae542f4" containerName="kube-rbac-proxy"
Jul 09 09:47:30.904042 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:30.904024    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf"
Jul 09 09:47:30.906301 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:30.906287    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-predictor-serving-cert\""
Jul 09 09:47:30.906301 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:30.906295    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-kube-rbac-proxy-sar-config\""
Jul 09 09:47:30.915113 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:30.915092    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf"]
Jul 09 09:47:31.014182 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:31.014150    2549 generic.go:356] "Generic (PLEG): container finished" podID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerID="8133717aa948924d4878b70eaf58293af432699b86cb2eba06aaea5d9d531346" exitCode=2
Jul 09 09:47:31.014327 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:31.014185    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" event={"ID":"2c3f2b62-e10b-42fe-99d8-2163cb678e39","Type":"ContainerDied","Data":"8133717aa948924d4878b70eaf58293af432699b86cb2eba06aaea5d9d531346"}
Jul 09 09:47:31.028586 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:31.028561    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/189b2206-4a46-4c59-bacb-320f1959a0a0-proxy-tls\") pod \"message-dumper-predictor-6575bcfdfd-jxrdf\" (UID: \"189b2206-4a46-4c59-bacb-320f1959a0a0\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf"
Jul 09 09:47:31.028712 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:31.028648    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ns8w\" (UniqueName: \"kubernetes.io/projected/189b2206-4a46-4c59-bacb-320f1959a0a0-kube-api-access-2ns8w\") pod \"message-dumper-predictor-6575bcfdfd-jxrdf\" (UID: \"189b2206-4a46-4c59-bacb-320f1959a0a0\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf"
Jul 09 09:47:31.028712 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:31.028669    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/189b2206-4a46-4c59-bacb-320f1959a0a0-kube-rbac-proxy-sar-config\") pod \"message-dumper-predictor-6575bcfdfd-jxrdf\" (UID: \"189b2206-4a46-4c59-bacb-320f1959a0a0\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf"
Jul 09 09:47:31.130015 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:31.129931    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ns8w\" (UniqueName: \"kubernetes.io/projected/189b2206-4a46-4c59-bacb-320f1959a0a0-kube-api-access-2ns8w\") pod \"message-dumper-predictor-6575bcfdfd-jxrdf\" (UID: \"189b2206-4a46-4c59-bacb-320f1959a0a0\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf"
Jul 09 09:47:31.130015 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:31.129978    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/189b2206-4a46-4c59-bacb-320f1959a0a0-kube-rbac-proxy-sar-config\") pod \"message-dumper-predictor-6575bcfdfd-jxrdf\" (UID: \"189b2206-4a46-4c59-bacb-320f1959a0a0\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf"
Jul 09 09:47:31.130015 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:31.130015    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/189b2206-4a46-4c59-bacb-320f1959a0a0-proxy-tls\") pod \"message-dumper-predictor-6575bcfdfd-jxrdf\" (UID: \"189b2206-4a46-4c59-bacb-320f1959a0a0\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf"
Jul 09 09:47:31.130253 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:47:31.130109    2549 secret.go:189] Couldn't get secret kserve-ci-e2e-test/message-dumper-predictor-serving-cert: secret "message-dumper-predictor-serving-cert" not found
Jul 09 09:47:31.130253 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:47:31.130165    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/189b2206-4a46-4c59-bacb-320f1959a0a0-proxy-tls podName:189b2206-4a46-4c59-bacb-320f1959a0a0 nodeName:}" failed. No retries permitted until 2026-07-09 09:47:31.630146085 +0000 UTC m=+809.801304622 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/189b2206-4a46-4c59-bacb-320f1959a0a0-proxy-tls") pod "message-dumper-predictor-6575bcfdfd-jxrdf" (UID: "189b2206-4a46-4c59-bacb-320f1959a0a0") : secret "message-dumper-predictor-serving-cert" not found
Jul 09 09:47:31.130606 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:31.130587    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/189b2206-4a46-4c59-bacb-320f1959a0a0-kube-rbac-proxy-sar-config\") pod \"message-dumper-predictor-6575bcfdfd-jxrdf\" (UID: \"189b2206-4a46-4c59-bacb-320f1959a0a0\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf"
Jul 09 09:47:31.140889 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:31.140842    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ns8w\" (UniqueName: \"kubernetes.io/projected/189b2206-4a46-4c59-bacb-320f1959a0a0-kube-api-access-2ns8w\") pod \"message-dumper-predictor-6575bcfdfd-jxrdf\" (UID: \"189b2206-4a46-4c59-bacb-320f1959a0a0\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf"
Jul 09 09:47:31.634326 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:31.634294    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/189b2206-4a46-4c59-bacb-320f1959a0a0-proxy-tls\") pod \"message-dumper-predictor-6575bcfdfd-jxrdf\" (UID: \"189b2206-4a46-4c59-bacb-320f1959a0a0\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf"
Jul 09 09:47:31.636643 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:31.636620    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/189b2206-4a46-4c59-bacb-320f1959a0a0-proxy-tls\") pod \"message-dumper-predictor-6575bcfdfd-jxrdf\" (UID: \"189b2206-4a46-4c59-bacb-320f1959a0a0\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf"
Jul 09 09:47:31.814860 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:31.814836    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf"
Jul 09 09:47:31.943831 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:31.943808    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf"]
Jul 09 09:47:31.945923 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:47:31.945894    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod189b2206_4a46_4c59_bacb_320f1959a0a0.slice/crio-3ea1bf1b9e93a69e56607fcf78acbfc4b39a40d3f2868d37d79cef0db45f7cff WatchSource:0}: Error finding container 3ea1bf1b9e93a69e56607fcf78acbfc4b39a40d3f2868d37d79cef0db45f7cff: Status 404 returned error can't find the container with id 3ea1bf1b9e93a69e56607fcf78acbfc4b39a40d3f2868d37d79cef0db45f7cff
Jul 09 09:47:31.948007 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:31.947989    2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 09:47:32.019979 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:32.019942    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf" event={"ID":"189b2206-4a46-4c59-bacb-320f1959a0a0","Type":"ContainerStarted","Data":"3ea1bf1b9e93a69e56607fcf78acbfc4b39a40d3f2868d37d79cef0db45f7cff"}
Jul 09 09:47:33.614922 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:33.614837    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.34:8643/healthz\": dial tcp 10.134.0.34:8643: connect: connection refused"
Jul 09 09:47:34.033591 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:34.033550    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf" event={"ID":"189b2206-4a46-4c59-bacb-320f1959a0a0","Type":"ContainerStarted","Data":"734f2ffce2d55ddec6118bd8fa71973d9d74e2f39ea1cf8f852349196b6a509d"}
Jul 09 09:47:34.033768 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:34.033627    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf" event={"ID":"189b2206-4a46-4c59-bacb-320f1959a0a0","Type":"ContainerStarted","Data":"06bb0228b0b72238cb69b49538bd4e403201b24952556442cbc06d637a90a6eb"}
Jul 09 09:47:34.033851 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:34.033837    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf"
Jul 09 09:47:34.034012 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:34.033991    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf"
Jul 09 09:47:34.035550 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:34.035527    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf"
Jul 09 09:47:34.054700 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:34.054660    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf" podStartSLOduration=2.698417555 podStartE2EDuration="4.054644699s" totalImagesPullingTime="1.356227144s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:47:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:47:34.051849985 +0000 UTC m=+812.223008542" watchObservedRunningTime="2026-07-09 09:47:34.054644699 +0000 UTC m=+812.225803316"
Jul 09 09:47:35.043293 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:35.043266    2549 generic.go:356] "Generic (PLEG): container finished" podID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerID="782b744435889c5f5586b97a9ad92285d7475f932bef4a9ab5e6bf689cf9b36d" exitCode=0
Jul 09 09:47:35.043601 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:35.043347    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" event={"ID":"2c3f2b62-e10b-42fe-99d8-2163cb678e39","Type":"ContainerDied","Data":"782b744435889c5f5586b97a9ad92285d7475f932bef4a9ab5e6bf689cf9b36d"}
Jul 09 09:47:38.614239 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:38.614202    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.34:8643/healthz\": dial tcp 10.134.0.34:8643: connect: connection refused"
Jul 09 09:47:38.618552 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:38.618525    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:5000: connect: connection refused"
Jul 09 09:47:38.618916 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:38.618889    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:47:41.051799 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:41.051772    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf"
Jul 09 09:47:43.614693 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:43.614654    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.34:8643/healthz\": dial tcp 10.134.0.34:8643: connect: connection refused"
Jul 09 09:47:43.615148 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:43.614800    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"
Jul 09 09:47:48.614614 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:48.614570    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.34:8643/healthz\": dial tcp 10.134.0.34:8643: connect: connection refused"
Jul 09 09:47:48.618897 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:48.618843    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:5000: connect: connection refused"
Jul 09 09:47:48.619179 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:48.619153    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:47:51.033597 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:51.033560    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"]
Jul 09 09:47:51.037813 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:51.037790    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"
Jul 09 09:47:51.040717 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:51.040695    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-logger-kube-rbac-proxy-sar-config\""
Jul 09 09:47:51.040837 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:51.040723    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-logger-predictor-serving-cert\""
Jul 09 09:47:51.052612 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:51.052593    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"]
Jul 09 09:47:51.102817 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:51.102792    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-kserve-provision-location\") pod \"isvc-logger-predictor-7997fc58c6-r7cl4\" (UID: \"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"
Jul 09 09:47:51.102941 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:51.102830    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk5wr\" (UniqueName: \"kubernetes.io/projected/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-kube-api-access-rk5wr\") pod \"isvc-logger-predictor-7997fc58c6-r7cl4\" (UID: \"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"
Jul 09 09:47:51.102998 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:51.102948    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-7997fc58c6-r7cl4\" (UID: \"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"
Jul 09 09:47:51.102998 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:51.102981    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-proxy-tls\") pod \"isvc-logger-predictor-7997fc58c6-r7cl4\" (UID: \"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"
Jul 09 09:47:51.103078 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:51.102997    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-cabundle-cert\") pod \"isvc-logger-predictor-7997fc58c6-r7cl4\" (UID: \"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"
Jul 09 09:47:51.204290 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:51.204263    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk5wr\" (UniqueName: \"kubernetes.io/projected/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-kube-api-access-rk5wr\") pod \"isvc-logger-predictor-7997fc58c6-r7cl4\" (UID: \"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"
Jul 09 09:47:51.204385 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:51.204338    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-7997fc58c6-r7cl4\" (UID: \"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"
Jul 09 09:47:51.204385 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:51.204361    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-proxy-tls\") pod \"isvc-logger-predictor-7997fc58c6-r7cl4\" (UID: \"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"
Jul 09 09:47:51.204385 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:51.204378    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-cabundle-cert\") pod \"isvc-logger-predictor-7997fc58c6-r7cl4\" (UID: \"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"
Jul 09 09:47:51.204532 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:51.204395    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-kserve-provision-location\") pod \"isvc-logger-predictor-7997fc58c6-r7cl4\" (UID: \"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"
Jul 09 09:47:51.204756 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:51.204731    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-kserve-provision-location\") pod \"isvc-logger-predictor-7997fc58c6-r7cl4\" (UID: \"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"
Jul 09 09:47:51.205063 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:51.205042    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-cabundle-cert\") pod \"isvc-logger-predictor-7997fc58c6-r7cl4\" (UID: \"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"
Jul 09 09:47:51.205168 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:51.205146    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-7997fc58c6-r7cl4\" (UID: \"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"
Jul 09 09:47:51.206750 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:51.206733    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-proxy-tls\") pod \"isvc-logger-predictor-7997fc58c6-r7cl4\" (UID: \"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"
Jul 09 09:47:51.213368 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:51.213345    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk5wr\" (UniqueName: \"kubernetes.io/projected/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-kube-api-access-rk5wr\") pod \"isvc-logger-predictor-7997fc58c6-r7cl4\" (UID: \"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"
Jul 09 09:47:51.347490 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:51.347433    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"
Jul 09 09:47:51.481438 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:51.481415    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"]
Jul 09 09:47:51.483826 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:47:51.483799    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod778f5ed3_0ef1_4f6a_81cd_4b22c84bbd4d.slice/crio-b7ad596e75078a7d09f8e2e13e0f906a3f6465aed4539dd2e73803407ee93a77 WatchSource:0}: Error finding container b7ad596e75078a7d09f8e2e13e0f906a3f6465aed4539dd2e73803407ee93a77: Status 404 returned error can't find the container with id b7ad596e75078a7d09f8e2e13e0f906a3f6465aed4539dd2e73803407ee93a77
Jul 09 09:47:52.128102 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:52.128069    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" event={"ID":"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d","Type":"ContainerStarted","Data":"3a21b1998ff995c8e3f78d59af4f370fd8b14b2314efe093e48ffb4aa07d31e4"}
Jul 09 09:47:52.128522 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:52.128131    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" event={"ID":"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d","Type":"ContainerStarted","Data":"b7ad596e75078a7d09f8e2e13e0f906a3f6465aed4539dd2e73803407ee93a77"}
Jul 09 09:47:53.614671 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:53.614627    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.34:8643/healthz\": dial tcp 10.134.0.34:8643: connect: connection refused"
Jul 09 09:47:56.147071 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:56.147036    2549 generic.go:356] "Generic (PLEG): container finished" podID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerID="3a21b1998ff995c8e3f78d59af4f370fd8b14b2314efe093e48ffb4aa07d31e4" exitCode=0
Jul 09 09:47:56.147467 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:56.147110    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" event={"ID":"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d","Type":"ContainerDied","Data":"3a21b1998ff995c8e3f78d59af4f370fd8b14b2314efe093e48ffb4aa07d31e4"}
Jul 09 09:47:57.154181 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:57.154147    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" event={"ID":"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d","Type":"ContainerStarted","Data":"2b4a8a02752c34d0d0c41324915853534048ba9c5fe7a8351b56c8d5f08c8635"}
Jul 09 09:47:57.154567 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:57.154215    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" event={"ID":"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d","Type":"ContainerStarted","Data":"7fb26030e823d03a43012f5127b3803118eb141497656e90b31307cc1b7b71dc"}
Jul 09 09:47:57.154567 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:57.154240    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" event={"ID":"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d","Type":"ContainerStarted","Data":"cdbfb51e61f064a787dedb1f91305562e59f1deaf8663042d82edd8433d32ece"}
Jul 09 09:47:57.154653 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:57.154586    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"
Jul 09 09:47:57.154653 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:57.154642    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"
Jul 09 09:47:57.155983 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:57.155950    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Jul 09 09:47:57.189200 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:57.189159    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podStartSLOduration=3.525969579 podStartE2EDuration="7.1891462s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.663176621s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:47:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:47:57.188717949 +0000 UTC m=+835.359876517" watchObservedRunningTime="2026-07-09 09:47:57.1891462 +0000 UTC m=+835.360304811"
Jul 09 09:47:58.158795 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:58.158753    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"
Jul 09 09:47:58.159341 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:58.159309    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Jul 09 09:47:58.160127 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:58.160104    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:47:58.613896 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:58.613840    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.34:8643/healthz\": dial tcp 10.134.0.34:8643: connect: connection refused"
Jul 09 09:47:58.619181 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:58.619157    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:5000: connect: connection refused"
Jul 09 09:47:58.619315 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:58.619299    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"
Jul 09 09:47:58.619534 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:58.619508    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:47:58.619656 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:58.619642    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"
Jul 09 09:47:59.164042 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:59.163995    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Jul 09 09:47:59.164460 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:47:59.164383    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:48:00.950181 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:00.950154    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"
Jul 09 09:48:00.997209 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:00.997182    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2c3f2b62-e10b-42fe-99d8-2163cb678e39-kserve-provision-location\") pod \"2c3f2b62-e10b-42fe-99d8-2163cb678e39\" (UID: \"2c3f2b62-e10b-42fe-99d8-2163cb678e39\") "
Jul 09 09:48:00.997379 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:00.997218    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2c3f2b62-e10b-42fe-99d8-2163cb678e39-kube-rbac-proxy-sar-config\") pod \"2c3f2b62-e10b-42fe-99d8-2163cb678e39\" (UID: \"2c3f2b62-e10b-42fe-99d8-2163cb678e39\") "
Jul 09 09:48:00.997379 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:00.997243    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2c3f2b62-e10b-42fe-99d8-2163cb678e39-proxy-tls\") pod \"2c3f2b62-e10b-42fe-99d8-2163cb678e39\" (UID: \"2c3f2b62-e10b-42fe-99d8-2163cb678e39\") "
Jul 09 09:48:00.997379 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:00.997281    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2c3f2b62-e10b-42fe-99d8-2163cb678e39-cabundle-cert\") pod \"2c3f2b62-e10b-42fe-99d8-2163cb678e39\" (UID: \"2c3f2b62-e10b-42fe-99d8-2163cb678e39\") "
Jul 09 09:48:00.997379 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:00.997336    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7tt5w\" (UniqueName: \"kubernetes.io/projected/2c3f2b62-e10b-42fe-99d8-2163cb678e39-kube-api-access-7tt5w\") pod \"2c3f2b62-e10b-42fe-99d8-2163cb678e39\" (UID: \"2c3f2b62-e10b-42fe-99d8-2163cb678e39\") "
Jul 09 09:48:00.997607 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:00.997556    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c3f2b62-e10b-42fe-99d8-2163cb678e39-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "2c3f2b62-e10b-42fe-99d8-2163cb678e39" (UID: "2c3f2b62-e10b-42fe-99d8-2163cb678e39"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:48:00.997607 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:00.997570    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c3f2b62-e10b-42fe-99d8-2163cb678e39-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "2c3f2b62-e10b-42fe-99d8-2163cb678e39" (UID: "2c3f2b62-e10b-42fe-99d8-2163cb678e39"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:48:00.997683 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:00.997652    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c3f2b62-e10b-42fe-99d8-2163cb678e39-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "2c3f2b62-e10b-42fe-99d8-2163cb678e39" (UID: "2c3f2b62-e10b-42fe-99d8-2163cb678e39"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:48:00.999411 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:00.999386    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c3f2b62-e10b-42fe-99d8-2163cb678e39-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "2c3f2b62-e10b-42fe-99d8-2163cb678e39" (UID: "2c3f2b62-e10b-42fe-99d8-2163cb678e39"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:48:00.999503 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:00.999424    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c3f2b62-e10b-42fe-99d8-2163cb678e39-kube-api-access-7tt5w" (OuterVolumeSpecName: "kube-api-access-7tt5w") pod "2c3f2b62-e10b-42fe-99d8-2163cb678e39" (UID: "2c3f2b62-e10b-42fe-99d8-2163cb678e39"). InnerVolumeSpecName "kube-api-access-7tt5w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:48:01.098609 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:01.098531    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tt5w\" (UniqueName: \"kubernetes.io/projected/2c3f2b62-e10b-42fe-99d8-2163cb678e39-kube-api-access-7tt5w\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:48:01.098609 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:01.098557    2549 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2c3f2b62-e10b-42fe-99d8-2163cb678e39-kserve-provision-location\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:48:01.098609 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:01.098566    2549 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2c3f2b62-e10b-42fe-99d8-2163cb678e39-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:48:01.098609 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:01.098576    2549 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2c3f2b62-e10b-42fe-99d8-2163cb678e39-proxy-tls\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:48:01.098609 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:01.098586    2549 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2c3f2b62-e10b-42fe-99d8-2163cb678e39-cabundle-cert\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:48:01.175602 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:01.175567    2549 generic.go:356] "Generic (PLEG): container finished" podID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerID="e49e33380ecfd8725d2369eb3f53c5ced6b26d20d4126e443610909438b588d5" exitCode=0
Jul 09 09:48:01.175756 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:01.175638    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"
Jul 09 09:48:01.175756 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:01.175642    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" event={"ID":"2c3f2b62-e10b-42fe-99d8-2163cb678e39","Type":"ContainerDied","Data":"e49e33380ecfd8725d2369eb3f53c5ced6b26d20d4126e443610909438b588d5"}
Jul 09 09:48:01.175756 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:01.175732    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59" event={"ID":"2c3f2b62-e10b-42fe-99d8-2163cb678e39","Type":"ContainerDied","Data":"e924c3ce4b1dec6e5fbc17bdb3f444043d2cfab9ae75a5195e50643b561fd504"}
Jul 09 09:48:01.175908 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:01.175764    2549 scope.go:117] "RemoveContainer" containerID="e49e33380ecfd8725d2369eb3f53c5ced6b26d20d4126e443610909438b588d5"
Jul 09 09:48:01.184630 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:01.184614    2549 scope.go:117] "RemoveContainer" containerID="8133717aa948924d4878b70eaf58293af432699b86cb2eba06aaea5d9d531346"
Jul 09 09:48:01.192153 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:01.192137    2549 scope.go:117] "RemoveContainer" containerID="782b744435889c5f5586b97a9ad92285d7475f932bef4a9ab5e6bf689cf9b36d"
Jul 09 09:48:01.198804 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:01.198782    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"]
Jul 09 09:48:01.199670 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:01.199653    2549 scope.go:117] "RemoveContainer" containerID="9864b363a0234d558d0167183c14658228523db25981cb829f6e6a6e963025e7"
Jul 09 09:48:01.202717 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:01.202698    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-7fd7b677bb-dsk59"]
Jul 09 09:48:01.207802 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:01.207787    2549 scope.go:117] "RemoveContainer" containerID="e49e33380ecfd8725d2369eb3f53c5ced6b26d20d4126e443610909438b588d5"
Jul 09 09:48:01.208087 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:48:01.208059    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e49e33380ecfd8725d2369eb3f53c5ced6b26d20d4126e443610909438b588d5\": container with ID starting with e49e33380ecfd8725d2369eb3f53c5ced6b26d20d4126e443610909438b588d5 not found: ID does not exist" containerID="e49e33380ecfd8725d2369eb3f53c5ced6b26d20d4126e443610909438b588d5"
Jul 09 09:48:01.208141 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:01.208094    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e49e33380ecfd8725d2369eb3f53c5ced6b26d20d4126e443610909438b588d5"} err="failed to get container status \"e49e33380ecfd8725d2369eb3f53c5ced6b26d20d4126e443610909438b588d5\": rpc error: code = NotFound desc = could not find container \"e49e33380ecfd8725d2369eb3f53c5ced6b26d20d4126e443610909438b588d5\": container with ID starting with e49e33380ecfd8725d2369eb3f53c5ced6b26d20d4126e443610909438b588d5 not found: ID does not exist"
Jul 09 09:48:01.208141 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:01.208112    2549 scope.go:117] "RemoveContainer" containerID="8133717aa948924d4878b70eaf58293af432699b86cb2eba06aaea5d9d531346"
Jul 09 09:48:01.208310 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:48:01.208297    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8133717aa948924d4878b70eaf58293af432699b86cb2eba06aaea5d9d531346\": container with ID starting with 8133717aa948924d4878b70eaf58293af432699b86cb2eba06aaea5d9d531346 not found: ID does not exist" containerID="8133717aa948924d4878b70eaf58293af432699b86cb2eba06aaea5d9d531346"
Jul 09 09:48:01.208350 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:01.208314    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8133717aa948924d4878b70eaf58293af432699b86cb2eba06aaea5d9d531346"} err="failed to get container status \"8133717aa948924d4878b70eaf58293af432699b86cb2eba06aaea5d9d531346\": rpc error: code = NotFound desc = could not find container \"8133717aa948924d4878b70eaf58293af432699b86cb2eba06aaea5d9d531346\": container with ID starting with 8133717aa948924d4878b70eaf58293af432699b86cb2eba06aaea5d9d531346 not found: ID does not exist"
Jul 09 09:48:01.208350 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:01.208327    2549 scope.go:117] "RemoveContainer" containerID="782b744435889c5f5586b97a9ad92285d7475f932bef4a9ab5e6bf689cf9b36d"
Jul 09 09:48:01.208499 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:48:01.208484    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"782b744435889c5f5586b97a9ad92285d7475f932bef4a9ab5e6bf689cf9b36d\": container with ID starting with 782b744435889c5f5586b97a9ad92285d7475f932bef4a9ab5e6bf689cf9b36d not found: ID does not exist" containerID="782b744435889c5f5586b97a9ad92285d7475f932bef4a9ab5e6bf689cf9b36d"
Jul 09 09:48:01.208539 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:01.208503    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"782b744435889c5f5586b97a9ad92285d7475f932bef4a9ab5e6bf689cf9b36d"} err="failed to get container status \"782b744435889c5f5586b97a9ad92285d7475f932bef4a9ab5e6bf689cf9b36d\": rpc error: code = NotFound desc = could not find container \"782b744435889c5f5586b97a9ad92285d7475f932bef4a9ab5e6bf689cf9b36d\": container with ID starting with 782b744435889c5f5586b97a9ad92285d7475f932bef4a9ab5e6bf689cf9b36d not found: ID does not exist"
Jul 09 09:48:01.208539 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:01.208515    2549 scope.go:117] "RemoveContainer" containerID="9864b363a0234d558d0167183c14658228523db25981cb829f6e6a6e963025e7"
Jul 09 09:48:01.208751 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:48:01.208733    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9864b363a0234d558d0167183c14658228523db25981cb829f6e6a6e963025e7\": container with ID starting with 9864b363a0234d558d0167183c14658228523db25981cb829f6e6a6e963025e7 not found: ID does not exist" containerID="9864b363a0234d558d0167183c14658228523db25981cb829f6e6a6e963025e7"
Jul 09 09:48:01.208792 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:01.208758    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9864b363a0234d558d0167183c14658228523db25981cb829f6e6a6e963025e7"} err="failed to get container status \"9864b363a0234d558d0167183c14658228523db25981cb829f6e6a6e963025e7\": rpc error: code = NotFound desc = could not find container \"9864b363a0234d558d0167183c14658228523db25981cb829f6e6a6e963025e7\": container with ID starting with 9864b363a0234d558d0167183c14658228523db25981cb829f6e6a6e963025e7 not found: ID does not exist"
Jul 09 09:48:02.385822 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:02.385793    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" path="/var/lib/kubelet/pods/2c3f2b62-e10b-42fe-99d8-2163cb678e39/volumes"
Jul 09 09:48:04.168077 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:04.168046    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"
Jul 09 09:48:04.168722 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:04.168692    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Jul 09 09:48:04.169006 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:04.168979    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:48:14.169135 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:14.169087    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Jul 09 09:48:14.169575 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:14.169489    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:48:24.168637 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:24.168586    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Jul 09 09:48:24.169161 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:24.169138    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:48:34.168553 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:34.168505    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Jul 09 09:48:34.169130 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:34.168992    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:48:44.169157 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:44.169115    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Jul 09 09:48:44.169701 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:44.169675    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:48:54.168624 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:54.168523    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Jul 09 09:48:54.169560 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:48:54.169526    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:49:04.168705 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:04.168660    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Jul 09 09:49:04.169279 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:04.169212    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:49:07.380048 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:07.380016    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"
Jul 09 09:49:07.380533 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:07.380151    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"
Jul 09 09:49:15.973915 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:15.973881    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_message-dumper-predictor-6575bcfdfd-jxrdf_189b2206-4a46-4c59-bacb-320f1959a0a0/kserve-container/0.log"
Jul 09 09:49:16.148854 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.148823    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"]
Jul 09 09:49:16.149178 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.149145    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="agent" containerID="cri-o://2b4a8a02752c34d0d0c41324915853534048ba9c5fe7a8351b56c8d5f08c8635" gracePeriod=30
Jul 09 09:49:16.149344 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.149171    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="kube-rbac-proxy" containerID="cri-o://7fb26030e823d03a43012f5127b3803118eb141497656e90b31307cc1b7b71dc" gracePeriod=30
Jul 09 09:49:16.149344 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.149126    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="kserve-container" containerID="cri-o://cdbfb51e61f064a787dedb1f91305562e59f1deaf8663042d82edd8433d32ece" gracePeriod=30
Jul 09 09:49:16.220672 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.220646    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc"]
Jul 09 09:49:16.221134 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.221118    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="kserve-container"
Jul 09 09:49:16.221199 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.221136    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="kserve-container"
Jul 09 09:49:16.221199 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.221155    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="storage-initializer"
Jul 09 09:49:16.221199 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.221160    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="storage-initializer"
Jul 09 09:49:16.221199 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.221166    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="kube-rbac-proxy"
Jul 09 09:49:16.221199 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.221173    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="kube-rbac-proxy"
Jul 09 09:49:16.221199 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.221179    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="agent"
Jul 09 09:49:16.221199 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.221184    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="agent"
Jul 09 09:49:16.221412 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.221250    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="agent"
Jul 09 09:49:16.221412 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.221258    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="kube-rbac-proxy"
Jul 09 09:49:16.221412 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.221266    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c3f2b62-e10b-42fe-99d8-2163cb678e39" containerName="kserve-container"
Jul 09 09:49:16.224713 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.224671    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc"
Jul 09 09:49:16.227452 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.227433    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v1-predictor-serving-cert\""
Jul 09 09:49:16.227559 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.227462    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v1-kube-rbac-proxy-sar-config\""
Jul 09 09:49:16.235235 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.235214    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc"]
Jul 09 09:49:16.269935 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.269915    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf"]
Jul 09 09:49:16.270135 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.270111    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf" podUID="189b2206-4a46-4c59-bacb-320f1959a0a0" containerName="kserve-container" containerID="cri-o://06bb0228b0b72238cb69b49538bd4e403201b24952556442cbc06d637a90a6eb" gracePeriod=30
Jul 09 09:49:16.270202 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.270132    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf" podUID="189b2206-4a46-4c59-bacb-320f1959a0a0" containerName="kube-rbac-proxy" containerID="cri-o://734f2ffce2d55ddec6118bd8fa71973d9d74e2f39ea1cf8f852349196b6a509d" gracePeriod=30
Jul 09 09:49:16.344624 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.344603    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2ec26e09-e732-4cef-886c-971a44ca358d-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc\" (UID: \"2ec26e09-e732-4cef-886c-971a44ca358d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc"
Jul 09 09:49:16.344721 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.344639    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2ec26e09-e732-4cef-886c-971a44ca358d-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc\" (UID: \"2ec26e09-e732-4cef-886c-971a44ca358d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc"
Jul 09 09:49:16.344721 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.344668    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2ec26e09-e732-4cef-886c-971a44ca358d-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc\" (UID: \"2ec26e09-e732-4cef-886c-971a44ca358d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc"
Jul 09 09:49:16.344798 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.344761    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2ec26e09-e732-4cef-886c-971a44ca358d-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc\" (UID: \"2ec26e09-e732-4cef-886c-971a44ca358d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc"
Jul 09 09:49:16.344833 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.344811    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76hl2\" (UniqueName: \"kubernetes.io/projected/2ec26e09-e732-4cef-886c-971a44ca358d-kube-api-access-76hl2\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc\" (UID: \"2ec26e09-e732-4cef-886c-971a44ca358d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc"
Jul 09 09:49:16.446238 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.446210    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76hl2\" (UniqueName: \"kubernetes.io/projected/2ec26e09-e732-4cef-886c-971a44ca358d-kube-api-access-76hl2\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc\" (UID: \"2ec26e09-e732-4cef-886c-971a44ca358d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc"
Jul 09 09:49:16.446354 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.446277    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2ec26e09-e732-4cef-886c-971a44ca358d-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc\" (UID: \"2ec26e09-e732-4cef-886c-971a44ca358d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc"
Jul 09 09:49:16.446354 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.446305    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2ec26e09-e732-4cef-886c-971a44ca358d-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc\" (UID: \"2ec26e09-e732-4cef-886c-971a44ca358d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc"
Jul 09 09:49:16.446354 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.446338    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2ec26e09-e732-4cef-886c-971a44ca358d-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc\" (UID: \"2ec26e09-e732-4cef-886c-971a44ca358d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc"
Jul 09 09:49:16.446533 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.446397    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2ec26e09-e732-4cef-886c-971a44ca358d-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc\" (UID: \"2ec26e09-e732-4cef-886c-971a44ca358d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc"
Jul 09 09:49:16.446794 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.446760    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2ec26e09-e732-4cef-886c-971a44ca358d-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc\" (UID: \"2ec26e09-e732-4cef-886c-971a44ca358d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc"
Jul 09 09:49:16.447161 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.447140    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2ec26e09-e732-4cef-886c-971a44ca358d-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc\" (UID: \"2ec26e09-e732-4cef-886c-971a44ca358d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc"
Jul 09 09:49:16.447234 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.447140    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2ec26e09-e732-4cef-886c-971a44ca358d-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc\" (UID: \"2ec26e09-e732-4cef-886c-971a44ca358d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc"
Jul 09 09:49:16.448622 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.448603    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2ec26e09-e732-4cef-886c-971a44ca358d-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc\" (UID: \"2ec26e09-e732-4cef-886c-971a44ca358d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc"
Jul 09 09:49:16.456437 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.456397    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76hl2\" (UniqueName: \"kubernetes.io/projected/2ec26e09-e732-4cef-886c-971a44ca358d-kube-api-access-76hl2\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc\" (UID: \"2ec26e09-e732-4cef-886c-971a44ca358d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc"
Jul 09 09:49:16.504078 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.504057    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf"
Jul 09 09:49:16.531013 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.530981    2549 generic.go:356] "Generic (PLEG): container finished" podID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerID="7fb26030e823d03a43012f5127b3803118eb141497656e90b31307cc1b7b71dc" exitCode=2
Jul 09 09:49:16.531142 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.531049    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" event={"ID":"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d","Type":"ContainerDied","Data":"7fb26030e823d03a43012f5127b3803118eb141497656e90b31307cc1b7b71dc"}
Jul 09 09:49:16.532583 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.532562    2549 generic.go:356] "Generic (PLEG): container finished" podID="189b2206-4a46-4c59-bacb-320f1959a0a0" containerID="734f2ffce2d55ddec6118bd8fa71973d9d74e2f39ea1cf8f852349196b6a509d" exitCode=2
Jul 09 09:49:16.532682 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.532586    2549 generic.go:356] "Generic (PLEG): container finished" podID="189b2206-4a46-4c59-bacb-320f1959a0a0" containerID="06bb0228b0b72238cb69b49538bd4e403201b24952556442cbc06d637a90a6eb" exitCode=2
Jul 09 09:49:16.532682 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.532602    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf"
Jul 09 09:49:16.532682 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.532632    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf" event={"ID":"189b2206-4a46-4c59-bacb-320f1959a0a0","Type":"ContainerDied","Data":"734f2ffce2d55ddec6118bd8fa71973d9d74e2f39ea1cf8f852349196b6a509d"}
Jul 09 09:49:16.532682 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.532679    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf" event={"ID":"189b2206-4a46-4c59-bacb-320f1959a0a0","Type":"ContainerDied","Data":"06bb0228b0b72238cb69b49538bd4e403201b24952556442cbc06d637a90a6eb"}
Jul 09 09:49:16.532826 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.532698    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf" event={"ID":"189b2206-4a46-4c59-bacb-320f1959a0a0","Type":"ContainerDied","Data":"3ea1bf1b9e93a69e56607fcf78acbfc4b39a40d3f2868d37d79cef0db45f7cff"}
Jul 09 09:49:16.532826 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.532724    2549 scope.go:117] "RemoveContainer" containerID="734f2ffce2d55ddec6118bd8fa71973d9d74e2f39ea1cf8f852349196b6a509d"
Jul 09 09:49:16.535964 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.535947    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc"
Jul 09 09:49:16.541165 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.541141    2549 scope.go:117] "RemoveContainer" containerID="06bb0228b0b72238cb69b49538bd4e403201b24952556442cbc06d637a90a6eb"
Jul 09 09:49:16.549605 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.549584    2549 scope.go:117] "RemoveContainer" containerID="734f2ffce2d55ddec6118bd8fa71973d9d74e2f39ea1cf8f852349196b6a509d"
Jul 09 09:49:16.549955 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:49:16.549845    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"734f2ffce2d55ddec6118bd8fa71973d9d74e2f39ea1cf8f852349196b6a509d\": container with ID starting with 734f2ffce2d55ddec6118bd8fa71973d9d74e2f39ea1cf8f852349196b6a509d not found: ID does not exist" containerID="734f2ffce2d55ddec6118bd8fa71973d9d74e2f39ea1cf8f852349196b6a509d"
Jul 09 09:49:16.549955 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.549885    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"734f2ffce2d55ddec6118bd8fa71973d9d74e2f39ea1cf8f852349196b6a509d"} err="failed to get container status \"734f2ffce2d55ddec6118bd8fa71973d9d74e2f39ea1cf8f852349196b6a509d\": rpc error: code = NotFound desc = could not find container \"734f2ffce2d55ddec6118bd8fa71973d9d74e2f39ea1cf8f852349196b6a509d\": container with ID starting with 734f2ffce2d55ddec6118bd8fa71973d9d74e2f39ea1cf8f852349196b6a509d not found: ID does not exist"
Jul 09 09:49:16.549955 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.549904    2549 scope.go:117] "RemoveContainer" containerID="06bb0228b0b72238cb69b49538bd4e403201b24952556442cbc06d637a90a6eb"
Jul 09 09:49:16.550158 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:49:16.550137    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06bb0228b0b72238cb69b49538bd4e403201b24952556442cbc06d637a90a6eb\": container with ID starting with 06bb0228b0b72238cb69b49538bd4e403201b24952556442cbc06d637a90a6eb not found: ID does not exist" containerID="06bb0228b0b72238cb69b49538bd4e403201b24952556442cbc06d637a90a6eb"
Jul 09 09:49:16.550197 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.550171    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06bb0228b0b72238cb69b49538bd4e403201b24952556442cbc06d637a90a6eb"} err="failed to get container status \"06bb0228b0b72238cb69b49538bd4e403201b24952556442cbc06d637a90a6eb\": rpc error: code = NotFound desc = could not find container \"06bb0228b0b72238cb69b49538bd4e403201b24952556442cbc06d637a90a6eb\": container with ID starting with 06bb0228b0b72238cb69b49538bd4e403201b24952556442cbc06d637a90a6eb not found: ID does not exist"
Jul 09 09:49:16.550197 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.550193    2549 scope.go:117] "RemoveContainer" containerID="734f2ffce2d55ddec6118bd8fa71973d9d74e2f39ea1cf8f852349196b6a509d"
Jul 09 09:49:16.550434 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.550417    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"734f2ffce2d55ddec6118bd8fa71973d9d74e2f39ea1cf8f852349196b6a509d"} err="failed to get container status \"734f2ffce2d55ddec6118bd8fa71973d9d74e2f39ea1cf8f852349196b6a509d\": rpc error: code = NotFound desc = could not find container \"734f2ffce2d55ddec6118bd8fa71973d9d74e2f39ea1cf8f852349196b6a509d\": container with ID starting with 734f2ffce2d55ddec6118bd8fa71973d9d74e2f39ea1cf8f852349196b6a509d not found: ID does not exist"
Jul 09 09:49:16.550478 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.550434    2549 scope.go:117] "RemoveContainer" containerID="06bb0228b0b72238cb69b49538bd4e403201b24952556442cbc06d637a90a6eb"
Jul 09 09:49:16.550722 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.550702    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06bb0228b0b72238cb69b49538bd4e403201b24952556442cbc06d637a90a6eb"} err="failed to get container status \"06bb0228b0b72238cb69b49538bd4e403201b24952556442cbc06d637a90a6eb\": rpc error: code = NotFound desc = could not find container \"06bb0228b0b72238cb69b49538bd4e403201b24952556442cbc06d637a90a6eb\": container with ID starting with 06bb0228b0b72238cb69b49538bd4e403201b24952556442cbc06d637a90a6eb not found: ID does not exist"
Jul 09 09:49:16.648256 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.648229    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ns8w\" (UniqueName: \"kubernetes.io/projected/189b2206-4a46-4c59-bacb-320f1959a0a0-kube-api-access-2ns8w\") pod \"189b2206-4a46-4c59-bacb-320f1959a0a0\" (UID: \"189b2206-4a46-4c59-bacb-320f1959a0a0\") "
Jul 09 09:49:16.648419 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.648406    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/189b2206-4a46-4c59-bacb-320f1959a0a0-proxy-tls\") pod \"189b2206-4a46-4c59-bacb-320f1959a0a0\" (UID: \"189b2206-4a46-4c59-bacb-320f1959a0a0\") "
Jul 09 09:49:16.648464 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.648431    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/189b2206-4a46-4c59-bacb-320f1959a0a0-kube-rbac-proxy-sar-config\") pod \"189b2206-4a46-4c59-bacb-320f1959a0a0\" (UID: \"189b2206-4a46-4c59-bacb-320f1959a0a0\") "
Jul 09 09:49:16.648831 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.648808    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/189b2206-4a46-4c59-bacb-320f1959a0a0-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "189b2206-4a46-4c59-bacb-320f1959a0a0" (UID: "189b2206-4a46-4c59-bacb-320f1959a0a0"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:49:16.650205 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.650178    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/189b2206-4a46-4c59-bacb-320f1959a0a0-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "189b2206-4a46-4c59-bacb-320f1959a0a0" (UID: "189b2206-4a46-4c59-bacb-320f1959a0a0"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:49:16.650317 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.650225    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/189b2206-4a46-4c59-bacb-320f1959a0a0-kube-api-access-2ns8w" (OuterVolumeSpecName: "kube-api-access-2ns8w") pod "189b2206-4a46-4c59-bacb-320f1959a0a0" (UID: "189b2206-4a46-4c59-bacb-320f1959a0a0"). InnerVolumeSpecName "kube-api-access-2ns8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:49:16.669196 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.669172    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc"]
Jul 09 09:49:16.671021 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:49:16.671000    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ec26e09_e732_4cef_886c_971a44ca358d.slice/crio-5e45e05d59df036d017e3a9c4a1d59edacdfd4551dcf89ada183fba87e713f91 WatchSource:0}: Error finding container 5e45e05d59df036d017e3a9c4a1d59edacdfd4551dcf89ada183fba87e713f91: Status 404 returned error can't find the container with id 5e45e05d59df036d017e3a9c4a1d59edacdfd4551dcf89ada183fba87e713f91
Jul 09 09:49:16.750063 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.750013    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ns8w\" (UniqueName: \"kubernetes.io/projected/189b2206-4a46-4c59-bacb-320f1959a0a0-kube-api-access-2ns8w\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:49:16.750063 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.750034    2549 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/189b2206-4a46-4c59-bacb-320f1959a0a0-proxy-tls\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:49:16.750063 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.750043    2549 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/189b2206-4a46-4c59-bacb-320f1959a0a0-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:49:16.857845 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.857816    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf"]
Jul 09 09:49:16.863343 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:16.863323    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-jxrdf"]
Jul 09 09:49:17.380046 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:17.380006    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Jul 09 09:49:17.380459 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:17.380359    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:49:17.539126 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:17.539097    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc" event={"ID":"2ec26e09-e732-4cef-886c-971a44ca358d","Type":"ContainerStarted","Data":"6989fcbcd211ca98951f5104ee557bbb428d1a101a22e68d0277f4d2875bfc07"}
Jul 09 09:49:17.539309 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:17.539156    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc" event={"ID":"2ec26e09-e732-4cef-886c-971a44ca358d","Type":"ContainerStarted","Data":"5e45e05d59df036d017e3a9c4a1d59edacdfd4551dcf89ada183fba87e713f91"}
Jul 09 09:49:18.385517 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:18.385481    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="189b2206-4a46-4c59-bacb-320f1959a0a0" path="/var/lib/kubelet/pods/189b2206-4a46-4c59-bacb-320f1959a0a0/volumes"
Jul 09 09:49:19.164516 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:19.164479    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.36:8643/healthz\": dial tcp 10.134.0.36:8643: connect: connection refused"
Jul 09 09:49:20.555750 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:20.555675    2549 generic.go:356] "Generic (PLEG): container finished" podID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerID="cdbfb51e61f064a787dedb1f91305562e59f1deaf8663042d82edd8433d32ece" exitCode=0
Jul 09 09:49:20.556076 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:20.555748    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" event={"ID":"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d","Type":"ContainerDied","Data":"cdbfb51e61f064a787dedb1f91305562e59f1deaf8663042d82edd8433d32ece"}
Jul 09 09:49:24.164884 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:24.164816    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.36:8643/healthz\": dial tcp 10.134.0.36:8643: connect: connection refused"
Jul 09 09:49:27.380269 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:27.380224    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Jul 09 09:49:27.380706 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:27.380593    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:49:29.164796 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:29.164759    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.36:8643/healthz\": dial tcp 10.134.0.36:8643: connect: connection refused"
Jul 09 09:49:29.165227 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:29.164937    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"
Jul 09 09:49:34.164623 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:34.164578    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.36:8643/healthz\": dial tcp 10.134.0.36:8643: connect: connection refused"
Jul 09 09:49:37.380439 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:37.380393    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Jul 09 09:49:37.380958 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:37.380555    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"
Jul 09 09:49:37.380958 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:37.380667    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 09:49:37.380958 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:37.380799    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"
Jul 09 09:49:37.632400 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:37.632319    2549 generic.go:356] "Generic (PLEG): container finished" podID="2ec26e09-e732-4cef-886c-971a44ca358d" containerID="6989fcbcd211ca98951f5104ee557bbb428d1a101a22e68d0277f4d2875bfc07" exitCode=0
Jul 09 09:49:37.632400 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:37.632390    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc" event={"ID":"2ec26e09-e732-4cef-886c-971a44ca358d","Type":"ContainerDied","Data":"6989fcbcd211ca98951f5104ee557bbb428d1a101a22e68d0277f4d2875bfc07"}
Jul 09 09:49:39.164937 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:39.164895    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.36:8643/healthz\": dial tcp 10.134.0.36:8643: connect: connection refused"
Jul 09 09:49:44.165058 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:44.165017    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.36:8643/healthz\": dial tcp 10.134.0.36:8643: connect: connection refused"
Jul 09 09:49:46.305295 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.305272    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"
Jul 09 09:49:46.395292 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.395259    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-kserve-provision-location\") pod \"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d\" (UID: \"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d\") "
Jul 09 09:49:46.395508 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.395316    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-proxy-tls\") pod \"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d\" (UID: \"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d\") "
Jul 09 09:49:46.395508 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.395367    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-cabundle-cert\") pod \"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d\" (UID: \"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d\") "
Jul 09 09:49:46.395508 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.395472    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rk5wr\" (UniqueName: \"kubernetes.io/projected/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-kube-api-access-rk5wr\") pod \"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d\" (UID: \"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d\") "
Jul 09 09:49:46.395508 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.395496    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-kube-rbac-proxy-sar-config\") pod \"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d\" (UID: \"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d\") "
Jul 09 09:49:46.395784 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.395608    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" (UID: "778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:49:46.395784 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.395771    2549 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-kserve-provision-location\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:49:46.395899 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.395823    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" (UID: "778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:49:46.395989 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.395967    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" (UID: "778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:49:46.397748 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.397702    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" (UID: "778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:49:46.397897 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.397846    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-kube-api-access-rk5wr" (OuterVolumeSpecName: "kube-api-access-rk5wr") pod "778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" (UID: "778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d"). InnerVolumeSpecName "kube-api-access-rk5wr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:49:46.497051 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.497025    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rk5wr\" (UniqueName: \"kubernetes.io/projected/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-kube-api-access-rk5wr\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:49:46.497051 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.497048    2549 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:49:46.497051 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.497057    2549 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-proxy-tls\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:49:46.497248 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.497066    2549 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d-cabundle-cert\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:49:46.689221 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.689135    2549 generic.go:356] "Generic (PLEG): container finished" podID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerID="2b4a8a02752c34d0d0c41324915853534048ba9c5fe7a8351b56c8d5f08c8635" exitCode=137
Jul 09 09:49:46.689221 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.689190    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"
Jul 09 09:49:46.689441 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.689219    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" event={"ID":"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d","Type":"ContainerDied","Data":"2b4a8a02752c34d0d0c41324915853534048ba9c5fe7a8351b56c8d5f08c8635"}
Jul 09 09:49:46.689441 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.689300    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4" event={"ID":"778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d","Type":"ContainerDied","Data":"b7ad596e75078a7d09f8e2e13e0f906a3f6465aed4539dd2e73803407ee93a77"}
Jul 09 09:49:46.689441 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.689347    2549 scope.go:117] "RemoveContainer" containerID="2b4a8a02752c34d0d0c41324915853534048ba9c5fe7a8351b56c8d5f08c8635"
Jul 09 09:49:46.699539 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.699519    2549 scope.go:117] "RemoveContainer" containerID="7fb26030e823d03a43012f5127b3803118eb141497656e90b31307cc1b7b71dc"
Jul 09 09:49:46.709556 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.709539    2549 scope.go:117] "RemoveContainer" containerID="cdbfb51e61f064a787dedb1f91305562e59f1deaf8663042d82edd8433d32ece"
Jul 09 09:49:46.716475 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.716439    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"]
Jul 09 09:49:46.719019 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.718994    2549 scope.go:117] "RemoveContainer" containerID="3a21b1998ff995c8e3f78d59af4f370fd8b14b2314efe093e48ffb4aa07d31e4"
Jul 09 09:49:46.720460 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.720441    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-7997fc58c6-r7cl4"]
Jul 09 09:49:46.727852 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.727835    2549 scope.go:117] "RemoveContainer" containerID="2b4a8a02752c34d0d0c41324915853534048ba9c5fe7a8351b56c8d5f08c8635"
Jul 09 09:49:46.728141 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:49:46.728113    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b4a8a02752c34d0d0c41324915853534048ba9c5fe7a8351b56c8d5f08c8635\": container with ID starting with 2b4a8a02752c34d0d0c41324915853534048ba9c5fe7a8351b56c8d5f08c8635 not found: ID does not exist" containerID="2b4a8a02752c34d0d0c41324915853534048ba9c5fe7a8351b56c8d5f08c8635"
Jul 09 09:49:46.728242 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.728145    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b4a8a02752c34d0d0c41324915853534048ba9c5fe7a8351b56c8d5f08c8635"} err="failed to get container status \"2b4a8a02752c34d0d0c41324915853534048ba9c5fe7a8351b56c8d5f08c8635\": rpc error: code = NotFound desc = could not find container \"2b4a8a02752c34d0d0c41324915853534048ba9c5fe7a8351b56c8d5f08c8635\": container with ID starting with 2b4a8a02752c34d0d0c41324915853534048ba9c5fe7a8351b56c8d5f08c8635 not found: ID does not exist"
Jul 09 09:49:46.728242 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.728166    2549 scope.go:117] "RemoveContainer" containerID="7fb26030e823d03a43012f5127b3803118eb141497656e90b31307cc1b7b71dc"
Jul 09 09:49:46.728527 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:49:46.728508    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fb26030e823d03a43012f5127b3803118eb141497656e90b31307cc1b7b71dc\": container with ID starting with 7fb26030e823d03a43012f5127b3803118eb141497656e90b31307cc1b7b71dc not found: ID does not exist" containerID="7fb26030e823d03a43012f5127b3803118eb141497656e90b31307cc1b7b71dc"
Jul 09 09:49:46.728589 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.728531    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fb26030e823d03a43012f5127b3803118eb141497656e90b31307cc1b7b71dc"} err="failed to get container status \"7fb26030e823d03a43012f5127b3803118eb141497656e90b31307cc1b7b71dc\": rpc error: code = NotFound desc = could not find container \"7fb26030e823d03a43012f5127b3803118eb141497656e90b31307cc1b7b71dc\": container with ID starting with 7fb26030e823d03a43012f5127b3803118eb141497656e90b31307cc1b7b71dc not found: ID does not exist"
Jul 09 09:49:46.728589 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.728548    2549 scope.go:117] "RemoveContainer" containerID="cdbfb51e61f064a787dedb1f91305562e59f1deaf8663042d82edd8433d32ece"
Jul 09 09:49:46.729224 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:49:46.728849    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdbfb51e61f064a787dedb1f91305562e59f1deaf8663042d82edd8433d32ece\": container with ID starting with cdbfb51e61f064a787dedb1f91305562e59f1deaf8663042d82edd8433d32ece not found: ID does not exist" containerID="cdbfb51e61f064a787dedb1f91305562e59f1deaf8663042d82edd8433d32ece"
Jul 09 09:49:46.729224 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.728890    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdbfb51e61f064a787dedb1f91305562e59f1deaf8663042d82edd8433d32ece"} err="failed to get container status \"cdbfb51e61f064a787dedb1f91305562e59f1deaf8663042d82edd8433d32ece\": rpc error: code = NotFound desc = could not find container \"cdbfb51e61f064a787dedb1f91305562e59f1deaf8663042d82edd8433d32ece\": container with ID starting with cdbfb51e61f064a787dedb1f91305562e59f1deaf8663042d82edd8433d32ece not found: ID does not exist"
Jul 09 09:49:46.729224 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.728911    2549 scope.go:117] "RemoveContainer" containerID="3a21b1998ff995c8e3f78d59af4f370fd8b14b2314efe093e48ffb4aa07d31e4"
Jul 09 09:49:46.729224 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:49:46.729143    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a21b1998ff995c8e3f78d59af4f370fd8b14b2314efe093e48ffb4aa07d31e4\": container with ID starting with 3a21b1998ff995c8e3f78d59af4f370fd8b14b2314efe093e48ffb4aa07d31e4 not found: ID does not exist" containerID="3a21b1998ff995c8e3f78d59af4f370fd8b14b2314efe093e48ffb4aa07d31e4"
Jul 09 09:49:46.729224 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:46.729163    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a21b1998ff995c8e3f78d59af4f370fd8b14b2314efe093e48ffb4aa07d31e4"} err="failed to get container status \"3a21b1998ff995c8e3f78d59af4f370fd8b14b2314efe093e48ffb4aa07d31e4\": rpc error: code = NotFound desc = could not find container \"3a21b1998ff995c8e3f78d59af4f370fd8b14b2314efe093e48ffb4aa07d31e4\": container with ID starting with 3a21b1998ff995c8e3f78d59af4f370fd8b14b2314efe093e48ffb4aa07d31e4 not found: ID does not exist"
Jul 09 09:49:48.387484 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:49:48.387447    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" path="/var/lib/kubelet/pods/778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d/volumes"
Jul 09 09:51:49.337197 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:51:49.337162    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc" event={"ID":"2ec26e09-e732-4cef-886c-971a44ca358d","Type":"ContainerStarted","Data":"e369fb71f16cc81aca0d6d972785ea638b7001022c90c184ebf1addbf7d7f35c"}
Jul 09 09:51:49.337690 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:51:49.337227    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc"
Jul 09 09:51:49.337690 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:51:49.337247    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc" event={"ID":"2ec26e09-e732-4cef-886c-971a44ca358d","Type":"ContainerStarted","Data":"27e2bc2044720a6a4b1bbe0509ac98dcf40c87663e3fa297f8ce01f6facd2464"}
Jul 09 09:51:49.337690 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:51:49.337394    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc"
Jul 09 09:51:49.338772 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:51:49.338744    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc" podUID="2ec26e09-e732-4cef-886c-971a44ca358d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Jul 09 09:51:49.370157 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:51:49.370105    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc" podStartSLOduration=2.764299856 podStartE2EDuration="2m33.370090688s" totalImagesPullingTime="2m10.590434683s" totalInitContainerRuntime="20.015356149s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:49:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:51:49.368912967 +0000 UTC m=+1067.540071522" watchObservedRunningTime="2026-07-09 09:51:49.370090688 +0000 UTC m=+1067.541249246"
Jul 09 09:51:50.342248 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:51:50.342205    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc" podUID="2ec26e09-e732-4cef-886c-971a44ca358d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Jul 09 09:51:55.347916 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:51:55.347814    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc"
Jul 09 09:51:55.348562 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:51:55.348527    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc" podUID="2ec26e09-e732-4cef-886c-971a44ca358d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Jul 09 09:52:05.349813 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:05.349786    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc"
Jul 09 09:52:06.414575 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:06.414538    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc"]
Jul 09 09:52:06.415091 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:06.414775    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc" podUID="2ec26e09-e732-4cef-886c-971a44ca358d" containerName="kserve-container" containerID="cri-o://27e2bc2044720a6a4b1bbe0509ac98dcf40c87663e3fa297f8ce01f6facd2464" gracePeriod=30
Jul 09 09:52:06.415091 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:06.414804    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc" podUID="2ec26e09-e732-4cef-886c-971a44ca358d" containerName="kube-rbac-proxy" containerID="cri-o://e369fb71f16cc81aca0d6d972785ea638b7001022c90c184ebf1addbf7d7f35c" gracePeriod=30
Jul 09 09:52:07.421173 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:07.421139    2549 generic.go:356] "Generic (PLEG): container finished" podID="2ec26e09-e732-4cef-886c-971a44ca358d" containerID="e369fb71f16cc81aca0d6d972785ea638b7001022c90c184ebf1addbf7d7f35c" exitCode=2
Jul 09 09:52:07.421554 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:07.421215    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc" event={"ID":"2ec26e09-e732-4cef-886c-971a44ca358d","Type":"ContainerDied","Data":"e369fb71f16cc81aca0d6d972785ea638b7001022c90c184ebf1addbf7d7f35c"}
Jul 09 09:52:10.343350 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:10.343319    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc" podUID="2ec26e09-e732-4cef-886c-971a44ca358d" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.37:8643/healthz\": dial tcp 10.134.0.37:8643: connect: connection refused"
Jul 09 09:52:10.436760 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:10.436728    2549 generic.go:356] "Generic (PLEG): container finished" podID="2ec26e09-e732-4cef-886c-971a44ca358d" containerID="27e2bc2044720a6a4b1bbe0509ac98dcf40c87663e3fa297f8ce01f6facd2464" exitCode=0
Jul 09 09:52:10.436934 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:10.436791    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc" event={"ID":"2ec26e09-e732-4cef-886c-971a44ca358d","Type":"ContainerDied","Data":"27e2bc2044720a6a4b1bbe0509ac98dcf40c87663e3fa297f8ce01f6facd2464"}
Jul 09 09:52:10.491944 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:10.491922    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc"
Jul 09 09:52:10.589110 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:10.589047    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2ec26e09-e732-4cef-886c-971a44ca358d-cabundle-cert\") pod \"2ec26e09-e732-4cef-886c-971a44ca358d\" (UID: \"2ec26e09-e732-4cef-886c-971a44ca358d\") "
Jul 09 09:52:10.589219 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:10.589123    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2ec26e09-e732-4cef-886c-971a44ca358d-proxy-tls\") pod \"2ec26e09-e732-4cef-886c-971a44ca358d\" (UID: \"2ec26e09-e732-4cef-886c-971a44ca358d\") "
Jul 09 09:52:10.589219 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:10.589152    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2ec26e09-e732-4cef-886c-971a44ca358d-kserve-provision-location\") pod \"2ec26e09-e732-4cef-886c-971a44ca358d\" (UID: \"2ec26e09-e732-4cef-886c-971a44ca358d\") "
Jul 09 09:52:10.589219 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:10.589183    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2ec26e09-e732-4cef-886c-971a44ca358d-kube-rbac-proxy-sar-config\") pod \"2ec26e09-e732-4cef-886c-971a44ca358d\" (UID: \"2ec26e09-e732-4cef-886c-971a44ca358d\") "
Jul 09 09:52:10.589347 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:10.589226    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76hl2\" (UniqueName: \"kubernetes.io/projected/2ec26e09-e732-4cef-886c-971a44ca358d-kube-api-access-76hl2\") pod \"2ec26e09-e732-4cef-886c-971a44ca358d\" (UID: \"2ec26e09-e732-4cef-886c-971a44ca358d\") "
Jul 09 09:52:10.589453 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:10.589422    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ec26e09-e732-4cef-886c-971a44ca358d-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "2ec26e09-e732-4cef-886c-971a44ca358d" (UID: "2ec26e09-e732-4cef-886c-971a44ca358d"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:52:10.589597 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:10.589573    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ec26e09-e732-4cef-886c-971a44ca358d-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "2ec26e09-e732-4cef-886c-971a44ca358d" (UID: "2ec26e09-e732-4cef-886c-971a44ca358d"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:52:10.591227 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:10.591206    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ec26e09-e732-4cef-886c-971a44ca358d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "2ec26e09-e732-4cef-886c-971a44ca358d" (UID: "2ec26e09-e732-4cef-886c-971a44ca358d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:52:10.591356 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:10.591331    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ec26e09-e732-4cef-886c-971a44ca358d-kube-api-access-76hl2" (OuterVolumeSpecName: "kube-api-access-76hl2") pod "2ec26e09-e732-4cef-886c-971a44ca358d" (UID: "2ec26e09-e732-4cef-886c-971a44ca358d"). InnerVolumeSpecName "kube-api-access-76hl2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:52:10.593858 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:10.593838    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ec26e09-e732-4cef-886c-971a44ca358d-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "2ec26e09-e732-4cef-886c-971a44ca358d" (UID: "2ec26e09-e732-4cef-886c-971a44ca358d"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:52:10.690150 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:10.690124    2549 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2ec26e09-e732-4cef-886c-971a44ca358d-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:52:10.690150 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:10.690151    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76hl2\" (UniqueName: \"kubernetes.io/projected/2ec26e09-e732-4cef-886c-971a44ca358d-kube-api-access-76hl2\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:52:10.690285 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:10.690164    2549 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2ec26e09-e732-4cef-886c-971a44ca358d-cabundle-cert\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:52:10.690285 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:10.690177    2549 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2ec26e09-e732-4cef-886c-971a44ca358d-proxy-tls\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:52:10.690285 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:10.690188    2549 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2ec26e09-e732-4cef-886c-971a44ca358d-kserve-provision-location\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:52:11.443162 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.443133    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc" event={"ID":"2ec26e09-e732-4cef-886c-971a44ca358d","Type":"ContainerDied","Data":"5e45e05d59df036d017e3a9c4a1d59edacdfd4551dcf89ada183fba87e713f91"}
Jul 09 09:52:11.443679 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.443180    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc"
Jul 09 09:52:11.443679 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.443205    2549 scope.go:117] "RemoveContainer" containerID="e369fb71f16cc81aca0d6d972785ea638b7001022c90c184ebf1addbf7d7f35c"
Jul 09 09:52:11.451574 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.451557    2549 scope.go:117] "RemoveContainer" containerID="27e2bc2044720a6a4b1bbe0509ac98dcf40c87663e3fa297f8ce01f6facd2464"
Jul 09 09:52:11.458614 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.458599    2549 scope.go:117] "RemoveContainer" containerID="6989fcbcd211ca98951f5104ee557bbb428d1a101a22e68d0277f4d2875bfc07"
Jul 09 09:52:11.466002 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.465984    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc"]
Jul 09 09:52:11.471700 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.471679    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-zqkhc"]
Jul 09 09:52:11.771799 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.771767    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"]
Jul 09 09:52:11.772123 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.772111    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="189b2206-4a46-4c59-bacb-320f1959a0a0" containerName="kube-rbac-proxy"
Jul 09 09:52:11.772175 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.772124    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="189b2206-4a46-4c59-bacb-320f1959a0a0" containerName="kube-rbac-proxy"
Jul 09 09:52:11.772175 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.772136    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="agent"
Jul 09 09:52:11.772175 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.772142    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="agent"
Jul 09 09:52:11.772175 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.772150    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ec26e09-e732-4cef-886c-971a44ca358d" containerName="kserve-container"
Jul 09 09:52:11.772175 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.772155    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ec26e09-e732-4cef-886c-971a44ca358d" containerName="kserve-container"
Jul 09 09:52:11.772175 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.772163    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ec26e09-e732-4cef-886c-971a44ca358d" containerName="storage-initializer"
Jul 09 09:52:11.772175 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.772168    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ec26e09-e732-4cef-886c-971a44ca358d" containerName="storage-initializer"
Jul 09 09:52:11.772175 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.772178    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="kube-rbac-proxy"
Jul 09 09:52:11.772455 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.772183    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="kube-rbac-proxy"
Jul 09 09:52:11.772455 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.772189    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="189b2206-4a46-4c59-bacb-320f1959a0a0" containerName="kserve-container"
Jul 09 09:52:11.772455 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.772195    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="189b2206-4a46-4c59-bacb-320f1959a0a0" containerName="kserve-container"
Jul 09 09:52:11.772455 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.772210    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="kserve-container"
Jul 09 09:52:11.772455 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.772217    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="kserve-container"
Jul 09 09:52:11.772455 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.772223    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="storage-initializer"
Jul 09 09:52:11.772455 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.772228    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="storage-initializer"
Jul 09 09:52:11.772455 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.772234    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ec26e09-e732-4cef-886c-971a44ca358d" containerName="kube-rbac-proxy"
Jul 09 09:52:11.772455 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.772239    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ec26e09-e732-4cef-886c-971a44ca358d" containerName="kube-rbac-proxy"
Jul 09 09:52:11.772455 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.772284    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="agent"
Jul 09 09:52:11.772455 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.772292    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="189b2206-4a46-4c59-bacb-320f1959a0a0" containerName="kube-rbac-proxy"
Jul 09 09:52:11.772455 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.772300    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="kserve-container"
Jul 09 09:52:11.772455 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.772305    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ec26e09-e732-4cef-886c-971a44ca358d" containerName="kserve-container"
Jul 09 09:52:11.772455 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.772312    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ec26e09-e732-4cef-886c-971a44ca358d" containerName="kube-rbac-proxy"
Jul 09 09:52:11.772455 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.772318    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="189b2206-4a46-4c59-bacb-320f1959a0a0" containerName="kserve-container"
Jul 09 09:52:11.772455 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.772324    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="778f5ed3-0ef1-4f6a-81cd-4b22c84bbd4d" containerName="kube-rbac-proxy"
Jul 09 09:52:11.805599 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.805570    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"]
Jul 09 09:52:11.805739 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.805698    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"
Jul 09 09:52:11.808876 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.808833    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-predictor-serving-cert\""
Jul 09 09:52:11.809010 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.808953    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-kube-rbac-proxy-sar-config\""
Jul 09 09:52:11.809184 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.809167    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 09 09:52:11.809277 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.809219    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 09:52:11.809277 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.809238    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-2mhmw\""
Jul 09 09:52:11.809390 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.809300    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 09 09:52:11.809390 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.809340    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 09:52:11.899064 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.899038    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/92d0cf50-e00a-4f3c-9d62-51aba1026713-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-zn8lw\" (UID: \"92d0cf50-e00a-4f3c-9d62-51aba1026713\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"
Jul 09 09:52:11.899540 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.899089    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/92d0cf50-e00a-4f3c-9d62-51aba1026713-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-zn8lw\" (UID: \"92d0cf50-e00a-4f3c-9d62-51aba1026713\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"
Jul 09 09:52:11.899651 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.899571    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/92d0cf50-e00a-4f3c-9d62-51aba1026713-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-zn8lw\" (UID: \"92d0cf50-e00a-4f3c-9d62-51aba1026713\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"
Jul 09 09:52:11.899651 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.899623    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/92d0cf50-e00a-4f3c-9d62-51aba1026713-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-zn8lw\" (UID: \"92d0cf50-e00a-4f3c-9d62-51aba1026713\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"
Jul 09 09:52:11.899787 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:11.899760    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr5n5\" (UniqueName: \"kubernetes.io/projected/92d0cf50-e00a-4f3c-9d62-51aba1026713-kube-api-access-dr5n5\") pod \"isvc-autogluon-v2-predictor-67b849d456-zn8lw\" (UID: \"92d0cf50-e00a-4f3c-9d62-51aba1026713\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"
Jul 09 09:52:12.000798 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:12.000767    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/92d0cf50-e00a-4f3c-9d62-51aba1026713-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-zn8lw\" (UID: \"92d0cf50-e00a-4f3c-9d62-51aba1026713\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"
Jul 09 09:52:12.000798 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:12.000797    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/92d0cf50-e00a-4f3c-9d62-51aba1026713-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-zn8lw\" (UID: \"92d0cf50-e00a-4f3c-9d62-51aba1026713\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"
Jul 09 09:52:12.001012 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:12.000819    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/92d0cf50-e00a-4f3c-9d62-51aba1026713-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-zn8lw\" (UID: \"92d0cf50-e00a-4f3c-9d62-51aba1026713\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"
Jul 09 09:52:12.001012 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:12.000836    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr5n5\" (UniqueName: \"kubernetes.io/projected/92d0cf50-e00a-4f3c-9d62-51aba1026713-kube-api-access-dr5n5\") pod \"isvc-autogluon-v2-predictor-67b849d456-zn8lw\" (UID: \"92d0cf50-e00a-4f3c-9d62-51aba1026713\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"
Jul 09 09:52:12.001012 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:12.000926    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/92d0cf50-e00a-4f3c-9d62-51aba1026713-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-zn8lw\" (UID: \"92d0cf50-e00a-4f3c-9d62-51aba1026713\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"
Jul 09 09:52:12.001012 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:52:12.000967    2549 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-autogluon-v2-predictor-serving-cert: secret "isvc-autogluon-v2-predictor-serving-cert" not found
Jul 09 09:52:12.001164 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:52:12.001037    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92d0cf50-e00a-4f3c-9d62-51aba1026713-proxy-tls podName:92d0cf50-e00a-4f3c-9d62-51aba1026713 nodeName:}" failed. No retries permitted until 2026-07-09 09:52:12.501015336 +0000 UTC m=+1090.672173887 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/92d0cf50-e00a-4f3c-9d62-51aba1026713-proxy-tls") pod "isvc-autogluon-v2-predictor-67b849d456-zn8lw" (UID: "92d0cf50-e00a-4f3c-9d62-51aba1026713") : secret "isvc-autogluon-v2-predictor-serving-cert" not found
Jul 09 09:52:12.001240 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:12.001221    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/92d0cf50-e00a-4f3c-9d62-51aba1026713-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-zn8lw\" (UID: \"92d0cf50-e00a-4f3c-9d62-51aba1026713\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"
Jul 09 09:52:12.001535 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:12.001518    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/92d0cf50-e00a-4f3c-9d62-51aba1026713-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-zn8lw\" (UID: \"92d0cf50-e00a-4f3c-9d62-51aba1026713\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"
Jul 09 09:52:12.001571 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:12.001536    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/92d0cf50-e00a-4f3c-9d62-51aba1026713-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-zn8lw\" (UID: \"92d0cf50-e00a-4f3c-9d62-51aba1026713\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"
Jul 09 09:52:12.009690 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:12.009665    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr5n5\" (UniqueName: \"kubernetes.io/projected/92d0cf50-e00a-4f3c-9d62-51aba1026713-kube-api-access-dr5n5\") pod \"isvc-autogluon-v2-predictor-67b849d456-zn8lw\" (UID: \"92d0cf50-e00a-4f3c-9d62-51aba1026713\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"
Jul 09 09:52:12.386229 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:12.386200    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ec26e09-e732-4cef-886c-971a44ca358d" path="/var/lib/kubelet/pods/2ec26e09-e732-4cef-886c-971a44ca358d/volumes"
Jul 09 09:52:12.504964 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:12.504933    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/92d0cf50-e00a-4f3c-9d62-51aba1026713-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-zn8lw\" (UID: \"92d0cf50-e00a-4f3c-9d62-51aba1026713\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"
Jul 09 09:52:12.505409 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:52:12.505133    2549 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-autogluon-v2-predictor-serving-cert: secret "isvc-autogluon-v2-predictor-serving-cert" not found
Jul 09 09:52:12.505409 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:52:12.505212    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92d0cf50-e00a-4f3c-9d62-51aba1026713-proxy-tls podName:92d0cf50-e00a-4f3c-9d62-51aba1026713 nodeName:}" failed. No retries permitted until 2026-07-09 09:52:13.505192286 +0000 UTC m=+1091.676350837 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/92d0cf50-e00a-4f3c-9d62-51aba1026713-proxy-tls") pod "isvc-autogluon-v2-predictor-67b849d456-zn8lw" (UID: "92d0cf50-e00a-4f3c-9d62-51aba1026713") : secret "isvc-autogluon-v2-predictor-serving-cert" not found
Jul 09 09:52:13.513345 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:13.513313    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/92d0cf50-e00a-4f3c-9d62-51aba1026713-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-zn8lw\" (UID: \"92d0cf50-e00a-4f3c-9d62-51aba1026713\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"
Jul 09 09:52:13.515616 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:13.515594    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/92d0cf50-e00a-4f3c-9d62-51aba1026713-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-zn8lw\" (UID: \"92d0cf50-e00a-4f3c-9d62-51aba1026713\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"
Jul 09 09:52:13.616369 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:13.616344    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"
Jul 09 09:52:13.767503 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:13.767478    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"]
Jul 09 09:52:13.769954 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:52:13.769928    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92d0cf50_e00a_4f3c_9d62_51aba1026713.slice/crio-34dae078e6ce5a7071f328694050bdda0d0f014834cf23f6ca62151fbca2d13a WatchSource:0}: Error finding container 34dae078e6ce5a7071f328694050bdda0d0f014834cf23f6ca62151fbca2d13a: Status 404 returned error can't find the container with id 34dae078e6ce5a7071f328694050bdda0d0f014834cf23f6ca62151fbca2d13a
Jul 09 09:52:14.458374 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:14.458342    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw" event={"ID":"92d0cf50-e00a-4f3c-9d62-51aba1026713","Type":"ContainerStarted","Data":"39c2fdf70a1b3690f767e36f352f53769b7f8179684bbba388450597f8e3b14d"}
Jul 09 09:52:14.458536 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:14.458397    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw" event={"ID":"92d0cf50-e00a-4f3c-9d62-51aba1026713","Type":"ContainerStarted","Data":"34dae078e6ce5a7071f328694050bdda0d0f014834cf23f6ca62151fbca2d13a"}
Jul 09 09:52:34.548569 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:34.548537    2549 generic.go:356] "Generic (PLEG): container finished" podID="92d0cf50-e00a-4f3c-9d62-51aba1026713" containerID="39c2fdf70a1b3690f767e36f352f53769b7f8179684bbba388450597f8e3b14d" exitCode=0
Jul 09 09:52:34.549025 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:34.548606    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw" event={"ID":"92d0cf50-e00a-4f3c-9d62-51aba1026713","Type":"ContainerDied","Data":"39c2fdf70a1b3690f767e36f352f53769b7f8179684bbba388450597f8e3b14d"}
Jul 09 09:52:35.554540 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:35.554510    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw" event={"ID":"92d0cf50-e00a-4f3c-9d62-51aba1026713","Type":"ContainerStarted","Data":"6dc13e012614f7289ae4678925bd9f5842a161ed29d9c5b3e406458a124c2b3c"}
Jul 09 09:52:35.555008 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:35.554576    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw" event={"ID":"92d0cf50-e00a-4f3c-9d62-51aba1026713","Type":"ContainerStarted","Data":"f0b3e418e48e4e95d7366f65334a7195b89a5752a72ac248cfadf707823c8dca"}
Jul 09 09:52:35.555008 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:35.554783    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"
Jul 09 09:52:35.577828 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:35.577785    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw" podStartSLOduration=4.601979575 podStartE2EDuration="24.577771367s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.975791792s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:52:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:52:35.576050644 +0000 UTC m=+1113.747209200" watchObservedRunningTime="2026-07-09 09:52:35.577771367 +0000 UTC m=+1113.748929922"
Jul 09 09:52:36.560005 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:36.559974    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"
Jul 09 09:52:42.569300 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:52:42.569268    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"
Jul 09 09:54:12.386146 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:12.386115    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"
Jul 09 09:54:21.912820 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:21.912788    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"]
Jul 09 09:54:21.913345 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:21.913033    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw" podUID="92d0cf50-e00a-4f3c-9d62-51aba1026713" containerName="kserve-container" containerID="cri-o://f0b3e418e48e4e95d7366f65334a7195b89a5752a72ac248cfadf707823c8dca" gracePeriod=30
Jul 09 09:54:21.913345 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:21.913077    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw" podUID="92d0cf50-e00a-4f3c-9d62-51aba1026713" containerName="kube-rbac-proxy" containerID="cri-o://6dc13e012614f7289ae4678925bd9f5842a161ed29d9c5b3e406458a124c2b3c" gracePeriod=30
Jul 09 09:54:22.049768 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:22.049733    2549 generic.go:356] "Generic (PLEG): container finished" podID="92d0cf50-e00a-4f3c-9d62-51aba1026713" containerID="6dc13e012614f7289ae4678925bd9f5842a161ed29d9c5b3e406458a124c2b3c" exitCode=2
Jul 09 09:54:22.049941 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:22.049804    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw" event={"ID":"92d0cf50-e00a-4f3c-9d62-51aba1026713","Type":"ContainerDied","Data":"6dc13e012614f7289ae4678925bd9f5842a161ed29d9c5b3e406458a124c2b3c"}
Jul 09 09:54:22.382796 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:22.382753    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw" podUID="92d0cf50-e00a-4f3c-9d62-51aba1026713" containerName="kserve-container" probeResult="failure" output="Get \"http://10.134.0.38:8080/v2/models/isvc-autogluon-v2/ready\": dial tcp 10.134.0.38:8080: connect: connection refused"
Jul 09 09:54:22.564967 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:22.564925    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw" podUID="92d0cf50-e00a-4f3c-9d62-51aba1026713" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.38:8643/healthz\": dial tcp 10.134.0.38:8643: connect: connection refused"
Jul 09 09:54:22.755631 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:22.755609    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"
Jul 09 09:54:22.865133 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:22.865106    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/92d0cf50-e00a-4f3c-9d62-51aba1026713-kube-rbac-proxy-sar-config\") pod \"92d0cf50-e00a-4f3c-9d62-51aba1026713\" (UID: \"92d0cf50-e00a-4f3c-9d62-51aba1026713\") "
Jul 09 09:54:22.865329 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:22.865166    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/92d0cf50-e00a-4f3c-9d62-51aba1026713-kserve-provision-location\") pod \"92d0cf50-e00a-4f3c-9d62-51aba1026713\" (UID: \"92d0cf50-e00a-4f3c-9d62-51aba1026713\") "
Jul 09 09:54:22.865329 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:22.865196    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/92d0cf50-e00a-4f3c-9d62-51aba1026713-cabundle-cert\") pod \"92d0cf50-e00a-4f3c-9d62-51aba1026713\" (UID: \"92d0cf50-e00a-4f3c-9d62-51aba1026713\") "
Jul 09 09:54:22.865453 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:22.865328    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dr5n5\" (UniqueName: \"kubernetes.io/projected/92d0cf50-e00a-4f3c-9d62-51aba1026713-kube-api-access-dr5n5\") pod \"92d0cf50-e00a-4f3c-9d62-51aba1026713\" (UID: \"92d0cf50-e00a-4f3c-9d62-51aba1026713\") "
Jul 09 09:54:22.865453 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:22.865373    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/92d0cf50-e00a-4f3c-9d62-51aba1026713-proxy-tls\") pod \"92d0cf50-e00a-4f3c-9d62-51aba1026713\" (UID: \"92d0cf50-e00a-4f3c-9d62-51aba1026713\") "
Jul 09 09:54:22.865564 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:22.865530    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92d0cf50-e00a-4f3c-9d62-51aba1026713-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "92d0cf50-e00a-4f3c-9d62-51aba1026713" (UID: "92d0cf50-e00a-4f3c-9d62-51aba1026713"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:54:22.865564 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:22.865543    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92d0cf50-e00a-4f3c-9d62-51aba1026713-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "92d0cf50-e00a-4f3c-9d62-51aba1026713" (UID: "92d0cf50-e00a-4f3c-9d62-51aba1026713"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:54:22.865716 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:22.865699    2549 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/92d0cf50-e00a-4f3c-9d62-51aba1026713-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:54:22.865761 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:22.865722    2549 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/92d0cf50-e00a-4f3c-9d62-51aba1026713-cabundle-cert\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:54:22.867385 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:22.867359    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92d0cf50-e00a-4f3c-9d62-51aba1026713-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "92d0cf50-e00a-4f3c-9d62-51aba1026713" (UID: "92d0cf50-e00a-4f3c-9d62-51aba1026713"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:54:22.867474 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:22.867385    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92d0cf50-e00a-4f3c-9d62-51aba1026713-kube-api-access-dr5n5" (OuterVolumeSpecName: "kube-api-access-dr5n5") pod "92d0cf50-e00a-4f3c-9d62-51aba1026713" (UID: "92d0cf50-e00a-4f3c-9d62-51aba1026713"). InnerVolumeSpecName "kube-api-access-dr5n5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:54:22.869804 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:22.869781    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92d0cf50-e00a-4f3c-9d62-51aba1026713-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "92d0cf50-e00a-4f3c-9d62-51aba1026713" (UID: "92d0cf50-e00a-4f3c-9d62-51aba1026713"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:54:22.967087 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:22.967005    2549 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/92d0cf50-e00a-4f3c-9d62-51aba1026713-kserve-provision-location\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:54:22.967087 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:22.967030    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dr5n5\" (UniqueName: \"kubernetes.io/projected/92d0cf50-e00a-4f3c-9d62-51aba1026713-kube-api-access-dr5n5\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:54:22.967087 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:22.967043    2549 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/92d0cf50-e00a-4f3c-9d62-51aba1026713-proxy-tls\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:54:23.057274 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:23.057229    2549 generic.go:356] "Generic (PLEG): container finished" podID="92d0cf50-e00a-4f3c-9d62-51aba1026713" containerID="f0b3e418e48e4e95d7366f65334a7195b89a5752a72ac248cfadf707823c8dca" exitCode=0
Jul 09 09:54:23.057461 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:23.057288    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw" event={"ID":"92d0cf50-e00a-4f3c-9d62-51aba1026713","Type":"ContainerDied","Data":"f0b3e418e48e4e95d7366f65334a7195b89a5752a72ac248cfadf707823c8dca"}
Jul 09 09:54:23.057461 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:23.057306    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"
Jul 09 09:54:23.057461 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:23.057353    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw" event={"ID":"92d0cf50-e00a-4f3c-9d62-51aba1026713","Type":"ContainerDied","Data":"34dae078e6ce5a7071f328694050bdda0d0f014834cf23f6ca62151fbca2d13a"}
Jul 09 09:54:23.057461 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:23.057380    2549 scope.go:117] "RemoveContainer" containerID="6dc13e012614f7289ae4678925bd9f5842a161ed29d9c5b3e406458a124c2b3c"
Jul 09 09:54:23.067165 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:23.067150    2549 scope.go:117] "RemoveContainer" containerID="f0b3e418e48e4e95d7366f65334a7195b89a5752a72ac248cfadf707823c8dca"
Jul 09 09:54:23.074656 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:23.074642    2549 scope.go:117] "RemoveContainer" containerID="39c2fdf70a1b3690f767e36f352f53769b7f8179684bbba388450597f8e3b14d"
Jul 09 09:54:23.081923 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:23.081906    2549 scope.go:117] "RemoveContainer" containerID="6dc13e012614f7289ae4678925bd9f5842a161ed29d9c5b3e406458a124c2b3c"
Jul 09 09:54:23.081984 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:23.081964    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"]
Jul 09 09:54:23.082143 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:54:23.082124    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6dc13e012614f7289ae4678925bd9f5842a161ed29d9c5b3e406458a124c2b3c\": container with ID starting with 6dc13e012614f7289ae4678925bd9f5842a161ed29d9c5b3e406458a124c2b3c not found: ID does not exist" containerID="6dc13e012614f7289ae4678925bd9f5842a161ed29d9c5b3e406458a124c2b3c"
Jul 09 09:54:23.082198 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:23.082149    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6dc13e012614f7289ae4678925bd9f5842a161ed29d9c5b3e406458a124c2b3c"} err="failed to get container status \"6dc13e012614f7289ae4678925bd9f5842a161ed29d9c5b3e406458a124c2b3c\": rpc error: code = NotFound desc = could not find container \"6dc13e012614f7289ae4678925bd9f5842a161ed29d9c5b3e406458a124c2b3c\": container with ID starting with 6dc13e012614f7289ae4678925bd9f5842a161ed29d9c5b3e406458a124c2b3c not found: ID does not exist"
Jul 09 09:54:23.082198 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:23.082167    2549 scope.go:117] "RemoveContainer" containerID="f0b3e418e48e4e95d7366f65334a7195b89a5752a72ac248cfadf707823c8dca"
Jul 09 09:54:23.082392 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:54:23.082375    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0b3e418e48e4e95d7366f65334a7195b89a5752a72ac248cfadf707823c8dca\": container with ID starting with f0b3e418e48e4e95d7366f65334a7195b89a5752a72ac248cfadf707823c8dca not found: ID does not exist" containerID="f0b3e418e48e4e95d7366f65334a7195b89a5752a72ac248cfadf707823c8dca"
Jul 09 09:54:23.082434 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:23.082397    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0b3e418e48e4e95d7366f65334a7195b89a5752a72ac248cfadf707823c8dca"} err="failed to get container status \"f0b3e418e48e4e95d7366f65334a7195b89a5752a72ac248cfadf707823c8dca\": rpc error: code = NotFound desc = could not find container \"f0b3e418e48e4e95d7366f65334a7195b89a5752a72ac248cfadf707823c8dca\": container with ID starting with f0b3e418e48e4e95d7366f65334a7195b89a5752a72ac248cfadf707823c8dca not found: ID does not exist"
Jul 09 09:54:23.082434 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:23.082412    2549 scope.go:117] "RemoveContainer" containerID="39c2fdf70a1b3690f767e36f352f53769b7f8179684bbba388450597f8e3b14d"
Jul 09 09:54:23.082638 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:54:23.082623    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39c2fdf70a1b3690f767e36f352f53769b7f8179684bbba388450597f8e3b14d\": container with ID starting with 39c2fdf70a1b3690f767e36f352f53769b7f8179684bbba388450597f8e3b14d not found: ID does not exist" containerID="39c2fdf70a1b3690f767e36f352f53769b7f8179684bbba388450597f8e3b14d"
Jul 09 09:54:23.082696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:23.082642    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39c2fdf70a1b3690f767e36f352f53769b7f8179684bbba388450597f8e3b14d"} err="failed to get container status \"39c2fdf70a1b3690f767e36f352f53769b7f8179684bbba388450597f8e3b14d\": rpc error: code = NotFound desc = could not find container \"39c2fdf70a1b3690f767e36f352f53769b7f8179684bbba388450597f8e3b14d\": container with ID starting with 39c2fdf70a1b3690f767e36f352f53769b7f8179684bbba388450597f8e3b14d not found: ID does not exist"
Jul 09 09:54:23.086111 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:23.086088    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-zn8lw"]
Jul 09 09:54:24.385000 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:24.384963    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92d0cf50-e00a-4f3c-9d62-51aba1026713" path="/var/lib/kubelet/pods/92d0cf50-e00a-4f3c-9d62-51aba1026713/volumes"
Jul 09 09:54:27.127054 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.127020    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c"]
Jul 09 09:54:27.127421 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.127381    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92d0cf50-e00a-4f3c-9d62-51aba1026713" containerName="kube-rbac-proxy"
Jul 09 09:54:27.127421 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.127397    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="92d0cf50-e00a-4f3c-9d62-51aba1026713" containerName="kube-rbac-proxy"
Jul 09 09:54:27.127421 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.127412    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92d0cf50-e00a-4f3c-9d62-51aba1026713" containerName="storage-initializer"
Jul 09 09:54:27.127421 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.127418    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="92d0cf50-e00a-4f3c-9d62-51aba1026713" containerName="storage-initializer"
Jul 09 09:54:27.127553 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.127433    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92d0cf50-e00a-4f3c-9d62-51aba1026713" containerName="kserve-container"
Jul 09 09:54:27.127553 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.127440    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="92d0cf50-e00a-4f3c-9d62-51aba1026713" containerName="kserve-container"
Jul 09 09:54:27.127553 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.127496    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="92d0cf50-e00a-4f3c-9d62-51aba1026713" containerName="kube-rbac-proxy"
Jul 09 09:54:27.127553 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.127506    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="92d0cf50-e00a-4f3c-9d62-51aba1026713" containerName="kserve-container"
Jul 09 09:54:27.131052 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.131032    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c"
Jul 09 09:54:27.133976 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.133956    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 09:54:27.134110 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.134076    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-variants-kube-rbac-proxy-sar-config\""
Jul 09 09:54:27.134797 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.134773    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 09 09:54:27.134914 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.134859    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 09 09:54:27.134914 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.134899    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-2mhmw\""
Jul 09 09:54:27.135033 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.134919    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-variants-predictor-serving-cert\""
Jul 09 09:54:27.135033 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.134965    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 09:54:27.143700 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.143677    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c"]
Jul 09 09:54:27.302372 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.302335    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2412a8d5-c29f-4761-be9a-c368c9316961-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c\" (UID: \"2412a8d5-c29f-4761-be9a-c368c9316961\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c"
Jul 09 09:54:27.302563 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.302379    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2412a8d5-c29f-4761-be9a-c368c9316961-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c\" (UID: \"2412a8d5-c29f-4761-be9a-c368c9316961\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c"
Jul 09 09:54:27.302563 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.302425    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6hkb\" (UniqueName: \"kubernetes.io/projected/2412a8d5-c29f-4761-be9a-c368c9316961-kube-api-access-t6hkb\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c\" (UID: \"2412a8d5-c29f-4761-be9a-c368c9316961\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c"
Jul 09 09:54:27.302563 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.302454    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2412a8d5-c29f-4761-be9a-c368c9316961-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c\" (UID: \"2412a8d5-c29f-4761-be9a-c368c9316961\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c"
Jul 09 09:54:27.302563 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.302543    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2412a8d5-c29f-4761-be9a-c368c9316961-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c\" (UID: \"2412a8d5-c29f-4761-be9a-c368c9316961\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c"
Jul 09 09:54:27.403656 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.403566    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6hkb\" (UniqueName: \"kubernetes.io/projected/2412a8d5-c29f-4761-be9a-c368c9316961-kube-api-access-t6hkb\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c\" (UID: \"2412a8d5-c29f-4761-be9a-c368c9316961\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c"
Jul 09 09:54:27.403656 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.403613    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2412a8d5-c29f-4761-be9a-c368c9316961-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c\" (UID: \"2412a8d5-c29f-4761-be9a-c368c9316961\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c"
Jul 09 09:54:27.403656 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.403649    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2412a8d5-c29f-4761-be9a-c368c9316961-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c\" (UID: \"2412a8d5-c29f-4761-be9a-c368c9316961\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c"
Jul 09 09:54:27.403982 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.403671    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2412a8d5-c29f-4761-be9a-c368c9316961-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c\" (UID: \"2412a8d5-c29f-4761-be9a-c368c9316961\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c"
Jul 09 09:54:27.403982 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.403694    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2412a8d5-c29f-4761-be9a-c368c9316961-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c\" (UID: \"2412a8d5-c29f-4761-be9a-c368c9316961\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c"
Jul 09 09:54:27.404078 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.404058    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2412a8d5-c29f-4761-be9a-c368c9316961-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c\" (UID: \"2412a8d5-c29f-4761-be9a-c368c9316961\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c"
Jul 09 09:54:27.404461 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.404436    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2412a8d5-c29f-4761-be9a-c368c9316961-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c\" (UID: \"2412a8d5-c29f-4761-be9a-c368c9316961\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c"
Jul 09 09:54:27.404559 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.404436    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2412a8d5-c29f-4761-be9a-c368c9316961-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c\" (UID: \"2412a8d5-c29f-4761-be9a-c368c9316961\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c"
Jul 09 09:54:27.406102 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.406086    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2412a8d5-c29f-4761-be9a-c368c9316961-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c\" (UID: \"2412a8d5-c29f-4761-be9a-c368c9316961\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c"
Jul 09 09:54:27.412631 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.412612    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6hkb\" (UniqueName: \"kubernetes.io/projected/2412a8d5-c29f-4761-be9a-c368c9316961-kube-api-access-t6hkb\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c\" (UID: \"2412a8d5-c29f-4761-be9a-c368c9316961\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c"
Jul 09 09:54:27.441418 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.441387    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c"
Jul 09 09:54:27.571122 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:27.571094    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c"]
Jul 09 09:54:27.575027 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:54:27.575002    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2412a8d5_c29f_4761_be9a_c368c9316961.slice/crio-d27b9975fd1e7edc72914f3ac060a7153b07860d33c80405ecd21eef940ded19 WatchSource:0}: Error finding container d27b9975fd1e7edc72914f3ac060a7153b07860d33c80405ecd21eef940ded19: Status 404 returned error can't find the container with id d27b9975fd1e7edc72914f3ac060a7153b07860d33c80405ecd21eef940ded19
Jul 09 09:54:28.083458 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:28.083420    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c" event={"ID":"2412a8d5-c29f-4761-be9a-c368c9316961","Type":"ContainerStarted","Data":"7cae47d9628dfa3983f7d2b85a9011e7a4a20905e78e7850b9d429888323550a"}
Jul 09 09:54:28.083600 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:28.083493    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c" event={"ID":"2412a8d5-c29f-4761-be9a-c368c9316961","Type":"ContainerStarted","Data":"d27b9975fd1e7edc72914f3ac060a7153b07860d33c80405ecd21eef940ded19"}
Jul 09 09:54:57.220427 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:57.220391    2549 generic.go:356] "Generic (PLEG): container finished" podID="2412a8d5-c29f-4761-be9a-c368c9316961" containerID="7cae47d9628dfa3983f7d2b85a9011e7a4a20905e78e7850b9d429888323550a" exitCode=0
Jul 09 09:54:57.220837 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:57.220450    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c" event={"ID":"2412a8d5-c29f-4761-be9a-c368c9316961","Type":"ContainerDied","Data":"7cae47d9628dfa3983f7d2b85a9011e7a4a20905e78e7850b9d429888323550a"}
Jul 09 09:54:58.227630 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:58.227598    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c" event={"ID":"2412a8d5-c29f-4761-be9a-c368c9316961","Type":"ContainerStarted","Data":"315796ae4fa746acce7dc7309dce17943d9169e397770c81d78e312facaf2ac4"}
Jul 09 09:54:58.228037 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:58.227655    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c" event={"ID":"2412a8d5-c29f-4761-be9a-c368c9316961","Type":"ContainerStarted","Data":"e293462ded93d8b0d98ec8cb136e572353e19574d0dda9e776958f9c8428e434"}
Jul 09 09:54:58.228037 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:58.227833    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c"
Jul 09 09:54:58.248433 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:58.248380    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c" podStartSLOduration=2.198448128 podStartE2EDuration="31.24835965s" totalImagesPullingTime="0s" totalInitContainerRuntime="29.049911522s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:54:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:54:58.246539922 +0000 UTC m=+1256.417698479" watchObservedRunningTime="2026-07-09 09:54:58.24835965 +0000 UTC m=+1256.419518207"
Jul 09 09:54:59.232328 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:54:59.232276    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c"
Jul 09 09:55:05.241819 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:55:05.241789    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c"
Jul 09 09:56:27.384329 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:27.384242    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c"
Jul 09 09:56:37.255212 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:37.255180    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c"]
Jul 09 09:56:37.255689 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:37.255411    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c" podUID="2412a8d5-c29f-4761-be9a-c368c9316961" containerName="kserve-container" containerID="cri-o://e293462ded93d8b0d98ec8cb136e572353e19574d0dda9e776958f9c8428e434" gracePeriod=30
Jul 09 09:56:37.255689 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:37.255466    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c" podUID="2412a8d5-c29f-4761-be9a-c368c9316961" containerName="kube-rbac-proxy" containerID="cri-o://315796ae4fa746acce7dc7309dce17943d9169e397770c81d78e312facaf2ac4" gracePeriod=30
Jul 09 09:56:37.381529 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:37.381489    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c" podUID="2412a8d5-c29f-4761-be9a-c368c9316961" containerName="kserve-container" probeResult="failure" output="Get \"http://10.134.0.39:8080/v2/models/isvc-autogluon-v2-variants/ready\": dial tcp 10.134.0.39:8080: connect: connection refused"
Jul 09 09:56:37.693079 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:37.692972    2549 generic.go:356] "Generic (PLEG): container finished" podID="2412a8d5-c29f-4761-be9a-c368c9316961" containerID="315796ae4fa746acce7dc7309dce17943d9169e397770c81d78e312facaf2ac4" exitCode=2
Jul 09 09:56:37.693079 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:37.693052    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c" event={"ID":"2412a8d5-c29f-4761-be9a-c368c9316961","Type":"ContainerDied","Data":"315796ae4fa746acce7dc7309dce17943d9169e397770c81d78e312facaf2ac4"}
Jul 09 09:56:38.097417 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.097386    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c"
Jul 09 09:56:38.211797 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.211762    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6hkb\" (UniqueName: \"kubernetes.io/projected/2412a8d5-c29f-4761-be9a-c368c9316961-kube-api-access-t6hkb\") pod \"2412a8d5-c29f-4761-be9a-c368c9316961\" (UID: \"2412a8d5-c29f-4761-be9a-c368c9316961\") "
Jul 09 09:56:38.211959 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.211813    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2412a8d5-c29f-4761-be9a-c368c9316961-kube-rbac-proxy-sar-config\") pod \"2412a8d5-c29f-4761-be9a-c368c9316961\" (UID: \"2412a8d5-c29f-4761-be9a-c368c9316961\") "
Jul 09 09:56:38.211959 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.211945    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2412a8d5-c29f-4761-be9a-c368c9316961-proxy-tls\") pod \"2412a8d5-c29f-4761-be9a-c368c9316961\" (UID: \"2412a8d5-c29f-4761-be9a-c368c9316961\") "
Jul 09 09:56:38.212062 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.212003    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2412a8d5-c29f-4761-be9a-c368c9316961-cabundle-cert\") pod \"2412a8d5-c29f-4761-be9a-c368c9316961\" (UID: \"2412a8d5-c29f-4761-be9a-c368c9316961\") "
Jul 09 09:56:38.212062 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.212031    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2412a8d5-c29f-4761-be9a-c368c9316961-kserve-provision-location\") pod \"2412a8d5-c29f-4761-be9a-c368c9316961\" (UID: \"2412a8d5-c29f-4761-be9a-c368c9316961\") "
Jul 09 09:56:38.212255 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.212220    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2412a8d5-c29f-4761-be9a-c368c9316961-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "2412a8d5-c29f-4761-be9a-c368c9316961" (UID: "2412a8d5-c29f-4761-be9a-c368c9316961"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:56:38.212418 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.212393    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2412a8d5-c29f-4761-be9a-c368c9316961-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "2412a8d5-c29f-4761-be9a-c368c9316961" (UID: "2412a8d5-c29f-4761-be9a-c368c9316961"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:56:38.214078 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.214054    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2412a8d5-c29f-4761-be9a-c368c9316961-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "2412a8d5-c29f-4761-be9a-c368c9316961" (UID: "2412a8d5-c29f-4761-be9a-c368c9316961"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:56:38.214323 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.214301    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2412a8d5-c29f-4761-be9a-c368c9316961-kube-api-access-t6hkb" (OuterVolumeSpecName: "kube-api-access-t6hkb") pod "2412a8d5-c29f-4761-be9a-c368c9316961" (UID: "2412a8d5-c29f-4761-be9a-c368c9316961"). InnerVolumeSpecName "kube-api-access-t6hkb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:56:38.217174 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.217150    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2412a8d5-c29f-4761-be9a-c368c9316961-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "2412a8d5-c29f-4761-be9a-c368c9316961" (UID: "2412a8d5-c29f-4761-be9a-c368c9316961"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:56:38.312911 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.312859    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6hkb\" (UniqueName: \"kubernetes.io/projected/2412a8d5-c29f-4761-be9a-c368c9316961-kube-api-access-t6hkb\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:56:38.312911 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.312909    2549 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2412a8d5-c29f-4761-be9a-c368c9316961-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:56:38.313302 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.312921    2549 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2412a8d5-c29f-4761-be9a-c368c9316961-proxy-tls\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:56:38.313302 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.312931    2549 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2412a8d5-c29f-4761-be9a-c368c9316961-cabundle-cert\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:56:38.313302 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.312939    2549 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2412a8d5-c29f-4761-be9a-c368c9316961-kserve-provision-location\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:56:38.699518 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.699426    2549 generic.go:356] "Generic (PLEG): container finished" podID="2412a8d5-c29f-4761-be9a-c368c9316961" containerID="e293462ded93d8b0d98ec8cb136e572353e19574d0dda9e776958f9c8428e434" exitCode=0
Jul 09 09:56:38.699518 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.699490    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c"
Jul 09 09:56:38.699518 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.699506    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c" event={"ID":"2412a8d5-c29f-4761-be9a-c368c9316961","Type":"ContainerDied","Data":"e293462ded93d8b0d98ec8cb136e572353e19574d0dda9e776958f9c8428e434"}
Jul 09 09:56:38.699732 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.699567    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c" event={"ID":"2412a8d5-c29f-4761-be9a-c368c9316961","Type":"ContainerDied","Data":"d27b9975fd1e7edc72914f3ac060a7153b07860d33c80405ecd21eef940ded19"}
Jul 09 09:56:38.699732 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.699596    2549 scope.go:117] "RemoveContainer" containerID="315796ae4fa746acce7dc7309dce17943d9169e397770c81d78e312facaf2ac4"
Jul 09 09:56:38.708044 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.708028    2549 scope.go:117] "RemoveContainer" containerID="e293462ded93d8b0d98ec8cb136e572353e19574d0dda9e776958f9c8428e434"
Jul 09 09:56:38.715215 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.715200    2549 scope.go:117] "RemoveContainer" containerID="7cae47d9628dfa3983f7d2b85a9011e7a4a20905e78e7850b9d429888323550a"
Jul 09 09:56:38.719709 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.719687    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c"]
Jul 09 09:56:38.722799 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.722785    2549 scope.go:117] "RemoveContainer" containerID="315796ae4fa746acce7dc7309dce17943d9169e397770c81d78e312facaf2ac4"
Jul 09 09:56:38.723053 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:56:38.723036    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"315796ae4fa746acce7dc7309dce17943d9169e397770c81d78e312facaf2ac4\": container with ID starting with 315796ae4fa746acce7dc7309dce17943d9169e397770c81d78e312facaf2ac4 not found: ID does not exist" containerID="315796ae4fa746acce7dc7309dce17943d9169e397770c81d78e312facaf2ac4"
Jul 09 09:56:38.723106 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.723061    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"315796ae4fa746acce7dc7309dce17943d9169e397770c81d78e312facaf2ac4"} err="failed to get container status \"315796ae4fa746acce7dc7309dce17943d9169e397770c81d78e312facaf2ac4\": rpc error: code = NotFound desc = could not find container \"315796ae4fa746acce7dc7309dce17943d9169e397770c81d78e312facaf2ac4\": container with ID starting with 315796ae4fa746acce7dc7309dce17943d9169e397770c81d78e312facaf2ac4 not found: ID does not exist"
Jul 09 09:56:38.723106 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.723078    2549 scope.go:117] "RemoveContainer" containerID="e293462ded93d8b0d98ec8cb136e572353e19574d0dda9e776958f9c8428e434"
Jul 09 09:56:38.723336 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:56:38.723316    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e293462ded93d8b0d98ec8cb136e572353e19574d0dda9e776958f9c8428e434\": container with ID starting with e293462ded93d8b0d98ec8cb136e572353e19574d0dda9e776958f9c8428e434 not found: ID does not exist" containerID="e293462ded93d8b0d98ec8cb136e572353e19574d0dda9e776958f9c8428e434"
Jul 09 09:56:38.723385 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.723347    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e293462ded93d8b0d98ec8cb136e572353e19574d0dda9e776958f9c8428e434"} err="failed to get container status \"e293462ded93d8b0d98ec8cb136e572353e19574d0dda9e776958f9c8428e434\": rpc error: code = NotFound desc = could not find container \"e293462ded93d8b0d98ec8cb136e572353e19574d0dda9e776958f9c8428e434\": container with ID starting with e293462ded93d8b0d98ec8cb136e572353e19574d0dda9e776958f9c8428e434 not found: ID does not exist"
Jul 09 09:56:38.723385 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.723365    2549 scope.go:117] "RemoveContainer" containerID="7cae47d9628dfa3983f7d2b85a9011e7a4a20905e78e7850b9d429888323550a"
Jul 09 09:56:38.723579 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:56:38.723556    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cae47d9628dfa3983f7d2b85a9011e7a4a20905e78e7850b9d429888323550a\": container with ID starting with 7cae47d9628dfa3983f7d2b85a9011e7a4a20905e78e7850b9d429888323550a not found: ID does not exist" containerID="7cae47d9628dfa3983f7d2b85a9011e7a4a20905e78e7850b9d429888323550a"
Jul 09 09:56:38.723631 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.723589    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cae47d9628dfa3983f7d2b85a9011e7a4a20905e78e7850b9d429888323550a"} err="failed to get container status \"7cae47d9628dfa3983f7d2b85a9011e7a4a20905e78e7850b9d429888323550a\": rpc error: code = NotFound desc = could not find container \"7cae47d9628dfa3983f7d2b85a9011e7a4a20905e78e7850b9d429888323550a\": container with ID starting with 7cae47d9628dfa3983f7d2b85a9011e7a4a20905e78e7850b9d429888323550a not found: ID does not exist"
Jul 09 09:56:38.726317 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:38.726298    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-lxp9c"]
Jul 09 09:56:40.385451 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:40.385416    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2412a8d5-c29f-4761-be9a-c368c9316961" path="/var/lib/kubelet/pods/2412a8d5-c29f-4761-be9a-c368c9316961/volumes"
Jul 09 09:56:42.478666 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.478636    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"]
Jul 09 09:56:42.479044 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.478982    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2412a8d5-c29f-4761-be9a-c368c9316961" containerName="kube-rbac-proxy"
Jul 09 09:56:42.479044 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.478994    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="2412a8d5-c29f-4761-be9a-c368c9316961" containerName="kube-rbac-proxy"
Jul 09 09:56:42.479044 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.479009    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2412a8d5-c29f-4761-be9a-c368c9316961" containerName="kserve-container"
Jul 09 09:56:42.479044 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.479015    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="2412a8d5-c29f-4761-be9a-c368c9316961" containerName="kserve-container"
Jul 09 09:56:42.479044 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.479025    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2412a8d5-c29f-4761-be9a-c368c9316961" containerName="storage-initializer"
Jul 09 09:56:42.479044 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.479031    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="2412a8d5-c29f-4761-be9a-c368c9316961" containerName="storage-initializer"
Jul 09 09:56:42.479245 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.479147    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="2412a8d5-c29f-4761-be9a-c368c9316961" containerName="kube-rbac-proxy"
Jul 09 09:56:42.479245 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.479157    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="2412a8d5-c29f-4761-be9a-c368c9316961" containerName="kserve-container"
Jul 09 09:56:42.483608 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.483591    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"
Jul 09 09:56:42.486021 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.486002    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-2mhmw\""
Jul 09 09:56:42.486136 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.486024    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 09 09:56:42.486210 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.486172    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-noslash-kube-rbac-proxy-sar-config\""
Jul 09 09:56:42.486278 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.486263    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 09:56:42.486831 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.486804    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-noslash-predictor-serving-cert\""
Jul 09 09:56:42.486831 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.486808    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 09:56:42.487044 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.486845    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 09 09:56:42.493231 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.493212    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"]
Jul 09 09:56:42.551399 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.551375    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx\" (UID: \"2fe7a350-91ef-44f9-8d7b-dda915d5b40d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"
Jul 09 09:56:42.551530 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.551410    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx\" (UID: \"2fe7a350-91ef-44f9-8d7b-dda915d5b40d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"
Jul 09 09:56:42.551530 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.551437    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx\" (UID: \"2fe7a350-91ef-44f9-8d7b-dda915d5b40d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"
Jul 09 09:56:42.551530 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.551500    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk92b\" (UniqueName: \"kubernetes.io/projected/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-kube-api-access-nk92b\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx\" (UID: \"2fe7a350-91ef-44f9-8d7b-dda915d5b40d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"
Jul 09 09:56:42.551530 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.551523    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx\" (UID: \"2fe7a350-91ef-44f9-8d7b-dda915d5b40d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"
Jul 09 09:56:42.652406 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.652367    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx\" (UID: \"2fe7a350-91ef-44f9-8d7b-dda915d5b40d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"
Jul 09 09:56:42.652561 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.652449    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx\" (UID: \"2fe7a350-91ef-44f9-8d7b-dda915d5b40d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"
Jul 09 09:56:42.652561 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.652491    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx\" (UID: \"2fe7a350-91ef-44f9-8d7b-dda915d5b40d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"
Jul 09 09:56:42.652703 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:56:42.652629    2549 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-serving-cert: secret "isvc-autogluon-v2-noslash-predictor-serving-cert" not found
Jul 09 09:56:42.652703 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.652633    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx\" (UID: \"2fe7a350-91ef-44f9-8d7b-dda915d5b40d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"
Jul 09 09:56:42.652781 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:56:42.652715    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-proxy-tls podName:2fe7a350-91ef-44f9-8d7b-dda915d5b40d nodeName:}" failed. No retries permitted until 2026-07-09 09:56:43.15269457 +0000 UTC m=+1361.323853116 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-proxy-tls") pod "isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx" (UID: "2fe7a350-91ef-44f9-8d7b-dda915d5b40d") : secret "isvc-autogluon-v2-noslash-predictor-serving-cert" not found
Jul 09 09:56:42.652781 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.652761    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk92b\" (UniqueName: \"kubernetes.io/projected/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-kube-api-access-nk92b\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx\" (UID: \"2fe7a350-91ef-44f9-8d7b-dda915d5b40d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"
Jul 09 09:56:42.653033 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.653004    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx\" (UID: \"2fe7a350-91ef-44f9-8d7b-dda915d5b40d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"
Jul 09 09:56:42.653210 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.653190    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx\" (UID: \"2fe7a350-91ef-44f9-8d7b-dda915d5b40d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"
Jul 09 09:56:42.653259 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.653205    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx\" (UID: \"2fe7a350-91ef-44f9-8d7b-dda915d5b40d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"
Jul 09 09:56:42.662098 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:42.662073    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk92b\" (UniqueName: \"kubernetes.io/projected/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-kube-api-access-nk92b\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx\" (UID: \"2fe7a350-91ef-44f9-8d7b-dda915d5b40d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"
Jul 09 09:56:43.157810 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:43.157774    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx\" (UID: \"2fe7a350-91ef-44f9-8d7b-dda915d5b40d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"
Jul 09 09:56:43.160206 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:43.160176    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx\" (UID: \"2fe7a350-91ef-44f9-8d7b-dda915d5b40d\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"
Jul 09 09:56:43.395248 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:43.395204    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"
Jul 09 09:56:43.523486 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:43.523463    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"]
Jul 09 09:56:43.525610 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:56:43.525580    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fe7a350_91ef_44f9_8d7b_dda915d5b40d.slice/crio-ea88a1fa816b03d6fdce5366adc634a4ae71dacf4f4bfd6ff880ff265b0ef082 WatchSource:0}: Error finding container ea88a1fa816b03d6fdce5366adc634a4ae71dacf4f4bfd6ff880ff265b0ef082: Status 404 returned error can't find the container with id ea88a1fa816b03d6fdce5366adc634a4ae71dacf4f4bfd6ff880ff265b0ef082
Jul 09 09:56:43.723152 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:43.723068    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx" event={"ID":"2fe7a350-91ef-44f9-8d7b-dda915d5b40d","Type":"ContainerStarted","Data":"71ba4788ff032c353e91d1d224f039db8e58e9f1bce375eccfc60ea7e46f594c"}
Jul 09 09:56:43.723152 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:43.723128    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx" event={"ID":"2fe7a350-91ef-44f9-8d7b-dda915d5b40d","Type":"ContainerStarted","Data":"ea88a1fa816b03d6fdce5366adc634a4ae71dacf4f4bfd6ff880ff265b0ef082"}
Jul 09 09:56:48.746950 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:48.746914    2549 generic.go:356] "Generic (PLEG): container finished" podID="2fe7a350-91ef-44f9-8d7b-dda915d5b40d" containerID="71ba4788ff032c353e91d1d224f039db8e58e9f1bce375eccfc60ea7e46f594c" exitCode=0
Jul 09 09:56:48.747338 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:48.746958    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx" event={"ID":"2fe7a350-91ef-44f9-8d7b-dda915d5b40d","Type":"ContainerDied","Data":"71ba4788ff032c353e91d1d224f039db8e58e9f1bce375eccfc60ea7e46f594c"}
Jul 09 09:56:49.754026 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:49.753993    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx" event={"ID":"2fe7a350-91ef-44f9-8d7b-dda915d5b40d","Type":"ContainerStarted","Data":"1fb5e812c4a511660993f25e6f9940af625cfd32e514ad0bf53d60129d0decbe"}
Jul 09 09:56:49.754431 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:49.754081    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx" event={"ID":"2fe7a350-91ef-44f9-8d7b-dda915d5b40d","Type":"ContainerStarted","Data":"81a7d5640af0dcd62c3e229bb7267967a9692b2ec748f27e623cfa3ecd0be14f"}
Jul 09 09:56:49.754431 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:49.754270    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"
Jul 09 09:56:49.776367 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:49.776324    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx" podStartSLOduration=3.129954947 podStartE2EDuration="7.776311468s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.646356521s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:56:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:56:49.775044636 +0000 UTC m=+1367.946203193" watchObservedRunningTime="2026-07-09 09:56:49.776311468 +0000 UTC m=+1367.947470024"
Jul 09 09:56:50.760882 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:50.760838    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"
Jul 09 09:56:56.770683 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:56:56.770653    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"
Jul 09 09:58:21.384595 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:21.384519    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"
Jul 09 09:58:22.648268 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:22.648236    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"]
Jul 09 09:58:22.648695 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:22.648431    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx" podUID="2fe7a350-91ef-44f9-8d7b-dda915d5b40d" containerName="kserve-container" containerID="cri-o://81a7d5640af0dcd62c3e229bb7267967a9692b2ec748f27e623cfa3ecd0be14f" gracePeriod=30
Jul 09 09:58:22.648695 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:22.648478    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx" podUID="2fe7a350-91ef-44f9-8d7b-dda915d5b40d" containerName="kube-rbac-proxy" containerID="cri-o://1fb5e812c4a511660993f25e6f9940af625cfd32e514ad0bf53d60129d0decbe" gracePeriod=30
Jul 09 09:58:23.181689 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:23.181658    2549 generic.go:356] "Generic (PLEG): container finished" podID="2fe7a350-91ef-44f9-8d7b-dda915d5b40d" containerID="1fb5e812c4a511660993f25e6f9940af625cfd32e514ad0bf53d60129d0decbe" exitCode=2
Jul 09 09:58:23.181857 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:23.181727    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx" event={"ID":"2fe7a350-91ef-44f9-8d7b-dda915d5b40d","Type":"ContainerDied","Data":"1fb5e812c4a511660993f25e6f9940af625cfd32e514ad0bf53d60129d0decbe"}
Jul 09 09:58:23.390225 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:23.390196    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"
Jul 09 09:58:23.523611 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:23.523574    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nk92b\" (UniqueName: \"kubernetes.io/projected/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-kube-api-access-nk92b\") pod \"2fe7a350-91ef-44f9-8d7b-dda915d5b40d\" (UID: \"2fe7a350-91ef-44f9-8d7b-dda915d5b40d\") "
Jul 09 09:58:23.523778 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:23.523632    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-proxy-tls\") pod \"2fe7a350-91ef-44f9-8d7b-dda915d5b40d\" (UID: \"2fe7a350-91ef-44f9-8d7b-dda915d5b40d\") "
Jul 09 09:58:23.523778 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:23.523656    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-cabundle-cert\") pod \"2fe7a350-91ef-44f9-8d7b-dda915d5b40d\" (UID: \"2fe7a350-91ef-44f9-8d7b-dda915d5b40d\") "
Jul 09 09:58:23.523778 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:23.523699    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-kube-rbac-proxy-sar-config\") pod \"2fe7a350-91ef-44f9-8d7b-dda915d5b40d\" (UID: \"2fe7a350-91ef-44f9-8d7b-dda915d5b40d\") "
Jul 09 09:58:23.523778 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:23.523723    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-kserve-provision-location\") pod \"2fe7a350-91ef-44f9-8d7b-dda915d5b40d\" (UID: \"2fe7a350-91ef-44f9-8d7b-dda915d5b40d\") "
Jul 09 09:58:23.524100 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:23.524068    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "2fe7a350-91ef-44f9-8d7b-dda915d5b40d" (UID: "2fe7a350-91ef-44f9-8d7b-dda915d5b40d"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:58:23.524100 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:23.524075    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "2fe7a350-91ef-44f9-8d7b-dda915d5b40d" (UID: "2fe7a350-91ef-44f9-8d7b-dda915d5b40d"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:58:23.525631 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:23.525603    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-kube-api-access-nk92b" (OuterVolumeSpecName: "kube-api-access-nk92b") pod "2fe7a350-91ef-44f9-8d7b-dda915d5b40d" (UID: "2fe7a350-91ef-44f9-8d7b-dda915d5b40d"). InnerVolumeSpecName "kube-api-access-nk92b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:58:23.525700 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:23.525644    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "2fe7a350-91ef-44f9-8d7b-dda915d5b40d" (UID: "2fe7a350-91ef-44f9-8d7b-dda915d5b40d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:58:23.530029 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:23.530004    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "2fe7a350-91ef-44f9-8d7b-dda915d5b40d" (UID: "2fe7a350-91ef-44f9-8d7b-dda915d5b40d"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:58:23.624771 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:23.624737    2549 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-proxy-tls\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:58:23.624771 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:23.624770    2549 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-cabundle-cert\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:58:23.624771 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:23.624780    2549 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:58:23.625003 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:23.624788    2549 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-kserve-provision-location\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:58:23.625003 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:23.624797    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nk92b\" (UniqueName: \"kubernetes.io/projected/2fe7a350-91ef-44f9-8d7b-dda915d5b40d-kube-api-access-nk92b\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:58:24.187634 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:24.187594    2549 generic.go:356] "Generic (PLEG): container finished" podID="2fe7a350-91ef-44f9-8d7b-dda915d5b40d" containerID="81a7d5640af0dcd62c3e229bb7267967a9692b2ec748f27e623cfa3ecd0be14f" exitCode=0
Jul 09 09:58:24.188056 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:24.187645    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"
Jul 09 09:58:24.188056 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:24.187665    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx" event={"ID":"2fe7a350-91ef-44f9-8d7b-dda915d5b40d","Type":"ContainerDied","Data":"81a7d5640af0dcd62c3e229bb7267967a9692b2ec748f27e623cfa3ecd0be14f"}
Jul 09 09:58:24.188056 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:24.187716    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx" event={"ID":"2fe7a350-91ef-44f9-8d7b-dda915d5b40d","Type":"ContainerDied","Data":"ea88a1fa816b03d6fdce5366adc634a4ae71dacf4f4bfd6ff880ff265b0ef082"}
Jul 09 09:58:24.188056 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:24.187742    2549 scope.go:117] "RemoveContainer" containerID="1fb5e812c4a511660993f25e6f9940af625cfd32e514ad0bf53d60129d0decbe"
Jul 09 09:58:24.197000 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:24.196981    2549 scope.go:117] "RemoveContainer" containerID="81a7d5640af0dcd62c3e229bb7267967a9692b2ec748f27e623cfa3ecd0be14f"
Jul 09 09:58:24.204537 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:24.204523    2549 scope.go:117] "RemoveContainer" containerID="71ba4788ff032c353e91d1d224f039db8e58e9f1bce375eccfc60ea7e46f594c"
Jul 09 09:58:24.212852 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:24.212818    2549 scope.go:117] "RemoveContainer" containerID="1fb5e812c4a511660993f25e6f9940af625cfd32e514ad0bf53d60129d0decbe"
Jul 09 09:58:24.213287 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:58:24.213265    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fb5e812c4a511660993f25e6f9940af625cfd32e514ad0bf53d60129d0decbe\": container with ID starting with 1fb5e812c4a511660993f25e6f9940af625cfd32e514ad0bf53d60129d0decbe not found: ID does not exist" containerID="1fb5e812c4a511660993f25e6f9940af625cfd32e514ad0bf53d60129d0decbe"
Jul 09 09:58:24.213387 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:24.213300    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fb5e812c4a511660993f25e6f9940af625cfd32e514ad0bf53d60129d0decbe"} err="failed to get container status \"1fb5e812c4a511660993f25e6f9940af625cfd32e514ad0bf53d60129d0decbe\": rpc error: code = NotFound desc = could not find container \"1fb5e812c4a511660993f25e6f9940af625cfd32e514ad0bf53d60129d0decbe\": container with ID starting with 1fb5e812c4a511660993f25e6f9940af625cfd32e514ad0bf53d60129d0decbe not found: ID does not exist"
Jul 09 09:58:24.213387 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:24.213324    2549 scope.go:117] "RemoveContainer" containerID="81a7d5640af0dcd62c3e229bb7267967a9692b2ec748f27e623cfa3ecd0be14f"
Jul 09 09:58:24.213602 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:58:24.213583    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81a7d5640af0dcd62c3e229bb7267967a9692b2ec748f27e623cfa3ecd0be14f\": container with ID starting with 81a7d5640af0dcd62c3e229bb7267967a9692b2ec748f27e623cfa3ecd0be14f not found: ID does not exist" containerID="81a7d5640af0dcd62c3e229bb7267967a9692b2ec748f27e623cfa3ecd0be14f"
Jul 09 09:58:24.213649 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:24.213611    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81a7d5640af0dcd62c3e229bb7267967a9692b2ec748f27e623cfa3ecd0be14f"} err="failed to get container status \"81a7d5640af0dcd62c3e229bb7267967a9692b2ec748f27e623cfa3ecd0be14f\": rpc error: code = NotFound desc = could not find container \"81a7d5640af0dcd62c3e229bb7267967a9692b2ec748f27e623cfa3ecd0be14f\": container with ID starting with 81a7d5640af0dcd62c3e229bb7267967a9692b2ec748f27e623cfa3ecd0be14f not found: ID does not exist"
Jul 09 09:58:24.213649 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:24.213628    2549 scope.go:117] "RemoveContainer" containerID="71ba4788ff032c353e91d1d224f039db8e58e9f1bce375eccfc60ea7e46f594c"
Jul 09 09:58:24.213912 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:58:24.213892    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71ba4788ff032c353e91d1d224f039db8e58e9f1bce375eccfc60ea7e46f594c\": container with ID starting with 71ba4788ff032c353e91d1d224f039db8e58e9f1bce375eccfc60ea7e46f594c not found: ID does not exist" containerID="71ba4788ff032c353e91d1d224f039db8e58e9f1bce375eccfc60ea7e46f594c"
Jul 09 09:58:24.213971 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:24.213917    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71ba4788ff032c353e91d1d224f039db8e58e9f1bce375eccfc60ea7e46f594c"} err="failed to get container status \"71ba4788ff032c353e91d1d224f039db8e58e9f1bce375eccfc60ea7e46f594c\": rpc error: code = NotFound desc = could not find container \"71ba4788ff032c353e91d1d224f039db8e58e9f1bce375eccfc60ea7e46f594c\": container with ID starting with 71ba4788ff032c353e91d1d224f039db8e58e9f1bce375eccfc60ea7e46f594c not found: ID does not exist"
Jul 09 09:58:24.215218 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:24.215199    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"]
Jul 09 09:58:24.218647 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:24.218628    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-lm7jx"]
Jul 09 09:58:24.385226 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:24.385199    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fe7a350-91ef-44f9-8d7b-dda915d5b40d" path="/var/lib/kubelet/pods/2fe7a350-91ef-44f9-8d7b-dda915d5b40d/volumes"
Jul 09 09:58:27.826180 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:27.826149    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"]
Jul 09 09:58:27.826635 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:27.826505    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fe7a350-91ef-44f9-8d7b-dda915d5b40d" containerName="kserve-container"
Jul 09 09:58:27.826635 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:27.826517    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fe7a350-91ef-44f9-8d7b-dda915d5b40d" containerName="kserve-container"
Jul 09 09:58:27.826635 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:27.826535    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fe7a350-91ef-44f9-8d7b-dda915d5b40d" containerName="kube-rbac-proxy"
Jul 09 09:58:27.826635 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:27.826541    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fe7a350-91ef-44f9-8d7b-dda915d5b40d" containerName="kube-rbac-proxy"
Jul 09 09:58:27.826635 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:27.826562    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fe7a350-91ef-44f9-8d7b-dda915d5b40d" containerName="storage-initializer"
Jul 09 09:58:27.826635 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:27.826569    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fe7a350-91ef-44f9-8d7b-dda915d5b40d" containerName="storage-initializer"
Jul 09 09:58:27.826635 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:27.826620    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fe7a350-91ef-44f9-8d7b-dda915d5b40d" containerName="kserve-container"
Jul 09 09:58:27.826635 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:27.826629    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fe7a350-91ef-44f9-8d7b-dda915d5b40d" containerName="kube-rbac-proxy"
Jul 09 09:58:27.831560 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:27.831543    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"
Jul 09 09:58:27.835182 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:27.835163    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-2mhmw\""
Jul 09 09:58:27.835299 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:27.835164    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-ts-v1-kube-rbac-proxy-sar-config\""
Jul 09 09:58:27.836098 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:27.836077    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 09:58:27.836098 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:27.836092    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 09 09:58:27.836305 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:27.836105    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-ts-v1-predictor-serving-cert\""
Jul 09 09:58:27.836305 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:27.836115    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 09 09:58:27.836401 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:27.836388    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 09:58:27.845395 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:27.845374    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"]
Jul 09 09:58:27.960702 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:27.960672    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-cabundle-cert\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x\" (UID: \"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"
Jul 09 09:58:27.960702 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:27.960702    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tc59\" (UniqueName: \"kubernetes.io/projected/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-kube-api-access-6tc59\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x\" (UID: \"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"
Jul 09 09:58:27.960946 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:27.960736    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x\" (UID: \"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"
Jul 09 09:58:27.960946 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:27.960758    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-kserve-provision-location\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x\" (UID: \"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"
Jul 09 09:58:27.960946 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:27.960820    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x\" (UID: \"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"
Jul 09 09:58:28.061828 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:28.061791    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x\" (UID: \"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"
Jul 09 09:58:28.061828 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:28.061831    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-kserve-provision-location\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x\" (UID: \"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"
Jul 09 09:58:28.062069 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:28.061850    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x\" (UID: \"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"
Jul 09 09:58:28.062069 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:28.061928    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-cabundle-cert\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x\" (UID: \"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"
Jul 09 09:58:28.062069 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:28.061945    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tc59\" (UniqueName: \"kubernetes.io/projected/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-kube-api-access-6tc59\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x\" (UID: \"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"
Jul 09 09:58:28.062069 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:58:28.062052    2549 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-serving-cert: secret "isvc-autogluon-ts-v1-predictor-serving-cert" not found
Jul 09 09:58:28.062286 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:58:28.062141    2549 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-proxy-tls podName:9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d nodeName:}" failed. No retries permitted until 2026-07-09 09:58:28.562116672 +0000 UTC m=+1466.733275220 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-proxy-tls") pod "isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x" (UID: "9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d") : secret "isvc-autogluon-ts-v1-predictor-serving-cert" not found
Jul 09 09:58:28.062286 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:28.062248    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-kserve-provision-location\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x\" (UID: \"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"
Jul 09 09:58:28.062559 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:28.062542    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-cabundle-cert\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x\" (UID: \"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"
Jul 09 09:58:28.062596 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:28.062546    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x\" (UID: \"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"
Jul 09 09:58:28.073349 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:28.073324    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tc59\" (UniqueName: \"kubernetes.io/projected/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-kube-api-access-6tc59\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x\" (UID: \"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"
Jul 09 09:58:28.566218 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:28.566178    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x\" (UID: \"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"
Jul 09 09:58:28.568521 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:28.568498    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x\" (UID: \"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"
Jul 09 09:58:28.742143 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:28.742106    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"
Jul 09 09:58:28.869746 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:28.869721    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"]
Jul 09 09:58:28.871822 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:58:28.871796    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9afcb5e3_dc0c_4830_8dcc_63ffd2466e6d.slice/crio-dd2b8b0571c83b358bff645f024496a09859c1fe7f3e48bc406b18e5be8de72b WatchSource:0}: Error finding container dd2b8b0571c83b358bff645f024496a09859c1fe7f3e48bc406b18e5be8de72b: Status 404 returned error can't find the container with id dd2b8b0571c83b358bff645f024496a09859c1fe7f3e48bc406b18e5be8de72b
Jul 09 09:58:29.214332 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:29.214248    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x" event={"ID":"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d","Type":"ContainerStarted","Data":"bcb501587e1a431e787d99b0167f390bc2c33cac077d0e334b2a8994210e5ae3"}
Jul 09 09:58:29.214332 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:29.214312    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x" event={"ID":"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d","Type":"ContainerStarted","Data":"dd2b8b0571c83b358bff645f024496a09859c1fe7f3e48bc406b18e5be8de72b"}
Jul 09 09:58:38.259075 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:38.259045    2549 generic.go:356] "Generic (PLEG): container finished" podID="9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d" containerID="bcb501587e1a431e787d99b0167f390bc2c33cac077d0e334b2a8994210e5ae3" exitCode=0
Jul 09 09:58:38.259454 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:38.259115    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x" event={"ID":"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d","Type":"ContainerDied","Data":"bcb501587e1a431e787d99b0167f390bc2c33cac077d0e334b2a8994210e5ae3"}
Jul 09 09:58:39.266313 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:39.266277    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x" event={"ID":"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d","Type":"ContainerStarted","Data":"96a41d845cb76a031fd20c7ac45af8abbcb9a6ec8dd040e698ae36a42487f4ea"}
Jul 09 09:58:39.266719 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:39.266343    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x" event={"ID":"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d","Type":"ContainerStarted","Data":"2ad8f2e828c04c9f5bbc1fc45cf6da1f0c99f0962aea1a35568af5d1ad123287"}
Jul 09 09:58:39.266719 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:39.266535    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"
Jul 09 09:58:39.306310 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:39.306261    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x" podStartSLOduration=3.4164753709999998 podStartE2EDuration="12.306246098s" totalImagesPullingTime="0s" totalInitContainerRuntime="8.889770727s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:58:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:58:39.304332859 +0000 UTC m=+1477.475491428" watchObservedRunningTime="2026-07-09 09:58:39.306246098 +0000 UTC m=+1477.477404654"
Jul 09 09:58:40.272013 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:40.271978    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"
Jul 09 09:58:40.273246 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:40.273219    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x" podUID="9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Jul 09 09:58:41.277421 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:41.277388    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x" podUID="9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Jul 09 09:58:46.282669 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:46.282637    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"
Jul 09 09:58:46.283344 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:46.283315    2549 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x" podUID="9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Jul 09 09:58:56.283962 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:56.283931    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"
Jul 09 09:58:57.841295 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:57.841261    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"]
Jul 09 09:58:57.841680 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:57.841481    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x" podUID="9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d" containerName="kserve-container" containerID="cri-o://2ad8f2e828c04c9f5bbc1fc45cf6da1f0c99f0962aea1a35568af5d1ad123287" gracePeriod=30
Jul 09 09:58:57.841680 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:57.841511    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x" podUID="9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d" containerName="kube-rbac-proxy" containerID="cri-o://96a41d845cb76a031fd20c7ac45af8abbcb9a6ec8dd040e698ae36a42487f4ea" gracePeriod=30
Jul 09 09:58:58.354743 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:58.354709    2549 generic.go:356] "Generic (PLEG): container finished" podID="9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d" containerID="96a41d845cb76a031fd20c7ac45af8abbcb9a6ec8dd040e698ae36a42487f4ea" exitCode=2
Jul 09 09:58:58.354946 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:58.354784    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x" event={"ID":"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d","Type":"ContainerDied","Data":"96a41d845cb76a031fd20c7ac45af8abbcb9a6ec8dd040e698ae36a42487f4ea"}
Jul 09 09:58:58.677990 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:58.677966    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"
Jul 09 09:58:58.738556 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:58.738517    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-kserve-provision-location\") pod \"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d\" (UID: \"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d\") "
Jul 09 09:58:58.738733 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:58.738582    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-proxy-tls\") pod \"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d\" (UID: \"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d\") "
Jul 09 09:58:58.738733 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:58.738638    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-cabundle-cert\") pod \"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d\" (UID: \"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d\") "
Jul 09 09:58:58.738733 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:58.738666    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-kube-rbac-proxy-sar-config\") pod \"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d\" (UID: \"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d\") "
Jul 09 09:58:58.738733 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:58.738685    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6tc59\" (UniqueName: \"kubernetes.io/projected/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-kube-api-access-6tc59\") pod \"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d\" (UID: \"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d\") "
Jul 09 09:58:58.739083 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:58.739055    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d" (UID: "9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:58:58.739083 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:58.739056    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d" (UID: "9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 09:58:58.739646 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:58.739626    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d" (UID: "9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:58:58.740749 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:58.740724    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d" (UID: "9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 09:58:58.740749 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:58.740735    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-kube-api-access-6tc59" (OuterVolumeSpecName: "kube-api-access-6tc59") pod "9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d" (UID: "9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d"). InnerVolumeSpecName "kube-api-access-6tc59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:58:58.840146 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:58.840109    2549 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-cabundle-cert\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:58:58.840146 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:58.840141    2549 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-kube-rbac-proxy-sar-config\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:58:58.840146 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:58.840153    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tc59\" (UniqueName: \"kubernetes.io/projected/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-kube-api-access-6tc59\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:58:58.840378 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:58.840162    2549 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-kserve-provision-location\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:58:58.840378 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:58.840171    2549 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d-proxy-tls\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:58:59.361508 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:59.361473    2549 generic.go:356] "Generic (PLEG): container finished" podID="9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d" containerID="2ad8f2e828c04c9f5bbc1fc45cf6da1f0c99f0962aea1a35568af5d1ad123287" exitCode=0
Jul 09 09:58:59.361995 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:59.361535    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"
Jul 09 09:58:59.361995 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:59.361558    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x" event={"ID":"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d","Type":"ContainerDied","Data":"2ad8f2e828c04c9f5bbc1fc45cf6da1f0c99f0962aea1a35568af5d1ad123287"}
Jul 09 09:58:59.361995 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:59.361618    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x" event={"ID":"9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d","Type":"ContainerDied","Data":"dd2b8b0571c83b358bff645f024496a09859c1fe7f3e48bc406b18e5be8de72b"}
Jul 09 09:58:59.361995 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:59.361645    2549 scope.go:117] "RemoveContainer" containerID="96a41d845cb76a031fd20c7ac45af8abbcb9a6ec8dd040e698ae36a42487f4ea"
Jul 09 09:58:59.370389 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:59.370373    2549 scope.go:117] "RemoveContainer" containerID="2ad8f2e828c04c9f5bbc1fc45cf6da1f0c99f0962aea1a35568af5d1ad123287"
Jul 09 09:58:59.377554 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:59.377536    2549 scope.go:117] "RemoveContainer" containerID="bcb501587e1a431e787d99b0167f390bc2c33cac077d0e334b2a8994210e5ae3"
Jul 09 09:58:59.384688 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:59.384667    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"]
Jul 09 09:58:59.385060 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:59.385043    2549 scope.go:117] "RemoveContainer" containerID="96a41d845cb76a031fd20c7ac45af8abbcb9a6ec8dd040e698ae36a42487f4ea"
Jul 09 09:58:59.385309 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:58:59.385289    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96a41d845cb76a031fd20c7ac45af8abbcb9a6ec8dd040e698ae36a42487f4ea\": container with ID starting with 96a41d845cb76a031fd20c7ac45af8abbcb9a6ec8dd040e698ae36a42487f4ea not found: ID does not exist" containerID="96a41d845cb76a031fd20c7ac45af8abbcb9a6ec8dd040e698ae36a42487f4ea"
Jul 09 09:58:59.385358 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:59.385322    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96a41d845cb76a031fd20c7ac45af8abbcb9a6ec8dd040e698ae36a42487f4ea"} err="failed to get container status \"96a41d845cb76a031fd20c7ac45af8abbcb9a6ec8dd040e698ae36a42487f4ea\": rpc error: code = NotFound desc = could not find container \"96a41d845cb76a031fd20c7ac45af8abbcb9a6ec8dd040e698ae36a42487f4ea\": container with ID starting with 96a41d845cb76a031fd20c7ac45af8abbcb9a6ec8dd040e698ae36a42487f4ea not found: ID does not exist"
Jul 09 09:58:59.385358 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:59.385339    2549 scope.go:117] "RemoveContainer" containerID="2ad8f2e828c04c9f5bbc1fc45cf6da1f0c99f0962aea1a35568af5d1ad123287"
Jul 09 09:58:59.385588 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:58:59.385566    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ad8f2e828c04c9f5bbc1fc45cf6da1f0c99f0962aea1a35568af5d1ad123287\": container with ID starting with 2ad8f2e828c04c9f5bbc1fc45cf6da1f0c99f0962aea1a35568af5d1ad123287 not found: ID does not exist" containerID="2ad8f2e828c04c9f5bbc1fc45cf6da1f0c99f0962aea1a35568af5d1ad123287"
Jul 09 09:58:59.385658 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:59.385596    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ad8f2e828c04c9f5bbc1fc45cf6da1f0c99f0962aea1a35568af5d1ad123287"} err="failed to get container status \"2ad8f2e828c04c9f5bbc1fc45cf6da1f0c99f0962aea1a35568af5d1ad123287\": rpc error: code = NotFound desc = could not find container \"2ad8f2e828c04c9f5bbc1fc45cf6da1f0c99f0962aea1a35568af5d1ad123287\": container with ID starting with 2ad8f2e828c04c9f5bbc1fc45cf6da1f0c99f0962aea1a35568af5d1ad123287 not found: ID does not exist"
Jul 09 09:58:59.385658 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:59.385617    2549 scope.go:117] "RemoveContainer" containerID="bcb501587e1a431e787d99b0167f390bc2c33cac077d0e334b2a8994210e5ae3"
Jul 09 09:58:59.385837 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:58:59.385820    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcb501587e1a431e787d99b0167f390bc2c33cac077d0e334b2a8994210e5ae3\": container with ID starting with bcb501587e1a431e787d99b0167f390bc2c33cac077d0e334b2a8994210e5ae3 not found: ID does not exist" containerID="bcb501587e1a431e787d99b0167f390bc2c33cac077d0e334b2a8994210e5ae3"
Jul 09 09:58:59.386119 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:59.385840    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcb501587e1a431e787d99b0167f390bc2c33cac077d0e334b2a8994210e5ae3"} err="failed to get container status \"bcb501587e1a431e787d99b0167f390bc2c33cac077d0e334b2a8994210e5ae3\": rpc error: code = NotFound desc = could not find container \"bcb501587e1a431e787d99b0167f390bc2c33cac077d0e334b2a8994210e5ae3\": container with ID starting with bcb501587e1a431e787d99b0167f390bc2c33cac077d0e334b2a8994210e5ae3 not found: ID does not exist"
Jul 09 09:58:59.393061 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:58:59.393033    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-hz54x"]
Jul 09 09:59:00.385001 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:00.384964    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d" path="/var/lib/kubelet/pods/9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d/volumes"
Jul 09 09:59:04.327489 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:04.327453    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6wxdk/must-gather-ml6wr"]
Jul 09 09:59:04.327989 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:04.327933    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d" containerName="kube-rbac-proxy"
Jul 09 09:59:04.327989 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:04.327949    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d" containerName="kube-rbac-proxy"
Jul 09 09:59:04.327989 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:04.327962    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d" containerName="storage-initializer"
Jul 09 09:59:04.327989 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:04.327968    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d" containerName="storage-initializer"
Jul 09 09:59:04.327989 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:04.327978    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d" containerName="kserve-container"
Jul 09 09:59:04.327989 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:04.327985    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d" containerName="kserve-container"
Jul 09 09:59:04.328277 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:04.328036    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d" containerName="kserve-container"
Jul 09 09:59:04.328277 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:04.328044    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="9afcb5e3-dc0c-4830-8dcc-63ffd2466e6d" containerName="kube-rbac-proxy"
Jul 09 09:59:04.331075 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:04.331058    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6wxdk/must-gather-ml6wr"
Jul 09 09:59:04.334433 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:04.334406    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6wxdk\"/\"openshift-service-ca.crt\""
Jul 09 09:59:04.334433 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:04.334428    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6wxdk\"/\"default-dockercfg-ml6p5\""
Jul 09 09:59:04.334600 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:04.334405    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6wxdk\"/\"kube-root-ca.crt\""
Jul 09 09:59:04.343396 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:04.343372    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6wxdk/must-gather-ml6wr"]
Jul 09 09:59:04.387794 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:04.387762    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7efd0569-8298-44c9-bfe0-f6ad3b48fc49-must-gather-output\") pod \"must-gather-ml6wr\" (UID: \"7efd0569-8298-44c9-bfe0-f6ad3b48fc49\") " pod="openshift-must-gather-6wxdk/must-gather-ml6wr"
Jul 09 09:59:04.387956 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:04.387854    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmzft\" (UniqueName: \"kubernetes.io/projected/7efd0569-8298-44c9-bfe0-f6ad3b48fc49-kube-api-access-kmzft\") pod \"must-gather-ml6wr\" (UID: \"7efd0569-8298-44c9-bfe0-f6ad3b48fc49\") " pod="openshift-must-gather-6wxdk/must-gather-ml6wr"
Jul 09 09:59:04.488555 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:04.488518    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmzft\" (UniqueName: \"kubernetes.io/projected/7efd0569-8298-44c9-bfe0-f6ad3b48fc49-kube-api-access-kmzft\") pod \"must-gather-ml6wr\" (UID: \"7efd0569-8298-44c9-bfe0-f6ad3b48fc49\") " pod="openshift-must-gather-6wxdk/must-gather-ml6wr"
Jul 09 09:59:04.488751 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:04.488576    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7efd0569-8298-44c9-bfe0-f6ad3b48fc49-must-gather-output\") pod \"must-gather-ml6wr\" (UID: \"7efd0569-8298-44c9-bfe0-f6ad3b48fc49\") " pod="openshift-must-gather-6wxdk/must-gather-ml6wr"
Jul 09 09:59:04.488878 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:04.488844    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7efd0569-8298-44c9-bfe0-f6ad3b48fc49-must-gather-output\") pod \"must-gather-ml6wr\" (UID: \"7efd0569-8298-44c9-bfe0-f6ad3b48fc49\") " pod="openshift-must-gather-6wxdk/must-gather-ml6wr"
Jul 09 09:59:04.498039 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:04.498011    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmzft\" (UniqueName: \"kubernetes.io/projected/7efd0569-8298-44c9-bfe0-f6ad3b48fc49-kube-api-access-kmzft\") pod \"must-gather-ml6wr\" (UID: \"7efd0569-8298-44c9-bfe0-f6ad3b48fc49\") " pod="openshift-must-gather-6wxdk/must-gather-ml6wr"
Jul 09 09:59:04.652488 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:04.652410    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6wxdk/must-gather-ml6wr"
Jul 09 09:59:04.783043 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:04.783012    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6wxdk/must-gather-ml6wr"]
Jul 09 09:59:04.785223 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:59:04.785193    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7efd0569_8298_44c9_bfe0_f6ad3b48fc49.slice/crio-82bec7776285c99925189acd272e3f585cf46d64321d0a22d9505798d0996938 WatchSource:0}: Error finding container 82bec7776285c99925189acd272e3f585cf46d64321d0a22d9505798d0996938: Status 404 returned error can't find the container with id 82bec7776285c99925189acd272e3f585cf46d64321d0a22d9505798d0996938
Jul 09 09:59:04.787053 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:04.787032    2549 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 09:59:05.394732 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:05.394692    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6wxdk/must-gather-ml6wr" event={"ID":"7efd0569-8298-44c9-bfe0-f6ad3b48fc49","Type":"ContainerStarted","Data":"82bec7776285c99925189acd272e3f585cf46d64321d0a22d9505798d0996938"}
Jul 09 09:59:11.426584 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:11.426549    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6wxdk/must-gather-ml6wr" event={"ID":"7efd0569-8298-44c9-bfe0-f6ad3b48fc49","Type":"ContainerStarted","Data":"f3dac8dabfde09e7501530bce54f37ef80acb301f30ee76d6331d6befb327904"}
Jul 09 09:59:11.426986 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:11.426599    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6wxdk/must-gather-ml6wr" event={"ID":"7efd0569-8298-44c9-bfe0-f6ad3b48fc49","Type":"ContainerStarted","Data":"9d6df9639b286575ed8a07dce951ae91a616149ff9aac39a2a90f5a4aa1bf90b"}
Jul 09 09:59:11.447567 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:11.447456    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6wxdk/must-gather-ml6wr" podStartSLOduration=1.5487078539999999 podStartE2EDuration="7.447436698s" totalImagesPullingTime="5.898728844s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:59:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:59:11.446072406 +0000 UTC m=+1509.617230963" watchObservedRunningTime="2026-07-09 09:59:11.447436698 +0000 UTC m=+1509.618595255"
Jul 09 09:59:31.522141 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:31.522113    2549 generic.go:356] "Generic (PLEG): container finished" podID="7efd0569-8298-44c9-bfe0-f6ad3b48fc49" containerID="9d6df9639b286575ed8a07dce951ae91a616149ff9aac39a2a90f5a4aa1bf90b" exitCode=0
Jul 09 09:59:31.522579 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:31.522164    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6wxdk/must-gather-ml6wr" event={"ID":"7efd0569-8298-44c9-bfe0-f6ad3b48fc49","Type":"ContainerDied","Data":"9d6df9639b286575ed8a07dce951ae91a616149ff9aac39a2a90f5a4aa1bf90b"}
Jul 09 09:59:31.522579 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:31.522452    2549 scope.go:117] "RemoveContainer" containerID="9d6df9639b286575ed8a07dce951ae91a616149ff9aac39a2a90f5a4aa1bf90b"
Jul 09 09:59:32.154686 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:32.154660    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-6wxdk_must-gather-ml6wr_7efd0569-8298-44c9-bfe0-f6ad3b48fc49/gather/0.log"
Jul 09 09:59:32.717130 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:32.717099    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8pf7l/must-gather-s8k4t"]
Jul 09 09:59:32.720055 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:32.720033    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8pf7l/must-gather-s8k4t"
Jul 09 09:59:32.724981 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:32.724959    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8pf7l\"/\"kube-root-ca.crt\""
Jul 09 09:59:32.725274 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:32.725250    2549 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8pf7l\"/\"openshift-service-ca.crt\""
Jul 09 09:59:32.727837 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:32.727816    2549 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8pf7l\"/\"default-dockercfg-hpm5h\""
Jul 09 09:59:32.739946 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:32.739926    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8pf7l/must-gather-s8k4t"]
Jul 09 09:59:32.848707 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:32.848681    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6q2f\" (UniqueName: \"kubernetes.io/projected/0a7b0de2-4c5d-4960-84c1-7d72dd555eb0-kube-api-access-l6q2f\") pod \"must-gather-s8k4t\" (UID: \"0a7b0de2-4c5d-4960-84c1-7d72dd555eb0\") " pod="openshift-must-gather-8pf7l/must-gather-s8k4t"
Jul 09 09:59:32.848905 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:32.848728    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0a7b0de2-4c5d-4960-84c1-7d72dd555eb0-must-gather-output\") pod \"must-gather-s8k4t\" (UID: \"0a7b0de2-4c5d-4960-84c1-7d72dd555eb0\") " pod="openshift-must-gather-8pf7l/must-gather-s8k4t"
Jul 09 09:59:32.949784 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:32.949754    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0a7b0de2-4c5d-4960-84c1-7d72dd555eb0-must-gather-output\") pod \"must-gather-s8k4t\" (UID: \"0a7b0de2-4c5d-4960-84c1-7d72dd555eb0\") " pod="openshift-must-gather-8pf7l/must-gather-s8k4t"
Jul 09 09:59:32.949950 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:32.949855    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6q2f\" (UniqueName: \"kubernetes.io/projected/0a7b0de2-4c5d-4960-84c1-7d72dd555eb0-kube-api-access-l6q2f\") pod \"must-gather-s8k4t\" (UID: \"0a7b0de2-4c5d-4960-84c1-7d72dd555eb0\") " pod="openshift-must-gather-8pf7l/must-gather-s8k4t"
Jul 09 09:59:32.950096 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:32.950078    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0a7b0de2-4c5d-4960-84c1-7d72dd555eb0-must-gather-output\") pod \"must-gather-s8k4t\" (UID: \"0a7b0de2-4c5d-4960-84c1-7d72dd555eb0\") " pod="openshift-must-gather-8pf7l/must-gather-s8k4t"
Jul 09 09:59:32.958785 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:32.958767    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6q2f\" (UniqueName: \"kubernetes.io/projected/0a7b0de2-4c5d-4960-84c1-7d72dd555eb0-kube-api-access-l6q2f\") pod \"must-gather-s8k4t\" (UID: \"0a7b0de2-4c5d-4960-84c1-7d72dd555eb0\") " pod="openshift-must-gather-8pf7l/must-gather-s8k4t"
Jul 09 09:59:33.029807 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:33.029785    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8pf7l/must-gather-s8k4t"
Jul 09 09:59:33.162984 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:33.162936    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8pf7l/must-gather-s8k4t"]
Jul 09 09:59:33.165849 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:59:33.165822    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a7b0de2_4c5d_4960_84c1_7d72dd555eb0.slice/crio-46d63a52cbea358529a2592e2f2b336f97e4c9087dc932dd31c9e87cefdc8c7d WatchSource:0}: Error finding container 46d63a52cbea358529a2592e2f2b336f97e4c9087dc932dd31c9e87cefdc8c7d: Status 404 returned error can't find the container with id 46d63a52cbea358529a2592e2f2b336f97e4c9087dc932dd31c9e87cefdc8c7d
Jul 09 09:59:33.531514 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:33.531483    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8pf7l/must-gather-s8k4t" event={"ID":"0a7b0de2-4c5d-4960-84c1-7d72dd555eb0","Type":"ContainerStarted","Data":"46d63a52cbea358529a2592e2f2b336f97e4c9087dc932dd31c9e87cefdc8c7d"}
Jul 09 09:59:34.540039 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:34.539994    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8pf7l/must-gather-s8k4t" event={"ID":"0a7b0de2-4c5d-4960-84c1-7d72dd555eb0","Type":"ContainerStarted","Data":"8aa3a61f12a8f0649066d91737c5217cda853c576f23c14a50f10291df4afc33"}
Jul 09 09:59:34.540039 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:34.540042    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8pf7l/must-gather-s8k4t" event={"ID":"0a7b0de2-4c5d-4960-84c1-7d72dd555eb0","Type":"ContainerStarted","Data":"42be5cd20fa5c7e67fd3208091837c42d434ef0547d8e54d07826a5b9625b575"}
Jul 09 09:59:34.564696 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:34.564636    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8pf7l/must-gather-s8k4t" podStartSLOduration=1.9362723979999998 podStartE2EDuration="2.56458582s" totalImagesPullingTime="628.313422ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:59:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:59:34.563384727 +0000 UTC m=+1532.734543283" watchObservedRunningTime="2026-07-09 09:59:34.56458582 +0000 UTC m=+1532.735744378"
Jul 09 09:59:35.280128 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:35.280096    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ccqx8_282fa84e-5b6f-4cb1-b6bf-f50d9ad479ee/global-pull-secret-syncer/0.log"
Jul 09 09:59:35.495698 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:35.495668    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-c4jnr_f4defe16-3199-4cea-8b4e-88375572edbf/konnectivity-agent/0.log"
Jul 09 09:59:35.615903 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:35.615817    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-199.ec2.internal_d0bea77c1d44c6a069aa75a3ce77a6d6/haproxy/0.log"
Jul 09 09:59:37.537057 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:37.537026    2549 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-6wxdk/must-gather-ml6wr"]
Jul 09 09:59:37.538277 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:37.537813    2549 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-must-gather-6wxdk/must-gather-ml6wr" podUID="7efd0569-8298-44c9-bfe0-f6ad3b48fc49" containerName="copy" containerID="cri-o://f3dac8dabfde09e7501530bce54f37ef80acb301f30ee76d6331d6befb327904" gracePeriod=2
Jul 09 09:59:37.539473 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:37.539449    2549 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-6wxdk/must-gather-ml6wr"]
Jul 09 09:59:37.542175 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:59:37.542150    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-ml6wr\" is forbidden: User \"system:node:ip-10-0-137-199.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-6wxdk\": no relationship found between node 'ip-10-0-137-199.ec2.internal' and this object" podUID="7efd0569-8298-44c9-bfe0-f6ad3b48fc49" pod="openshift-must-gather-6wxdk/must-gather-ml6wr"
Jul 09 09:59:37.920531 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:37.920437    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-6wxdk_must-gather-ml6wr_7efd0569-8298-44c9-bfe0-f6ad3b48fc49/copy/0.log"
Jul 09 09:59:37.925855 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:37.925618    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6wxdk/must-gather-ml6wr"
Jul 09 09:59:37.927939 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:59:37.927898    2549 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-ml6wr\" is forbidden: User \"system:node:ip-10-0-137-199.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-6wxdk\": no relationship found between node 'ip-10-0-137-199.ec2.internal' and this object" podUID="7efd0569-8298-44c9-bfe0-f6ad3b48fc49" pod="openshift-must-gather-6wxdk/must-gather-ml6wr"
Jul 09 09:59:38.012198 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:38.010279    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmzft\" (UniqueName: \"kubernetes.io/projected/7efd0569-8298-44c9-bfe0-f6ad3b48fc49-kube-api-access-kmzft\") pod \"7efd0569-8298-44c9-bfe0-f6ad3b48fc49\" (UID: \"7efd0569-8298-44c9-bfe0-f6ad3b48fc49\") "
Jul 09 09:59:38.012198 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:38.010436    2549 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7efd0569-8298-44c9-bfe0-f6ad3b48fc49-must-gather-output\") pod \"7efd0569-8298-44c9-bfe0-f6ad3b48fc49\" (UID: \"7efd0569-8298-44c9-bfe0-f6ad3b48fc49\") "
Jul 09 09:59:38.012198 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:38.011858    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7efd0569-8298-44c9-bfe0-f6ad3b48fc49-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "7efd0569-8298-44c9-bfe0-f6ad3b48fc49" (UID: "7efd0569-8298-44c9-bfe0-f6ad3b48fc49"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 09:59:38.020399 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:38.020328    2549 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7efd0569-8298-44c9-bfe0-f6ad3b48fc49-kube-api-access-kmzft" (OuterVolumeSpecName: "kube-api-access-kmzft") pod "7efd0569-8298-44c9-bfe0-f6ad3b48fc49" (UID: "7efd0569-8298-44c9-bfe0-f6ad3b48fc49"). InnerVolumeSpecName "kube-api-access-kmzft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 09:59:38.111298 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:38.111262    2549 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7efd0569-8298-44c9-bfe0-f6ad3b48fc49-must-gather-output\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:59:38.111520 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:38.111504    2549 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmzft\" (UniqueName: \"kubernetes.io/projected/7efd0569-8298-44c9-bfe0-f6ad3b48fc49-kube-api-access-kmzft\") on node \"ip-10-0-137-199.ec2.internal\" DevicePath \"\""
Jul 09 09:59:38.389341 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:38.389255    2549 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7efd0569-8298-44c9-bfe0-f6ad3b48fc49" path="/var/lib/kubelet/pods/7efd0569-8298-44c9-bfe0-f6ad3b48fc49/volumes"
Jul 09 09:59:38.604359 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:38.604273    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-6wxdk_must-gather-ml6wr_7efd0569-8298-44c9-bfe0-f6ad3b48fc49/copy/0.log"
Jul 09 09:59:38.605382 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:38.605354    2549 generic.go:356] "Generic (PLEG): container finished" podID="7efd0569-8298-44c9-bfe0-f6ad3b48fc49" containerID="f3dac8dabfde09e7501530bce54f37ef80acb301f30ee76d6331d6befb327904" exitCode=143
Jul 09 09:59:38.605629 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:38.605615    2549 scope.go:117] "RemoveContainer" containerID="f3dac8dabfde09e7501530bce54f37ef80acb301f30ee76d6331d6befb327904"
Jul 09 09:59:38.605928 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:38.605911    2549 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6wxdk/must-gather-ml6wr"
Jul 09 09:59:38.622230 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:38.622208    2549 scope.go:117] "RemoveContainer" containerID="9d6df9639b286575ed8a07dce951ae91a616149ff9aac39a2a90f5a4aa1bf90b"
Jul 09 09:59:38.658266 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:38.658235    2549 scope.go:117] "RemoveContainer" containerID="f3dac8dabfde09e7501530bce54f37ef80acb301f30ee76d6331d6befb327904"
Jul 09 09:59:38.658906 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:59:38.658882    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3dac8dabfde09e7501530bce54f37ef80acb301f30ee76d6331d6befb327904\": container with ID starting with f3dac8dabfde09e7501530bce54f37ef80acb301f30ee76d6331d6befb327904 not found: ID does not exist" containerID="f3dac8dabfde09e7501530bce54f37ef80acb301f30ee76d6331d6befb327904"
Jul 09 09:59:38.659084 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:38.659055    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3dac8dabfde09e7501530bce54f37ef80acb301f30ee76d6331d6befb327904"} err="failed to get container status \"f3dac8dabfde09e7501530bce54f37ef80acb301f30ee76d6331d6befb327904\": rpc error: code = NotFound desc = could not find container \"f3dac8dabfde09e7501530bce54f37ef80acb301f30ee76d6331d6befb327904\": container with ID starting with f3dac8dabfde09e7501530bce54f37ef80acb301f30ee76d6331d6befb327904 not found: ID does not exist"
Jul 09 09:59:38.659213 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:38.659200    2549 scope.go:117] "RemoveContainer" containerID="9d6df9639b286575ed8a07dce951ae91a616149ff9aac39a2a90f5a4aa1bf90b"
Jul 09 09:59:38.659652 ip-10-0-137-199 kubenswrapper[2549]: E0709 09:59:38.659631    2549 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d6df9639b286575ed8a07dce951ae91a616149ff9aac39a2a90f5a4aa1bf90b\": container with ID starting with 9d6df9639b286575ed8a07dce951ae91a616149ff9aac39a2a90f5a4aa1bf90b not found: ID does not exist" containerID="9d6df9639b286575ed8a07dce951ae91a616149ff9aac39a2a90f5a4aa1bf90b"
Jul 09 09:59:38.659803 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:38.659782    2549 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d6df9639b286575ed8a07dce951ae91a616149ff9aac39a2a90f5a4aa1bf90b"} err="failed to get container status \"9d6df9639b286575ed8a07dce951ae91a616149ff9aac39a2a90f5a4aa1bf90b\": rpc error: code = NotFound desc = could not find container \"9d6df9639b286575ed8a07dce951ae91a616149ff9aac39a2a90f5a4aa1bf90b\": container with ID starting with 9d6df9639b286575ed8a07dce951ae91a616149ff9aac39a2a90f5a4aa1bf90b not found: ID does not exist"
Jul 09 09:59:39.417189 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:39.417160    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_12a35c5d-dbfa-4acd-bdc0-f651e73ab391/alertmanager/0.log"
Jul 09 09:59:39.452452 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:39.452386    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_12a35c5d-dbfa-4acd-bdc0-f651e73ab391/config-reloader/0.log"
Jul 09 09:59:39.490761 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:39.490723    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_12a35c5d-dbfa-4acd-bdc0-f651e73ab391/kube-rbac-proxy-web/0.log"
Jul 09 09:59:39.525900 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:39.525553    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_12a35c5d-dbfa-4acd-bdc0-f651e73ab391/kube-rbac-proxy/0.log"
Jul 09 09:59:39.555550 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:39.555501    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_12a35c5d-dbfa-4acd-bdc0-f651e73ab391/kube-rbac-proxy-metric/0.log"
Jul 09 09:59:39.584041 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:39.584008    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_12a35c5d-dbfa-4acd-bdc0-f651e73ab391/prom-label-proxy/0.log"
Jul 09 09:59:39.691565 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:39.691446    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_12a35c5d-dbfa-4acd-bdc0-f651e73ab391/init-config-reloader/0.log"
Jul 09 09:59:39.815995 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:39.815936    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-57lfj_37e2f403-c4b2-42d0-8613-8755b43545b8/kube-state-metrics/0.log"
Jul 09 09:59:39.846179 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:39.846149    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-57lfj_37e2f403-c4b2-42d0-8613-8755b43545b8/kube-rbac-proxy-main/0.log"
Jul 09 09:59:39.879896 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:39.879847    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-57lfj_37e2f403-c4b2-42d0-8613-8755b43545b8/kube-rbac-proxy-self/0.log"
Jul 09 09:59:39.920681 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:39.920642    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6485c4445-bcqjm_4bcbdffd-45cc-4293-a061-a8d0785819b4/metrics-server/0.log"
Jul 09 09:59:39.960841 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:39.960805    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-dxx4z_fa21ce94-f749-4993-9894-dfd2a9988132/monitoring-plugin/0.log"
Jul 09 09:59:40.222920 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:40.222770    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tv629_25727dba-8eab-4eb5-85e2-6afa43578f4a/node-exporter/0.log"
Jul 09 09:59:40.254658 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:40.254630    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tv629_25727dba-8eab-4eb5-85e2-6afa43578f4a/kube-rbac-proxy/0.log"
Jul 09 09:59:40.283877 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:40.283751    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tv629_25727dba-8eab-4eb5-85e2-6afa43578f4a/init-textfile/0.log"
Jul 09 09:59:40.318134 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:40.318110    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-pprmx_ac99270f-01d2-4095-b9f6-84129e22918e/kube-rbac-proxy-main/0.log"
Jul 09 09:59:40.349676 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:40.349606    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-pprmx_ac99270f-01d2-4095-b9f6-84129e22918e/kube-rbac-proxy-self/0.log"
Jul 09 09:59:40.378090 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:40.378057    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-pprmx_ac99270f-01d2-4095-b9f6-84129e22918e/openshift-state-metrics/0.log"
Jul 09 09:59:40.418786 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:40.418760    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a57c8eee-2c09-4e5d-aa20-4894697ed0b8/prometheus/0.log"
Jul 09 09:59:40.441814 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:40.441784    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a57c8eee-2c09-4e5d-aa20-4894697ed0b8/config-reloader/0.log"
Jul 09 09:59:40.477703 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:40.477621    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a57c8eee-2c09-4e5d-aa20-4894697ed0b8/thanos-sidecar/0.log"
Jul 09 09:59:40.505141 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:40.505105    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a57c8eee-2c09-4e5d-aa20-4894697ed0b8/kube-rbac-proxy-web/0.log"
Jul 09 09:59:40.530264 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:40.530232    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a57c8eee-2c09-4e5d-aa20-4894697ed0b8/kube-rbac-proxy/0.log"
Jul 09 09:59:40.557852 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:40.557825    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a57c8eee-2c09-4e5d-aa20-4894697ed0b8/kube-rbac-proxy-thanos/0.log"
Jul 09 09:59:40.585703 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:40.585658    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a57c8eee-2c09-4e5d-aa20-4894697ed0b8/init-config-reloader/0.log"
Jul 09 09:59:40.621675 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:40.621635    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-8vkwv_94a5d665-56aa-49cf-8cf6-c9ad52a2227e/prometheus-operator/0.log"
Jul 09 09:59:40.647187 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:40.647158    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-8vkwv_94a5d665-56aa-49cf-8cf6-c9ad52a2227e/kube-rbac-proxy/0.log"
Jul 09 09:59:40.676194 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:40.676160    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-566fz_e4dd43e4-2a1a-4232-9441-d19b5a5f76b6/prometheus-operator-admission-webhook/0.log"
Jul 09 09:59:40.744379 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:40.744336    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-77f97ff54d-d5rnf_cf4837df-9a6d-40ee-9c81-c22cee47a344/telemeter-client/0.log"
Jul 09 09:59:40.777981 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:40.777935    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-77f97ff54d-d5rnf_cf4837df-9a6d-40ee-9c81-c22cee47a344/reload/0.log"
Jul 09 09:59:40.809583 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:40.809551    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-77f97ff54d-d5rnf_cf4837df-9a6d-40ee-9c81-c22cee47a344/kube-rbac-proxy/0.log"
Jul 09 09:59:40.848046 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:40.848012    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dc479b7c8-ln75k_cf8a07e1-0685-4dcc-b0f9-f3228e2438c1/thanos-query/0.log"
Jul 09 09:59:40.885227 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:40.885146    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dc479b7c8-ln75k_cf8a07e1-0685-4dcc-b0f9-f3228e2438c1/kube-rbac-proxy-web/0.log"
Jul 09 09:59:40.928161 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:40.928124    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dc479b7c8-ln75k_cf8a07e1-0685-4dcc-b0f9-f3228e2438c1/kube-rbac-proxy/0.log"
Jul 09 09:59:40.972239 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:40.972213    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dc479b7c8-ln75k_cf8a07e1-0685-4dcc-b0f9-f3228e2438c1/prom-label-proxy/0.log"
Jul 09 09:59:41.005144 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:41.005115    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dc479b7c8-ln75k_cf8a07e1-0685-4dcc-b0f9-f3228e2438c1/kube-rbac-proxy-rules/0.log"
Jul 09 09:59:41.039664 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:41.039638    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5dc479b7c8-ln75k_cf8a07e1-0685-4dcc-b0f9-f3228e2438c1/kube-rbac-proxy-metrics/0.log"
Jul 09 09:59:43.370148 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:43.370115    2549 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8pf7l/perf-node-gather-daemonset-7qwph"]
Jul 09 09:59:43.370733 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:43.370711    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7efd0569-8298-44c9-bfe0-f6ad3b48fc49" containerName="copy"
Jul 09 09:59:43.370733 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:43.370735    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="7efd0569-8298-44c9-bfe0-f6ad3b48fc49" containerName="copy"
Jul 09 09:59:43.370900 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:43.370779    2549 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7efd0569-8298-44c9-bfe0-f6ad3b48fc49" containerName="gather"
Jul 09 09:59:43.370900 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:43.370789    2549 state_mem.go:107] "Deleted CPUSet assignment" podUID="7efd0569-8298-44c9-bfe0-f6ad3b48fc49" containerName="gather"
Jul 09 09:59:43.370900 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:43.370894    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="7efd0569-8298-44c9-bfe0-f6ad3b48fc49" containerName="gather"
Jul 09 09:59:43.371066 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:43.370911    2549 memory_manager.go:368] "RemoveStaleState removing state" podUID="7efd0569-8298-44c9-bfe0-f6ad3b48fc49" containerName="copy"
Jul 09 09:59:43.374439 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:43.374418    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8pf7l/perf-node-gather-daemonset-7qwph"
Jul 09 09:59:43.390058 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:43.390034    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8pf7l/perf-node-gather-daemonset-7qwph"]
Jul 09 09:59:43.400791 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:43.400768    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-8bcd4cfd5-7szgp_e57c6981-978d-44fc-9f55-c20077bce0bc/console/0.log"
Jul 09 09:59:43.461567 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:43.461533    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j68f\" (UniqueName: \"kubernetes.io/projected/81b0d459-7ce8-4566-b353-6ec4f680b64d-kube-api-access-9j68f\") pod \"perf-node-gather-daemonset-7qwph\" (UID: \"81b0d459-7ce8-4566-b353-6ec4f680b64d\") " pod="openshift-must-gather-8pf7l/perf-node-gather-daemonset-7qwph"
Jul 09 09:59:43.461567 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:43.461583    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/81b0d459-7ce8-4566-b353-6ec4f680b64d-podres\") pod \"perf-node-gather-daemonset-7qwph\" (UID: \"81b0d459-7ce8-4566-b353-6ec4f680b64d\") " pod="openshift-must-gather-8pf7l/perf-node-gather-daemonset-7qwph"
Jul 09 09:59:43.461825 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:43.461683    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81b0d459-7ce8-4566-b353-6ec4f680b64d-sys\") pod \"perf-node-gather-daemonset-7qwph\" (UID: \"81b0d459-7ce8-4566-b353-6ec4f680b64d\") " pod="openshift-must-gather-8pf7l/perf-node-gather-daemonset-7qwph"
Jul 09 09:59:43.461825 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:43.461716    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81b0d459-7ce8-4566-b353-6ec4f680b64d-lib-modules\") pod \"perf-node-gather-daemonset-7qwph\" (UID: \"81b0d459-7ce8-4566-b353-6ec4f680b64d\") " pod="openshift-must-gather-8pf7l/perf-node-gather-daemonset-7qwph"
Jul 09 09:59:43.461825 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:43.461757    2549 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/81b0d459-7ce8-4566-b353-6ec4f680b64d-proc\") pod \"perf-node-gather-daemonset-7qwph\" (UID: \"81b0d459-7ce8-4566-b353-6ec4f680b64d\") " pod="openshift-must-gather-8pf7l/perf-node-gather-daemonset-7qwph"
Jul 09 09:59:43.563197 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:43.563152    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/81b0d459-7ce8-4566-b353-6ec4f680b64d-proc\") pod \"perf-node-gather-daemonset-7qwph\" (UID: \"81b0d459-7ce8-4566-b353-6ec4f680b64d\") " pod="openshift-must-gather-8pf7l/perf-node-gather-daemonset-7qwph"
Jul 09 09:59:43.563377 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:43.563204    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j68f\" (UniqueName: \"kubernetes.io/projected/81b0d459-7ce8-4566-b353-6ec4f680b64d-kube-api-access-9j68f\") pod \"perf-node-gather-daemonset-7qwph\" (UID: \"81b0d459-7ce8-4566-b353-6ec4f680b64d\") " pod="openshift-must-gather-8pf7l/perf-node-gather-daemonset-7qwph"
Jul 09 09:59:43.563377 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:43.563262    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/81b0d459-7ce8-4566-b353-6ec4f680b64d-podres\") pod \"perf-node-gather-daemonset-7qwph\" (UID: \"81b0d459-7ce8-4566-b353-6ec4f680b64d\") " pod="openshift-must-gather-8pf7l/perf-node-gather-daemonset-7qwph"
Jul 09 09:59:43.563377 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:43.563265    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/81b0d459-7ce8-4566-b353-6ec4f680b64d-proc\") pod \"perf-node-gather-daemonset-7qwph\" (UID: \"81b0d459-7ce8-4566-b353-6ec4f680b64d\") " pod="openshift-must-gather-8pf7l/perf-node-gather-daemonset-7qwph"
Jul 09 09:59:43.563377 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:43.563359    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81b0d459-7ce8-4566-b353-6ec4f680b64d-sys\") pod \"perf-node-gather-daemonset-7qwph\" (UID: \"81b0d459-7ce8-4566-b353-6ec4f680b64d\") " pod="openshift-must-gather-8pf7l/perf-node-gather-daemonset-7qwph"
Jul 09 09:59:43.563568 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:43.563385    2549 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81b0d459-7ce8-4566-b353-6ec4f680b64d-lib-modules\") pod \"perf-node-gather-daemonset-7qwph\" (UID: \"81b0d459-7ce8-4566-b353-6ec4f680b64d\") " pod="openshift-must-gather-8pf7l/perf-node-gather-daemonset-7qwph"
Jul 09 09:59:43.563568 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:43.563391    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/81b0d459-7ce8-4566-b353-6ec4f680b64d-podres\") pod \"perf-node-gather-daemonset-7qwph\" (UID: \"81b0d459-7ce8-4566-b353-6ec4f680b64d\") " pod="openshift-must-gather-8pf7l/perf-node-gather-daemonset-7qwph"
Jul 09 09:59:43.563568 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:43.563513    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81b0d459-7ce8-4566-b353-6ec4f680b64d-lib-modules\") pod \"perf-node-gather-daemonset-7qwph\" (UID: \"81b0d459-7ce8-4566-b353-6ec4f680b64d\") " pod="openshift-must-gather-8pf7l/perf-node-gather-daemonset-7qwph"
Jul 09 09:59:43.563568 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:43.563516    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81b0d459-7ce8-4566-b353-6ec4f680b64d-sys\") pod \"perf-node-gather-daemonset-7qwph\" (UID: \"81b0d459-7ce8-4566-b353-6ec4f680b64d\") " pod="openshift-must-gather-8pf7l/perf-node-gather-daemonset-7qwph"
Jul 09 09:59:43.572826 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:43.572796    2549 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j68f\" (UniqueName: \"kubernetes.io/projected/81b0d459-7ce8-4566-b353-6ec4f680b64d-kube-api-access-9j68f\") pod \"perf-node-gather-daemonset-7qwph\" (UID: \"81b0d459-7ce8-4566-b353-6ec4f680b64d\") " pod="openshift-must-gather-8pf7l/perf-node-gather-daemonset-7qwph"
Jul 09 09:59:43.684990 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:43.684896    2549 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8pf7l/perf-node-gather-daemonset-7qwph"
Jul 09 09:59:43.845911 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:43.845874    2549 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8pf7l/perf-node-gather-daemonset-7qwph"]
Jul 09 09:59:43.850691 ip-10-0-137-199 kubenswrapper[2549]: W0709 09:59:43.850664    2549 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod81b0d459_7ce8_4566_b353_6ec4f680b64d.slice/crio-5217a627018309943d84c799350a2e5d760c614237d5eedbb972058f68284903 WatchSource:0}: Error finding container 5217a627018309943d84c799350a2e5d760c614237d5eedbb972058f68284903: Status 404 returned error can't find the container with id 5217a627018309943d84c799350a2e5d760c614237d5eedbb972058f68284903
Jul 09 09:59:44.647018 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:44.646978    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8pf7l/perf-node-gather-daemonset-7qwph" event={"ID":"81b0d459-7ce8-4566-b353-6ec4f680b64d","Type":"ContainerStarted","Data":"fa4fd277a810270f26a956f5aacaa49b38f5d527ce0edc4d11d869e1282fddbe"}
Jul 09 09:59:44.647501 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:44.647044    2549 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8pf7l/perf-node-gather-daemonset-7qwph" event={"ID":"81b0d459-7ce8-4566-b353-6ec4f680b64d","Type":"ContainerStarted","Data":"5217a627018309943d84c799350a2e5d760c614237d5eedbb972058f68284903"}
Jul 09 09:59:44.647501 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:44.647144    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8pf7l/perf-node-gather-daemonset-7qwph"
Jul 09 09:59:44.672047 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:44.672000    2549 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8pf7l/perf-node-gather-daemonset-7qwph" podStartSLOduration=1.671985045 podStartE2EDuration="1.671985045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:59:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:59:44.671132809 +0000 UTC m=+1542.842291366" watchObservedRunningTime="2026-07-09 09:59:44.671985045 +0000 UTC m=+1542.843143601"
Jul 09 09:59:45.145005 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:45.144980    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-scljx_685afdc3-9a1c-4ec4-85f2-1cefd4b7c2e6/dns/0.log"
Jul 09 09:59:45.170663 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:45.170634    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-scljx_685afdc3-9a1c-4ec4-85f2-1cefd4b7c2e6/kube-rbac-proxy/0.log"
Jul 09 09:59:45.224130 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:45.224097    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8xp94_83ccd32a-ea2e-4ad9-95c7-f79c33043432/dns-node-resolver/0.log"
Jul 09 09:59:45.862940 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:45.862913    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nz6rq_2e407356-17fe-4a9f-81da-417042050209/node-ca/0.log"
Jul 09 09:59:47.067639 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:47.067611    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-67wzn_a04fe9a6-e137-44cc-b31b-8dddd4e3a1f3/serve-healthcheck-canary/0.log"
Jul 09 09:59:47.624373 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:47.624345    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fvmrl_fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3/kube-rbac-proxy/0.log"
Jul 09 09:59:47.649799 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:47.649777    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fvmrl_fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3/exporter/0.log"
Jul 09 09:59:47.672002 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:47.671975    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fvmrl_fbc5cbd7-ae05-4a1b-a0cc-7d5a4be5c1d3/extractor/0.log"
Jul 09 09:59:50.139619 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:50.139586    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-dmpbt_2e836b72-ca73-496d-a127-2631e76ed06e/s3-init/0.log"
Jul 09 09:59:50.188474 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:50.188447    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-dmpbt_2e836b72-ca73-496d-a127-2631e76ed06e/download-hf-model/0.log"
Jul 09 09:59:50.275610 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:50.275585    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-tls-init-custom-fqtpr_3defb281-5ccb-49ea-9f28-9a16562662cc/s3-tls-init-custom/0.log"
Jul 09 09:59:50.324440 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:50.324415    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-tls-init-custom-fqtpr_3defb281-5ccb-49ea-9f28-9a16562662cc/download-hf-model/0.log"
Jul 09 09:59:50.378434 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:50.378412    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-tls-init-serving-wzg5j_02e38491-5d3e-4457-9df1-eda978e82010/s3-tls-init-serving/0.log"
Jul 09 09:59:50.416377 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:50.416317    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-tls-init-serving-wzg5j_02e38491-5d3e-4457-9df1-eda978e82010/download-hf-model/0.log"
Jul 09 09:59:50.507566 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:50.507539    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-tls-custom-8b44d767d-79m62_f5106f13-399a-4cc9-aaa3-dda3c3a2dda2/seaweedfs-tls-custom/0.log"
Jul 09 09:59:50.547902 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:50.547860    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-tls-serving-67b7477c6d-vfc9f_7d6bd83e-4062-43b0-a9ef-94c2d72d31dc/seaweedfs-tls-serving/0.log"
Jul 09 09:59:50.663235 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:50.663207    2549 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8pf7l/perf-node-gather-daemonset-7qwph"
Jul 09 09:59:57.433640 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:57.433615    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9h8vf_eae6a875-8c48-46ae-bac2-cbf1b0cfc34f/kube-multus-additional-cni-plugins/0.log"
Jul 09 09:59:57.482166 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:57.482144    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9h8vf_eae6a875-8c48-46ae-bac2-cbf1b0cfc34f/egress-router-binary-copy/0.log"
Jul 09 09:59:57.518274 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:57.518248    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9h8vf_eae6a875-8c48-46ae-bac2-cbf1b0cfc34f/cni-plugins/0.log"
Jul 09 09:59:57.549370 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:57.549347    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9h8vf_eae6a875-8c48-46ae-bac2-cbf1b0cfc34f/bond-cni-plugin/0.log"
Jul 09 09:59:57.580421 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:57.580399    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9h8vf_eae6a875-8c48-46ae-bac2-cbf1b0cfc34f/routeoverride-cni/0.log"
Jul 09 09:59:57.612892 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:57.612853    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9h8vf_eae6a875-8c48-46ae-bac2-cbf1b0cfc34f/whereabouts-cni-bincopy/0.log"
Jul 09 09:59:57.641545 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:57.641523    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9h8vf_eae6a875-8c48-46ae-bac2-cbf1b0cfc34f/whereabouts-cni/0.log"
Jul 09 09:59:57.896900 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:57.896855    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mfcn7_79826bec-2b69-42e5-87f1-94c078b40ac4/kube-multus/0.log"
Jul 09 09:59:58.105244 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:58.105219    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rqzsx_72ab801d-5d43-4fbc-a26a-60fab96be57b/network-metrics-daemon/0.log"
Jul 09 09:59:58.134220 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:58.134191    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rqzsx_72ab801d-5d43-4fbc-a26a-60fab96be57b/kube-rbac-proxy/0.log"
Jul 09 09:59:59.025584 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:59.025548    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j2nr7_35942e40-0ffc-472e-bc29-be3077dc61d5/ovn-controller/0.log"
Jul 09 09:59:59.054027 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:59.054001    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j2nr7_35942e40-0ffc-472e-bc29-be3077dc61d5/ovn-acl-logging/0.log"
Jul 09 09:59:59.087815 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:59.087784    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j2nr7_35942e40-0ffc-472e-bc29-be3077dc61d5/kube-rbac-proxy-node/0.log"
Jul 09 09:59:59.153682 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:59.153656    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j2nr7_35942e40-0ffc-472e-bc29-be3077dc61d5/kube-rbac-proxy-ovn-metrics/0.log"
Jul 09 09:59:59.259414 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:59.259388    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j2nr7_35942e40-0ffc-472e-bc29-be3077dc61d5/northd/0.log"
Jul 09 09:59:59.348973 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:59.348902    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j2nr7_35942e40-0ffc-472e-bc29-be3077dc61d5/nbdb/0.log"
Jul 09 09:59:59.390329 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:59.390308    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j2nr7_35942e40-0ffc-472e-bc29-be3077dc61d5/sbdb/0.log"
Jul 09 09:59:59.475238 ip-10-0-137-199 kubenswrapper[2549]: I0709 09:59:59.475209    2549 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j2nr7_35942e40-0ffc-472e-bc29-be3077dc61d5/ovnkube-controller/0.log"