Jul 03 14:08:04.668149 ip-10-0-142-233 systemd[1]: Starting Kubernetes Kubelet...
Jul 03 14:08:05.090711 ip-10-0-142-233 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 03 14:08:05.090711 ip-10-0-142-233 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 03 14:08:05.090711 ip-10-0-142-233 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 03 14:08:05.090711 ip-10-0-142-233 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 03 14:08:05.090711 ip-10-0-142-233 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 03 14:08:05.091423 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.090771    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 03 14:08:05.097451 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097428    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 03 14:08:05.097451 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097447    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 03 14:08:05.097451 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097453    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 03 14:08:05.097451 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097457    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 03 14:08:05.097625 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097460    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 03 14:08:05.097625 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097464    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 03 14:08:05.097625 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097467    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 03 14:08:05.097625 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097470    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 03 14:08:05.097625 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097473    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 03 14:08:05.097625 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097476    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 03 14:08:05.097625 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097478    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 03 14:08:05.097625 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097481    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 03 14:08:05.097625 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097484    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 03 14:08:05.097625 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097487    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 03 14:08:05.097625 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097489    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 03 14:08:05.097625 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097491    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 03 14:08:05.097625 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097494    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 03 14:08:05.097625 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097496    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 03 14:08:05.097625 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097499    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 03 14:08:05.097625 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097501    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 03 14:08:05.097625 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097504    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 03 14:08:05.097625 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097507    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 03 14:08:05.097625 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097510    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 03 14:08:05.097625 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097512    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 03 14:08:05.098123 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097515    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 03 14:08:05.098123 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097522    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 03 14:08:05.098123 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097524    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 03 14:08:05.098123 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097527    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 03 14:08:05.098123 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097530    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 03 14:08:05.098123 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097534    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 03 14:08:05.098123 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097538    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 03 14:08:05.098123 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097542    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 03 14:08:05.098123 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097562    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 03 14:08:05.098123 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097566    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 03 14:08:05.098123 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097570    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 03 14:08:05.098123 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097573    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 03 14:08:05.098123 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097576    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 03 14:08:05.098123 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097580    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 03 14:08:05.098123 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097583    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 03 14:08:05.098123 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097586    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 03 14:08:05.098123 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097589    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 03 14:08:05.098123 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097592    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 03 14:08:05.098123 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097596    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 03 14:08:05.098634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097599    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 03 14:08:05.098634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097602    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 03 14:08:05.098634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097604    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 03 14:08:05.098634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097607    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 03 14:08:05.098634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097610    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 03 14:08:05.098634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097613    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 03 14:08:05.098634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097615    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 03 14:08:05.098634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097619    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 03 14:08:05.098634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097621    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 03 14:08:05.098634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097624    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 03 14:08:05.098634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097627    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 03 14:08:05.098634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097630    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 03 14:08:05.098634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097632    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 03 14:08:05.098634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097635    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 03 14:08:05.098634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097640    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 03 14:08:05.098634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097643    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 03 14:08:05.098634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097646    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 03 14:08:05.098634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097649    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 03 14:08:05.098634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097652    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 03 14:08:05.099108 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097654    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 03 14:08:05.099108 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097657    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 03 14:08:05.099108 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097660    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 03 14:08:05.099108 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097662    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 03 14:08:05.099108 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097665    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 03 14:08:05.099108 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097668    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 03 14:08:05.099108 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097671    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 03 14:08:05.099108 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097674    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 03 14:08:05.099108 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097677    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 03 14:08:05.099108 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097680    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 03 14:08:05.099108 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097683    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 03 14:08:05.099108 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097685    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 03 14:08:05.099108 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097687    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 03 14:08:05.099108 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097690    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 03 14:08:05.099108 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097693    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 03 14:08:05.099108 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097695    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 03 14:08:05.099108 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097698    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 03 14:08:05.099108 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097701    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 03 14:08:05.099108 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097703    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 03 14:08:05.099108 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097706    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 03 14:08:05.099618 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097708    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 03 14:08:05.099618 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097711    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 03 14:08:05.099618 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097714    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 03 14:08:05.099618 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097717    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 03 14:08:05.099618 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097719    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 03 14:08:05.099618 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097722    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 03 14:08:05.099618 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097724    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 03 14:08:05.099618 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097726    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 03 14:08:05.099618 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097729    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 03 14:08:05.099618 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097731    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 03 14:08:05.099618 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097734    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 03 14:08:05.099618 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097736    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 03 14:08:05.099618 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097739    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 03 14:08:05.099618 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097741    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 03 14:08:05.099618 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097744    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 03 14:08:05.099618 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097747    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 03 14:08:05.099618 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.097749    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 03 14:08:05.099618 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099436    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 03 14:08:05.099618 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099445    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 03 14:08:05.099618 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099449    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 03 14:08:05.100125 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099452    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 03 14:08:05.100125 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099455    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 03 14:08:05.100125 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099458    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 03 14:08:05.100125 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099461    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 03 14:08:05.100125 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099463    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 03 14:08:05.100125 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099466    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 03 14:08:05.100125 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099468    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 03 14:08:05.100125 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099471    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 03 14:08:05.100125 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099473    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 03 14:08:05.100125 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099476    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 03 14:08:05.100125 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099478    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 03 14:08:05.100125 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099481    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 03 14:08:05.100125 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099483    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 03 14:08:05.100125 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099487    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 03 14:08:05.100125 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099491    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 03 14:08:05.100125 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099493    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 03 14:08:05.100125 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099496    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 03 14:08:05.100125 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099499    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 03 14:08:05.100125 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099501    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 03 14:08:05.100125 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099504    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 03 14:08:05.100657 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099507    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 03 14:08:05.100657 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099509    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 03 14:08:05.100657 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099512    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 03 14:08:05.100657 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099515    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 03 14:08:05.100657 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099517    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 03 14:08:05.100657 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099520    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 03 14:08:05.100657 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099523    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 03 14:08:05.100657 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099526    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 03 14:08:05.100657 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099529    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 03 14:08:05.100657 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099532    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 03 14:08:05.100657 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099534    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 03 14:08:05.100657 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099537    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 03 14:08:05.100657 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099540    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 03 14:08:05.100657 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099542    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 03 14:08:05.100657 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099545    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 03 14:08:05.100657 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099547    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 03 14:08:05.100657 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099550    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 03 14:08:05.100657 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099552    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 03 14:08:05.100657 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099555    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 03 14:08:05.100657 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099557    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 03 14:08:05.101163 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099560    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 03 14:08:05.101163 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099562    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 03 14:08:05.101163 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099565    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 03 14:08:05.101163 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099567    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 03 14:08:05.101163 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099569    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 03 14:08:05.101163 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099573    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 03 14:08:05.101163 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099576    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 03 14:08:05.101163 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099579    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 03 14:08:05.101163 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099582    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 03 14:08:05.101163 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099585    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 03 14:08:05.101163 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099588    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 03 14:08:05.101163 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099590    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 03 14:08:05.101163 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099594    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 03 14:08:05.101163 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099598    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 03 14:08:05.101163 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099601    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 03 14:08:05.101163 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099603    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 03 14:08:05.101163 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099606    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 03 14:08:05.101163 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099610    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 03 14:08:05.101163 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099613    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 03 14:08:05.101634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099615    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 03 14:08:05.101634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099618    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 03 14:08:05.101634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099620    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 03 14:08:05.101634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099623    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 03 14:08:05.101634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099626    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 03 14:08:05.101634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099629    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 03 14:08:05.101634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099631    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 03 14:08:05.101634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099634    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 03 14:08:05.101634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099637    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 03 14:08:05.101634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099639    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 03 14:08:05.101634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099642    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 03 14:08:05.101634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099644    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 03 14:08:05.101634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099647    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 03 14:08:05.101634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099650    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 03 14:08:05.101634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099654    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 03 14:08:05.101634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099657    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 03 14:08:05.101634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099660    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 03 14:08:05.101634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099664    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 03 14:08:05.101634 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099668    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 03 14:08:05.102116 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099672    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 03 14:08:05.102116 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099674    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 03 14:08:05.102116 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099678    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 03 14:08:05.102116 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099680    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 03 14:08:05.102116 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099683    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 03 14:08:05.102116 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099685    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 03 14:08:05.102116 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099688    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 03 14:08:05.102116 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099691    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 03 14:08:05.102116 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099693    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 03 14:08:05.102116 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099696    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 03 14:08:05.102116 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099698    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 03 14:08:05.102116 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099705    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 03 14:08:05.102116 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099707    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 03 14:08:05.102116 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099710    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 03 14:08:05.102116 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099712    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 03 14:08:05.102116 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099715    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 03 14:08:05.102116 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099717    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 03 14:08:05.102116 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.099720    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 03 14:08:05.102116 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100142    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 03 14:08:05.102116 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100149    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 03 14:08:05.102607 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100154    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 03 14:08:05.102607 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100157    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 03 14:08:05.102607 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100160    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 03 14:08:05.102607 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100163    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 03 14:08:05.102607 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100165    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 03 14:08:05.102607 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100168    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 03 14:08:05.102607 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100171    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 03 14:08:05.102607 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100173    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 03 14:08:05.102607 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100176    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 03 14:08:05.102607 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100179    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 03 14:08:05.102607 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100181    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 03 14:08:05.102607 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100184    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 03 14:08:05.102607 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100187    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 03 14:08:05.102607 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100190    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 03 14:08:05.102607 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100193    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 03 14:08:05.102607 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100196    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 03 14:08:05.102607 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100199    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 03 14:08:05.102607 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100201    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 03 14:08:05.102607 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100204    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 03 14:08:05.102607 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100206    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 03 14:08:05.103112 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100209    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 03 14:08:05.103112 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100211    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 03 14:08:05.103112 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100214    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 03 14:08:05.103112 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100218    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 03 14:08:05.103112 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100221    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 03 14:08:05.103112 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100223    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 03 14:08:05.103112 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100226    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 03 14:08:05.103112 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100228    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 03 14:08:05.103112 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100231    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 03 14:08:05.103112 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100233    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 03 14:08:05.103112 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100236    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 03 14:08:05.103112 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100238    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 03 14:08:05.103112 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100241    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 03 14:08:05.103112 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100243    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 03 14:08:05.103112 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100246    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 03 14:08:05.103112 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100249    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 03 14:08:05.103112 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100251    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 03 14:08:05.103112 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100254    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 03 14:08:05.103112 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100256    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 03 14:08:05.103112 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100258    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 03 14:08:05.103604 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100261    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 03 14:08:05.103604 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100263    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 03 14:08:05.103604 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100266    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 03 14:08:05.103604 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100269    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 03 14:08:05.103604 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100271    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 03 14:08:05.103604 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100274    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 03 14:08:05.103604 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100276    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 03 14:08:05.103604 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100280    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 03 14:08:05.103604 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100282    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 03 14:08:05.103604 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100285    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 03 14:08:05.103604 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100287    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 03 14:08:05.103604 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100289    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 03 14:08:05.103604 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100292    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 03 14:08:05.103604 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100294    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 03 14:08:05.103604 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100298    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 03 14:08:05.103604 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100301    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 03 14:08:05.103604 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100305    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 03 14:08:05.103604 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100309    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 03 14:08:05.103604 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100312    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 03 14:08:05.104095 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100315    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 03 14:08:05.104095 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100318    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 03 14:08:05.104095 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100321    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 03 14:08:05.104095 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100324    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 03 14:08:05.104095 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100327    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 03 14:08:05.104095 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100329    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 03 14:08:05.104095 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100332    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 03 14:08:05.104095 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100334    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 03 14:08:05.104095 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100336    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 03 14:08:05.104095 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100339    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 03 14:08:05.104095 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100341    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 03 14:08:05.104095 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100344    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 03 14:08:05.104095 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100347    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 03 14:08:05.104095 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100350    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 03 14:08:05.104095 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100352    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 03 14:08:05.104095 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100356    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 03 14:08:05.104095 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100359    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 03 14:08:05.104095 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100361    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 03 14:08:05.104095 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100364    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 03 14:08:05.104095 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100366    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 03 14:08:05.104576 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100368    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 03 14:08:05.104576 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100371    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 03 14:08:05.104576 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100375    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 03 14:08:05.104576 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100378    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 03 14:08:05.104576 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100381    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 03 14:08:05.104576 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100384    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 03 14:08:05.104576 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100386    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 03 14:08:05.104576 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100389    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 03 14:08:05.104576 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100393    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 03 14:08:05.104576 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100396    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 03 14:08:05.104576 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100398    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 03 14:08:05.104576 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100401    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 03 14:08:05.104576 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100403    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 03 14:08:05.104576 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100406    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 03 14:08:05.104576 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100409    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 03 14:08:05.104576 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100411    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 03 14:08:05.104576 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100414    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 03 14:08:05.104576 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.100416    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 03 14:08:05.104576 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100490    2563 flags.go:64] FLAG: --address="0.0.0.0"
Jul 03 14:08:05.104576 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100497    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 03 14:08:05.105110 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100504    2563 flags.go:64] FLAG: --anonymous-auth="true"
Jul 03 14:08:05.105110 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100509    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 03 14:08:05.105110 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100513    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 03 14:08:05.105110 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100516    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 03 14:08:05.105110 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100520    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 03 14:08:05.105110 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100524    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 03 14:08:05.105110 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100528    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 03 14:08:05.105110 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100530    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 03 14:08:05.105110 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100534    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 03 14:08:05.105110 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100537    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 03 14:08:05.105110 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100540    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 03 14:08:05.105110 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100543    2563 flags.go:64] FLAG: --cgroup-root=""
Jul 03 14:08:05.105110 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100545    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 03 14:08:05.105110 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100548    2563 flags.go:64] FLAG: --client-ca-file=""
Jul 03 14:08:05.105110 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100552    2563 flags.go:64] FLAG: --cloud-provider="external"
Jul 03 14:08:05.105110 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100555    2563 flags.go:64] FLAG: --cluster-dns="[]"
Jul 03 14:08:05.105110 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100559    2563 flags.go:64] FLAG: --cluster-domain=""
Jul 03 14:08:05.105110 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100562    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 03 14:08:05.105110 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100565    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 03 14:08:05.105110 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100568    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 03 14:08:05.105110 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100572    2563 flags.go:64] FLAG: --container-log-max-files="5"
Jul 03 14:08:05.105110 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100576    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 03 14:08:05.105110 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100579    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 03 14:08:05.105654 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100582    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 03 14:08:05.105654 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100585    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 03 14:08:05.105654 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100588    2563 flags.go:64] FLAG: --contention-profiling="false"
Jul 03 14:08:05.105654 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100591    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 03 14:08:05.105654 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100594    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 03 14:08:05.105654 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100597    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 03 14:08:05.105654 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100600    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 03 14:08:05.105654 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100604    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 03 14:08:05.105654 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100607    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 03 14:08:05.105654 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100610    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 03 14:08:05.105654 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100613    2563 flags.go:64] FLAG: --enable-load-reader="false"
Jul 03 14:08:05.105654 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100616    2563 flags.go:64] FLAG: --enable-server="true"
Jul 03 14:08:05.105654 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100618    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 03 14:08:05.105654 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100623    2563 flags.go:64] FLAG: --event-burst="100"
Jul 03 14:08:05.105654 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100626    2563 flags.go:64] FLAG: --event-qps="50"
Jul 03 14:08:05.105654 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100629    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 03 14:08:05.105654 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100632    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 03 14:08:05.105654 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100635    2563 flags.go:64] FLAG: --eviction-hard=""
Jul 03 14:08:05.105654 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100639    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 03 14:08:05.105654 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100642    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 03 14:08:05.105654 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100645    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 03 14:08:05.105654 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100648    2563 flags.go:64] FLAG: --eviction-soft=""
Jul 03 14:08:05.105654 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100651    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 03 14:08:05.105654 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100654    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 03 14:08:05.105654 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100657    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 03 14:08:05.106332 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100661    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 03 14:08:05.106332 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100664    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 03 14:08:05.106332 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100667    2563 flags.go:64] FLAG: --fail-swap-on="true"
Jul 03 14:08:05.106332 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100670    2563 flags.go:64] FLAG: --feature-gates=""
Jul 03 14:08:05.106332 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100674    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 03 14:08:05.106332 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100678    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 03 14:08:05.106332 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100680    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 03 14:08:05.106332 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100684    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 03 14:08:05.106332 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100687    2563 flags.go:64] FLAG: --healthz-port="10248"
Jul 03 14:08:05.106332 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100690    2563 flags.go:64] FLAG: --help="false"
Jul 03 14:08:05.106332 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100693    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-142-233.ec2.internal"
Jul 03 14:08:05.106332 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100696    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 03 14:08:05.106332 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100699    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 03 14:08:05.106332 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100701    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 03 14:08:05.106332 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100705    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 03 14:08:05.106332 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100708    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 03 14:08:05.106332 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100711    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 03 14:08:05.106332 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100714    2563 flags.go:64] FLAG: --image-service-endpoint=""
Jul 03 14:08:05.106332 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100716    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 03 14:08:05.106332 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100719    2563 flags.go:64] FLAG: --kube-api-burst="100"
Jul 03 14:08:05.106332 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100722    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 03 14:08:05.106332 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100725    2563 flags.go:64] FLAG: --kube-api-qps="50"
Jul 03 14:08:05.106332 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100728    2563 flags.go:64] FLAG: --kube-reserved=""
Jul 03 14:08:05.106332 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100731    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 03 14:08:05.106935 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100734    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 03 14:08:05.106935 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100737    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 03 14:08:05.106935 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100740    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 03 14:08:05.106935 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100742    2563 flags.go:64] FLAG: --lock-file=""
Jul 03 14:08:05.106935 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100745    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 03 14:08:05.106935 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100747    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 03 14:08:05.106935 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100750    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 03 14:08:05.106935 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100756    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 03 14:08:05.106935 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100759    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 03 14:08:05.106935 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100762    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 03 14:08:05.106935 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100765    2563 flags.go:64] FLAG: --logging-format="text"
Jul 03 14:08:05.106935 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100768    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 03 14:08:05.106935 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100771    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 03 14:08:05.106935 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100774    2563 flags.go:64] FLAG: --manifest-url=""
Jul 03 14:08:05.106935 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100777    2563 flags.go:64] FLAG: --manifest-url-header=""
Jul 03 14:08:05.106935 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100781    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 03 14:08:05.106935 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100784    2563 flags.go:64] FLAG: --max-open-files="1000000"
Jul 03 14:08:05.106935 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100788    2563 flags.go:64] FLAG: --max-pods="110"
Jul 03 14:08:05.106935 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100791    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 03 14:08:05.106935 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100794    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 03 14:08:05.106935 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100797    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 03 14:08:05.106935 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100800    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 03 14:08:05.106935 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100803    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 03 14:08:05.106935 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100806    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 03 14:08:05.106935 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100809    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 03 14:08:05.107545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100816    2563 flags.go:64] FLAG: --node-status-max-images="50"
Jul 03 14:08:05.107545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100819    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 03 14:08:05.107545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100822    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 03 14:08:05.107545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100825    2563 flags.go:64] FLAG: --pod-cidr=""
Jul 03 14:08:05.107545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100828    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 03 14:08:05.107545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100834    2563 flags.go:64] FLAG: --pod-manifest-path=""
Jul 03 14:08:05.107545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100837    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 03 14:08:05.107545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100840    2563 flags.go:64] FLAG: --pods-per-core="0"
Jul 03 14:08:05.107545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100843    2563 flags.go:64] FLAG: --port="10250"
Jul 03 14:08:05.107545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100846    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 03 14:08:05.107545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100849    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09c132d93124fee88"
Jul 03 14:08:05.107545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100852    2563 flags.go:64] FLAG: --qos-reserved=""
Jul 03 14:08:05.107545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100854    2563 flags.go:64] FLAG: --read-only-port="10255"
Jul 03 14:08:05.107545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100857    2563 flags.go:64] FLAG: --register-node="true"
Jul 03 14:08:05.107545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100860    2563 flags.go:64] FLAG: --register-with-taints=""
Jul 03 14:08:05.107545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100864    2563 flags.go:64] FLAG: --registry-burst="10"
Jul 03 14:08:05.107545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100867    2563 flags.go:64] FLAG: --registry-qps="5"
Jul 03 14:08:05.107545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100870    2563 flags.go:64] FLAG: --reserved-cpus=""
Jul 03 14:08:05.107545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100873    2563 flags.go:64] FLAG: --reserved-memory=""
Jul 03 14:08:05.107545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100876    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 03 14:08:05.107545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100879    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 03 14:08:05.107545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100882    2563 flags.go:64] FLAG: --rotate-certificates="false"
Jul 03 14:08:05.107545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100885    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 03 14:08:05.107545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100903    2563 flags.go:64] FLAG: --runonce="false"
Jul 03 14:08:05.107545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100907    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 03 14:08:05.108170 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100910    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 03 14:08:05.108170 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100913    2563 flags.go:64] FLAG: --seccomp-default="false"
Jul 03 14:08:05.108170 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100916    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 03 14:08:05.108170 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100919    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 03 14:08:05.108170 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100926    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 03 14:08:05.108170 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100929    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 03 14:08:05.108170 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100932    2563 flags.go:64] FLAG: --storage-driver-password="root"
Jul 03 14:08:05.108170 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100935    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 03 14:08:05.108170 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100938    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 03 14:08:05.108170 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100941    2563 flags.go:64] FLAG: --storage-driver-user="root"
Jul 03 14:08:05.108170 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100944    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 03 14:08:05.108170 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100947    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 03 14:08:05.108170 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100950    2563 flags.go:64] FLAG: --system-cgroups=""
Jul 03 14:08:05.108170 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100952    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 03 14:08:05.108170 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100958    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 03 14:08:05.108170 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100960    2563 flags.go:64] FLAG: --tls-cert-file=""
Jul 03 14:08:05.108170 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100963    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 03 14:08:05.108170 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100967    2563 flags.go:64] FLAG: --tls-min-version=""
Jul 03 14:08:05.108170 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100969    2563 flags.go:64] FLAG: --tls-private-key-file=""
Jul 03 14:08:05.108170 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100972    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 03 14:08:05.108170 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100975    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 03 14:08:05.108170 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100978    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 03 14:08:05.108170 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100980    2563 flags.go:64] FLAG: --v="2"
Jul 03 14:08:05.108170 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100984    2563 flags.go:64] FLAG: --version="false"
Jul 03 14:08:05.108170 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100989    2563 flags.go:64] FLAG: --vmodule=""
Jul 03 14:08:05.108768 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100993    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 03 14:08:05.108768 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.100996    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 03 14:08:05.108768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101109    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 03 14:08:05.108768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101113    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 03 14:08:05.108768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101116    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 03 14:08:05.108768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101119    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 03 14:08:05.108768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101121    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 03 14:08:05.108768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101124    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 03 14:08:05.108768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101126    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 03 14:08:05.108768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101129    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 03 14:08:05.108768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101132    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 03 14:08:05.108768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101136    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 03 14:08:05.108768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101139    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 03 14:08:05.108768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101141    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 03 14:08:05.108768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101144    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 03 14:08:05.108768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101146    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 03 14:08:05.108768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101149    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 03 14:08:05.108768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101151    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 03 14:08:05.108768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101154    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 03 14:08:05.108768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101158    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 03 14:08:05.109273 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101161    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 03 14:08:05.109273 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101164    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 03 14:08:05.109273 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101167    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 03 14:08:05.109273 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101170    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 03 14:08:05.109273 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101173    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 03 14:08:05.109273 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101176    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 03 14:08:05.109273 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101179    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 03 14:08:05.109273 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101181    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 03 14:08:05.109273 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101186    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 03 14:08:05.109273 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101189    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 03 14:08:05.109273 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101191    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 03 14:08:05.109273 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101194    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 03 14:08:05.109273 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101197    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 03 14:08:05.109273 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101200    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 03 14:08:05.109273 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101202    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 03 14:08:05.109273 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101205    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 03 14:08:05.109273 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101208    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 03 14:08:05.109273 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101210    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 03 14:08:05.109273 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101213    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 03 14:08:05.109273 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101215    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 03 14:08:05.109811 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101217    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 03 14:08:05.109811 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101220    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 03 14:08:05.109811 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101222    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 03 14:08:05.109811 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101226    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 03 14:08:05.109811 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101229    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 03 14:08:05.109811 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101231    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 03 14:08:05.109811 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101234    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 03 14:08:05.109811 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101236    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 03 14:08:05.109811 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101239    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 03 14:08:05.109811 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101242    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 03 14:08:05.109811 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101244    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 03 14:08:05.109811 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101246    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 03 14:08:05.109811 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101249    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 03 14:08:05.109811 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101251    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 03 14:08:05.109811 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101254    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 03 14:08:05.109811 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101256    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 03 14:08:05.109811 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101258    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 03 14:08:05.109811 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101261    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 03 14:08:05.109811 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101265    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 03 14:08:05.110284 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101268    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 03 14:08:05.110284 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101273    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 03 14:08:05.110284 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101276    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 03 14:08:05.110284 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101279    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 03 14:08:05.110284 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101282    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 03 14:08:05.110284 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101285    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 03 14:08:05.110284 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101288    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 03 14:08:05.110284 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101290    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 03 14:08:05.110284 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101293    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 03 14:08:05.110284 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101295    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 03 14:08:05.110284 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101298    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 03 14:08:05.110284 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101300    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 03 14:08:05.110284 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101303    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 03 14:08:05.110284 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101305    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 03 14:08:05.110284 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101308    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 03 14:08:05.110284 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101310    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 03 14:08:05.110284 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101314    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 03 14:08:05.110284 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101317    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 03 14:08:05.110284 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101319    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 03 14:08:05.110284 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101322    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 03 14:08:05.110768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101325    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 03 14:08:05.110768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101327    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 03 14:08:05.110768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101329    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 03 14:08:05.110768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101332    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 03 14:08:05.110768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101334    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 03 14:08:05.110768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101337    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 03 14:08:05.110768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101340    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 03 14:08:05.110768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101342    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 03 14:08:05.110768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101344    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 03 14:08:05.110768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101347    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 03 14:08:05.110768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101350    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 03 14:08:05.110768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101352    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 03 14:08:05.110768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101354    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 03 14:08:05.110768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101358    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 03 14:08:05.110768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101361    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 03 14:08:05.110768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101364    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 03 14:08:05.110768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101368    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 03 14:08:05.110768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101371    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 03 14:08:05.110768 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101373    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 03 14:08:05.111249 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101376    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 03 14:08:05.111249 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101379    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 03 14:08:05.111249 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.101381    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 03 14:08:05.111249 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.102219    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 03 14:08:05.111249 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.109305    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 03 14:08:05.111249 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.109319    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 03 14:08:05.111249 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.109338    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 03 14:08:05.111249 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.109344    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 03 14:08:05.111249 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109399    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 03 14:08:05.111249 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109403    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 03 14:08:05.111249 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109408    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 03 14:08:05.111249 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109412    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 03 14:08:05.111249 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109415    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 03 14:08:05.111249 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109418    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 03 14:08:05.111600 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109422    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 03 14:08:05.111600 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109425    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 03 14:08:05.111600 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109428    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 03 14:08:05.111600 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109430    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 03 14:08:05.111600 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109433    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 03 14:08:05.111600 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109436    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 03 14:08:05.111600 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109438    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 03 14:08:05.111600 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109441    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 03 14:08:05.111600 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109443    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 03 14:08:05.111600 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109446    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 03 14:08:05.111600 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109448    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 03 14:08:05.111600 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109451    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 03 14:08:05.111600 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109453    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 03 14:08:05.111600 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109456    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 03 14:08:05.111600 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109459    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 03 14:08:05.111600 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109462    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 03 14:08:05.111600 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109464    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 03 14:08:05.111600 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109467    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 03 14:08:05.111600 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109470    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 03 14:08:05.111600 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109473    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 03 14:08:05.112106 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109476    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 03 14:08:05.112106 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109479    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 03 14:08:05.112106 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109481    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 03 14:08:05.112106 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109485    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 03 14:08:05.112106 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109488    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 03 14:08:05.112106 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109490    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 03 14:08:05.112106 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109493    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 03 14:08:05.112106 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109496    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 03 14:08:05.112106 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109498    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 03 14:08:05.112106 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109502    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 03 14:08:05.112106 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109505    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 03 14:08:05.112106 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109508    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 03 14:08:05.112106 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109511    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 03 14:08:05.112106 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109514    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 03 14:08:05.112106 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109516    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 03 14:08:05.112106 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109519    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 03 14:08:05.112106 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109522    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 03 14:08:05.112106 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109525    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 03 14:08:05.112106 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109528    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 03 14:08:05.112566 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109530    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 03 14:08:05.112566 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109533    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 03 14:08:05.112566 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109535    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 03 14:08:05.112566 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109538    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 03 14:08:05.112566 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109540    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 03 14:08:05.112566 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109543    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 03 14:08:05.112566 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109545    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 03 14:08:05.112566 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109548    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 03 14:08:05.112566 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109551    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 03 14:08:05.112566 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109553    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 03 14:08:05.112566 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109556    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 03 14:08:05.112566 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109558    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 03 14:08:05.112566 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109562    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 03 14:08:05.112566 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109564    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 03 14:08:05.112566 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109567    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 03 14:08:05.112566 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109570    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 03 14:08:05.112566 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109573    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 03 14:08:05.112566 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109576    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 03 14:08:05.112566 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109579    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 03 14:08:05.112566 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109581    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 03 14:08:05.113061 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109584    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 03 14:08:05.113061 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109586    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 03 14:08:05.113061 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109588    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 03 14:08:05.113061 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109591    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 03 14:08:05.113061 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109594    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 03 14:08:05.113061 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109596    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 03 14:08:05.113061 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109599    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 03 14:08:05.113061 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109601    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 03 14:08:05.113061 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109604    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 03 14:08:05.113061 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109606    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 03 14:08:05.113061 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109609    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 03 14:08:05.113061 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109611    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 03 14:08:05.113061 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109614    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 03 14:08:05.113061 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109616    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 03 14:08:05.113061 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109619    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 03 14:08:05.113061 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109621    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 03 14:08:05.113061 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109624    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 03 14:08:05.113061 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109626    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 03 14:08:05.113061 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109629    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 03 14:08:05.113061 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109632    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 03 14:08:05.113542 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109634    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 03 14:08:05.113542 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109637    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 03 14:08:05.113542 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109639    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 03 14:08:05.113542 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109642    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 03 14:08:05.113542 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109644    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 03 14:08:05.113542 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109648    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 03 14:08:05.113542 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109650    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 03 14:08:05.113542 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109653    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 03 14:08:05.113542 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109656    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 03 14:08:05.113542 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109659    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 03 14:08:05.113542 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109662    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 03 14:08:05.113542 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109666    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 03 14:08:05.113542 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109669    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 03 14:08:05.113542 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109672    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 03 14:08:05.113542 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.109677    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 03 14:08:05.113924 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109810    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 03 14:08:05.113924 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109817    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 03 14:08:05.113924 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109820    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 03 14:08:05.113924 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109823    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 03 14:08:05.113924 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109826    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 03 14:08:05.113924 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109829    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 03 14:08:05.113924 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109831    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 03 14:08:05.113924 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109834    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 03 14:08:05.113924 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109837    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 03 14:08:05.113924 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109840    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 03 14:08:05.113924 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109842    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 03 14:08:05.113924 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109845    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 03 14:08:05.113924 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109847    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 03 14:08:05.113924 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109850    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 03 14:08:05.113924 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109852    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 03 14:08:05.113924 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109855    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 03 14:08:05.113924 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109857    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 03 14:08:05.113924 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109860    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 03 14:08:05.113924 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109862    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 03 14:08:05.113924 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109865    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 03 14:08:05.114405 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109868    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 03 14:08:05.114405 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109870    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 03 14:08:05.114405 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109873    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 03 14:08:05.114405 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109877    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 03 14:08:05.114405 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109881    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 03 14:08:05.114405 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109884    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 03 14:08:05.114405 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109903    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 03 14:08:05.114405 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109908    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 03 14:08:05.114405 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109911    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 03 14:08:05.114405 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109914    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 03 14:08:05.114405 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109916    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 03 14:08:05.114405 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109919    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 03 14:08:05.114405 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109922    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 03 14:08:05.114405 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109924    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 03 14:08:05.114405 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109927    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 03 14:08:05.114405 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109930    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 03 14:08:05.114405 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109932    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 03 14:08:05.114405 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109935    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 03 14:08:05.114405 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109937    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 03 14:08:05.114873 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109940    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 03 14:08:05.114873 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109943    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 03 14:08:05.114873 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109947    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 03 14:08:05.114873 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109950    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 03 14:08:05.114873 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109952    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 03 14:08:05.114873 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109955    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 03 14:08:05.114873 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109958    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 03 14:08:05.114873 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109960    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 03 14:08:05.114873 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109963    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 03 14:08:05.114873 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109965    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 03 14:08:05.114873 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109968    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 03 14:08:05.114873 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109970    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 03 14:08:05.114873 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109973    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 03 14:08:05.114873 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109975    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 03 14:08:05.114873 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109978    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 03 14:08:05.114873 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109980    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 03 14:08:05.114873 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109983    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 03 14:08:05.114873 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109986    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 03 14:08:05.114873 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109989    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 03 14:08:05.115359 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109992    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 03 14:08:05.115359 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109994    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 03 14:08:05.115359 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.109997    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 03 14:08:05.115359 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110000    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 03 14:08:05.115359 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110002    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 03 14:08:05.115359 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110005    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 03 14:08:05.115359 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110007    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 03 14:08:05.115359 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110010    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 03 14:08:05.115359 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110012    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 03 14:08:05.115359 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110015    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 03 14:08:05.115359 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110017    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 03 14:08:05.115359 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110020    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 03 14:08:05.115359 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110022    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 03 14:08:05.115359 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110025    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 03 14:08:05.115359 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110027    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 03 14:08:05.115359 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110030    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 03 14:08:05.115359 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110032    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 03 14:08:05.115359 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110035    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 03 14:08:05.115359 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110037    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 03 14:08:05.115359 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110040    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 03 14:08:05.115861 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110042    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 03 14:08:05.115861 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110045    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 03 14:08:05.115861 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110047    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 03 14:08:05.115861 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110050    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 03 14:08:05.115861 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110052    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 03 14:08:05.115861 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110055    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 03 14:08:05.115861 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110057    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 03 14:08:05.115861 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110060    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 03 14:08:05.115861 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110063    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 03 14:08:05.115861 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110065    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 03 14:08:05.115861 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110068    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 03 14:08:05.115861 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110071    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 03 14:08:05.115861 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110074    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 03 14:08:05.115861 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110077    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 03 14:08:05.115861 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110079    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 03 14:08:05.115861 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110081    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 03 14:08:05.115861 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110084    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 03 14:08:05.115861 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110086    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 03 14:08:05.115861 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110089    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 03 14:08:05.116483 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110091    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 03 14:08:05.116483 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:05.110095    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 03 14:08:05.116483 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.110101    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 03 14:08:05.116483 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.110211    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 03 14:08:05.116483 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.112616    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 03 14:08:05.116483 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.114208    2563 server.go:1019] "Starting client certificate rotation"
Jul 03 14:08:05.116483 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.114315    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 03 14:08:05.116483 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.114354    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 03 14:08:05.117096 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.117083    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 03 14:08:05.122148 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.122131    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 03 14:08:05.133727 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.133701    2563 log.go:25] "Validated CRI v1 runtime API"
Jul 03 14:08:05.136148 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.136132    2563 log.go:25] "Validated CRI v1 image API"
Jul 03 14:08:05.136913 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.136883    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 03 14:08:05.139516 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.139498    2563 fs.go:135] Filesystem UUIDs: map[5d527c6e-d2db-4423-b73f-a1bf98426795:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 7a35c5d4-0677-43aa-b255-6cbc4e9757c5:/dev/nvme0n1p4]
Jul 03 14:08:05.139589 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.139515    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 03 14:08:05.144838 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.144726    2563 manager.go:216] Machine: {Timestamp:2026-07-03 14:08:05.143324737 +0000 UTC m=+0.364537836 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3119290 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f59f9bf4d84215a1041ea9b8f6cce SystemUUID:ec2f59f9-bf4d-8421-5a10-41ea9b8f6cce BootID:128dab56-d43f-470d-8b93-0eaaa544fefc Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:96:c3:89:17:97 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:96:c3:89:17:97 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a6:65:d0:70:27:7d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 03 14:08:05.144838 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.144836    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 03 14:08:05.144967 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.144936    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 03 14:08:05.148718 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.148684    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 03 14:08:05.148869 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.148719    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-233.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 03 14:08:05.149637 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.149625    2563 topology_manager.go:138] "Creating topology manager with none policy"
Jul 03 14:08:05.149679 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.149641    2563 container_manager_linux.go:309] "Creating device plugin manager"
Jul 03 14:08:05.149679 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.149657    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 03 14:08:05.149679 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.149675    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 03 14:08:05.149792 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.149776    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 03 14:08:05.151942 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.151931    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 03 14:08:05.156517 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.156505    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 03 14:08:05.156652 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.156642    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 03 14:08:05.157520 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.157507    2563 kubelet.go:486] "Attempting to sync node with API server"
Jul 03 14:08:05.157561 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.157532    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 03 14:08:05.157561 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.157560    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 03 14:08:05.157613 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.157570    2563 kubelet.go:398] "Adding apiserver pod source"
Jul 03 14:08:05.157613 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.157603    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 03 14:08:05.158034 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.158023    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 03 14:08:05.158080 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.158042    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 03 14:08:05.160516 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.160496    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 03 14:08:05.163244 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.163214    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 03 14:08:05.163348 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.163262    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 03 14:08:05.164988 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.164976    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 03 14:08:05.165042 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.164993    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 03 14:08:05.165042 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.164999    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 03 14:08:05.165042 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.165005    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 03 14:08:05.165042 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.165014    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 03 14:08:05.165042 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.165020    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 03 14:08:05.165042 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.165028    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 03 14:08:05.165042 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.165033    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 03 14:08:05.165042 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.165040    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 03 14:08:05.165291 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.165053    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 03 14:08:05.165291 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.165069    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 03 14:08:05.165291 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.165075    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 03 14:08:05.167587 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.167574    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 03 14:08:05.167635 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.167588    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 03 14:08:05.168616 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:05.168579    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-233.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 03 14:08:05.168616 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:05.168579    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 03 14:08:05.171001 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.170988    2563 server.go:1268] "Started kubelet"
Jul 03 14:08:05.171107 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.171077    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 03 14:08:05.171735 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.171670    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 03 14:08:05.171913 ip-10-0-142-233 systemd[1]: Started Kubernetes Kubelet.
Jul 03 14:08:05.176110 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.175133    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 03 14:08:05.177004 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.176985    2563 server.go:310] "Adding debug handlers to kubelet server"
Jul 03 14:08:05.177101 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.177068    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 03 14:08:05.177688 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:05.177668    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 03 14:08:05.181801 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.181785    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-233.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 03 14:08:05.183684 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:05.181828    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-233.ec2.internal.18becccd6dfd18f8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-233.ec2.internal,UID:ip-10-0-142-233.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-233.ec2.internal,},FirstTimestamp:2026-07-03 14:08:05.170960632 +0000 UTC m=+0.392173731,LastTimestamp:2026-07-03 14:08:05.170960632 +0000 UTC m=+0.392173731,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-233.ec2.internal,}"
Jul 03 14:08:05.186179 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.186164    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 03 14:08:05.186243 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.186184    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 03 14:08:05.186753 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:05.186712    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-233.ec2.internal\" not found"
Jul 03 14:08:05.186912 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.186879    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 03 14:08:05.186912 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.186884    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 03 14:08:05.187038 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.186916    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 03 14:08:05.187096 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.187049    2563 reconstruct.go:97] "Volume reconstruction finished"
Jul 03 14:08:05.187096 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.187058    2563 reconciler.go:29] "Reconciler: start to sync state"
Jul 03 14:08:05.187183 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.187160    2563 factory.go:55] Registering systemd factory
Jul 03 14:08:05.187234 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.187184    2563 factory.go:223] Registration of the systemd container factory successfully
Jul 03 14:08:05.187446 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.187429    2563 factory.go:153] Registering CRI-O factory
Jul 03 14:08:05.187446 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.187448    2563 factory.go:223] Registration of the crio container factory successfully
Jul 03 14:08:05.187555 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.187503    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 03 14:08:05.187555 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.187530    2563 factory.go:103] Registering Raw factory
Jul 03 14:08:05.187555 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.187546    2563 manager.go:1196] Started watching for new ooms in manager
Jul 03 14:08:05.188022 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.188004    2563 manager.go:349] Starting recovery of all containers
Jul 03 14:08:05.189329 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:05.189303    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 03 14:08:05.195372 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:05.195344    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-233.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 03 14:08:05.198043 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.198027    2563 manager.go:354] Recovery completed
Jul 03 14:08:05.202492 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.202478    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 03 14:08:05.204366 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.204344    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-t2qgp"
Jul 03 14:08:05.204556 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.204543    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-233.ec2.internal" event="NodeHasSufficientMemory"
Jul 03 14:08:05.204604 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.204573    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-233.ec2.internal" event="NodeHasNoDiskPressure"
Jul 03 14:08:05.204604 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.204588    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-233.ec2.internal" event="NodeHasSufficientPID"
Jul 03 14:08:05.205129 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.205113    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 03 14:08:05.205129 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.205128    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 03 14:08:05.205217 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.205136    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 03 14:08:05.205217 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.205144    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 03 14:08:05.205818 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.205805    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-233.ec2.internal" event="NodeHasSufficientMemory"
Jul 03 14:08:05.205859 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.205831    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-233.ec2.internal" event="NodeHasNoDiskPressure"
Jul 03 14:08:05.205859 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.205842    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-233.ec2.internal" event="NodeHasSufficientPID"
Jul 03 14:08:05.206335 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:05.206240    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-233.ec2.internal.18becccd6ffdbf43  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-233.ec2.internal,UID:ip-10-0-142-233.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-233.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-233.ec2.internal,},FirstTimestamp:2026-07-03 14:08:05.204557635 +0000 UTC m=+0.425770737,LastTimestamp:2026-07-03 14:08:05.204557635 +0000 UTC m=+0.425770737,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-233.ec2.internal,}"
Jul 03 14:08:05.207291 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.207280    2563 policy_none.go:49] "None policy: Start"
Jul 03 14:08:05.207336 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.207296    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 03 14:08:05.207336 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.207306    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 03 14:08:05.209334 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.209321    2563 policy_none.go:47] "Start"
Jul 03 14:08:05.212078 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.212062    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-t2qgp"
Jul 03 14:08:05.245584 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.245444    2563 manager.go:337] "Starting Device Plugin manager"
Jul 03 14:08:05.262205 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:05.245605    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 03 14:08:05.262205 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.245617    2563 server.go:87] "Starting device plugin registration server"
Jul 03 14:08:05.262205 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.245860    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 03 14:08:05.262205 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.245871    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 03 14:08:05.262205 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.245952    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 03 14:08:05.262205 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.246032    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 03 14:08:05.262205 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.246038    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 03 14:08:05.262205 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:05.246526    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 03 14:08:05.262205 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:05.246572    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-233.ec2.internal\" not found"
Jul 03 14:08:05.279744 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.279713    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 03 14:08:05.280884 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.280867    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 03 14:08:05.280998 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.280915    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 03 14:08:05.280998 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.280946    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 03 14:08:05.280998 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:05.280982    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 03 14:08:05.283478 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.283459    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 03 14:08:05.346723 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.346638    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 03 14:08:05.347691 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.347676    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-233.ec2.internal" event="NodeHasSufficientMemory"
Jul 03 14:08:05.347809 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.347710    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-233.ec2.internal" event="NodeHasNoDiskPressure"
Jul 03 14:08:05.347809 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.347725    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-233.ec2.internal" event="NodeHasSufficientPID"
Jul 03 14:08:05.347809 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.347756    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-233.ec2.internal"
Jul 03 14:08:05.363315 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.363293    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-233.ec2.internal"
Jul 03 14:08:05.363414 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:05.363319    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-233.ec2.internal\": node \"ip-10-0-142-233.ec2.internal\" not found"
Jul 03 14:08:05.382076 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.382051    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-233.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-233.ec2.internal"]
Jul 03 14:08:05.382142 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.382119    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 03 14:08:05.383207 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.383193    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-233.ec2.internal" event="NodeHasSufficientMemory"
Jul 03 14:08:05.383276 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.383220    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-233.ec2.internal" event="NodeHasNoDiskPressure"
Jul 03 14:08:05.383276 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.383230    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-233.ec2.internal" event="NodeHasSufficientPID"
Jul 03 14:08:05.385452 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.385439    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 03 14:08:05.385584 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.385570    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-233.ec2.internal"
Jul 03 14:08:05.385642 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.385604    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 03 14:08:05.386559 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.386545    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-233.ec2.internal" event="NodeHasSufficientMemory"
Jul 03 14:08:05.386625 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.386570    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-233.ec2.internal" event="NodeHasNoDiskPressure"
Jul 03 14:08:05.386625 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.386584    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-233.ec2.internal" event="NodeHasSufficientPID"
Jul 03 14:08:05.386625 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.386615    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-233.ec2.internal" event="NodeHasSufficientMemory"
Jul 03 14:08:05.386733 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.386642    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-233.ec2.internal" event="NodeHasNoDiskPressure"
Jul 03 14:08:05.386733 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.386657    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-233.ec2.internal" event="NodeHasSufficientPID"
Jul 03 14:08:05.387517 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.387498    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d3d20294c54eebf1cdd45bd4f32b1e5e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-233.ec2.internal\" (UID: \"d3d20294c54eebf1cdd45bd4f32b1e5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-233.ec2.internal"
Jul 03 14:08:05.387585 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.387528    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3d20294c54eebf1cdd45bd4f32b1e5e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-233.ec2.internal\" (UID: \"d3d20294c54eebf1cdd45bd4f32b1e5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-233.ec2.internal"
Jul 03 14:08:05.387585 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.387544    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/61e1c42ac9a08ecfb3c8cc5e93139196-config\") pod \"kube-apiserver-proxy-ip-10-0-142-233.ec2.internal\" (UID: \"61e1c42ac9a08ecfb3c8cc5e93139196\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-233.ec2.internal"
Jul 03 14:08:05.389141 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.389126    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-233.ec2.internal"
Jul 03 14:08:05.389195 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.389152    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 03 14:08:05.389867 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.389851    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-233.ec2.internal" event="NodeHasSufficientMemory"
Jul 03 14:08:05.389957 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.389876    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-233.ec2.internal" event="NodeHasNoDiskPressure"
Jul 03 14:08:05.389957 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.389903    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-233.ec2.internal" event="NodeHasSufficientPID"
Jul 03 14:08:05.415812 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:05.415781    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-233.ec2.internal\" not found" node="ip-10-0-142-233.ec2.internal"
Jul 03 14:08:05.420330 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:05.420315    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-233.ec2.internal\" not found" node="ip-10-0-142-233.ec2.internal"
Jul 03 14:08:05.426257 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:05.426242    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-233.ec2.internal\" not found"
Jul 03 14:08:05.487731 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.487710    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/61e1c42ac9a08ecfb3c8cc5e93139196-config\") pod \"kube-apiserver-proxy-ip-10-0-142-233.ec2.internal\" (UID: \"61e1c42ac9a08ecfb3c8cc5e93139196\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-233.ec2.internal"
Jul 03 14:08:05.487871 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.487736    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d3d20294c54eebf1cdd45bd4f32b1e5e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-233.ec2.internal\" (UID: \"d3d20294c54eebf1cdd45bd4f32b1e5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-233.ec2.internal"
Jul 03 14:08:05.487871 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.487753    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3d20294c54eebf1cdd45bd4f32b1e5e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-233.ec2.internal\" (UID: \"d3d20294c54eebf1cdd45bd4f32b1e5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-233.ec2.internal"
Jul 03 14:08:05.487871 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.487765    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/61e1c42ac9a08ecfb3c8cc5e93139196-config\") pod \"kube-apiserver-proxy-ip-10-0-142-233.ec2.internal\" (UID: \"61e1c42ac9a08ecfb3c8cc5e93139196\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-233.ec2.internal"
Jul 03 14:08:05.487871 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.487806    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3d20294c54eebf1cdd45bd4f32b1e5e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-233.ec2.internal\" (UID: \"d3d20294c54eebf1cdd45bd4f32b1e5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-233.ec2.internal"
Jul 03 14:08:05.487871 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.487835    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d3d20294c54eebf1cdd45bd4f32b1e5e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-233.ec2.internal\" (UID: \"d3d20294c54eebf1cdd45bd4f32b1e5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-233.ec2.internal"
Jul 03 14:08:05.526353 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:05.526332    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-233.ec2.internal\" not found"
Jul 03 14:08:05.627341 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:05.627266    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-233.ec2.internal\" not found"
Jul 03 14:08:05.719987 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.719954    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-233.ec2.internal"
Jul 03 14:08:05.725032 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:05.724731    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-233.ec2.internal"
Jul 03 14:08:05.727849 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:05.727830    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-233.ec2.internal\" not found"
Jul 03 14:08:05.828628 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:05.828591    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-233.ec2.internal\" not found"
Jul 03 14:08:05.929182 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:05.929098    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-233.ec2.internal\" not found"
Jul 03 14:08:06.029620 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:06.029587    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-233.ec2.internal\" not found"
Jul 03 14:08:06.113125 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:06.113097    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 03 14:08:06.113531 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:06.113238    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 03 14:08:06.130394 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:06.130373    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-233.ec2.internal\" not found"
Jul 03 14:08:06.215441 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:06.215368    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-02 14:03:06 +0000 UTC" deadline="2028-02-16 15:05:52.510463617 +0000 UTC"
Jul 03 14:08:06.215441 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:06.215396    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14232h57m46.295071883s"
Jul 03 14:08:06.230469 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:06.230443    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-233.ec2.internal\" not found"
Jul 03 14:08:06.254944 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:06.254908    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61e1c42ac9a08ecfb3c8cc5e93139196.slice/crio-248c386e9a14f6ef8f9d6e98e50d7a19e406fa97c6e813677e8c281513626e7c WatchSource:0}: Error finding container 248c386e9a14f6ef8f9d6e98e50d7a19e406fa97c6e813677e8c281513626e7c: Status 404 returned error can't find the container with id 248c386e9a14f6ef8f9d6e98e50d7a19e406fa97c6e813677e8c281513626e7c
Jul 03 14:08:06.255275 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:06.255250    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3d20294c54eebf1cdd45bd4f32b1e5e.slice/crio-de9ae0db81f7161b56cace807f6af36fb7454a99d203ceb181088c79291ca4e8 WatchSource:0}: Error finding container de9ae0db81f7161b56cace807f6af36fb7454a99d203ceb181088c79291ca4e8: Status 404 returned error can't find the container with id de9ae0db81f7161b56cace807f6af36fb7454a99d203ceb181088c79291ca4e8
Jul 03 14:08:06.259798 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:06.259775    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 03 14:08:06.284528 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:06.284474    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-233.ec2.internal" event={"ID":"d3d20294c54eebf1cdd45bd4f32b1e5e","Type":"ContainerStarted","Data":"de9ae0db81f7161b56cace807f6af36fb7454a99d203ceb181088c79291ca4e8"}
Jul 03 14:08:06.285473 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:06.285452    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-233.ec2.internal" event={"ID":"61e1c42ac9a08ecfb3c8cc5e93139196","Type":"ContainerStarted","Data":"248c386e9a14f6ef8f9d6e98e50d7a19e406fa97c6e813677e8c281513626e7c"}
Jul 03 14:08:06.330959 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:06.330931    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-233.ec2.internal\" not found"
Jul 03 14:08:06.335076 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:06.335061    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 03 14:08:06.431770 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:06.431739    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-233.ec2.internal\" not found"
Jul 03 14:08:06.436511 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:06.436496    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 03 14:08:06.448937 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:06.448913    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 03 14:08:06.487423 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:06.487352    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-233.ec2.internal"
Jul 03 14:08:06.503816 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:06.503797    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 03 14:08:06.504818 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:06.504805    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-233.ec2.internal"
Jul 03 14:08:06.516013 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:06.515999    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 03 14:08:07.159219 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.159181    2563 apiserver.go:52] "Watching apiserver"
Jul 03 14:08:07.166009 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.165978    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 03 14:08:07.169426 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.169396    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-m4t7h","openshift-dns/node-resolver-9k9sd","openshift-image-registry/node-ca-7b795","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-233.ec2.internal","openshift-multus/multus-additional-cni-plugins-pz4j6","kube-system/kube-apiserver-proxy-ip-10-0-142-233.ec2.internal","openshift-multus/multus-frdqm","openshift-multus/network-metrics-daemon-9dzdm","openshift-network-diagnostics/network-check-target-4sq59","openshift-network-operator/iptables-alerter-pl747","openshift-ovn-kubernetes/ovnkube-node-xk86w","kube-system/global-pull-secret-syncer-rcksp","kube-system/konnectivity-agent-tw77w","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gq6s2"]
Jul 03 14:08:07.172247 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.172221    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pl747"
Jul 03 14:08:07.174503 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.174481    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tw77w"
Jul 03 14:08:07.176774 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.176755    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7b795"
Jul 03 14:08:07.177605 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.177583    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 03 14:08:07.177712 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.177692    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xhkk2\""
Jul 03 14:08:07.178053 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.178036    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 03 14:08:07.178131 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.178068    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 03 14:08:07.178672 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.178653    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 03 14:08:07.178906 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.178874    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fsfl7\""
Jul 03 14:08:07.179583 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.179167    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gq6s2"
Jul 03 14:08:07.179583 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.179564    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 03 14:08:07.179736 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.179643    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 03 14:08:07.179837 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.179820    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 03 14:08:07.180325 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.180054    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 03 14:08:07.180325 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.180130    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4jlm2\""
Jul 03 14:08:07.182226 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.181302    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 03 14:08:07.182226 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.181399    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 03 14:08:07.182226 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.181422    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 03 14:08:07.182226 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.181584    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pz4j6"
Jul 03 14:08:07.183762 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.183743    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:08:07.183866 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:07.183831    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dzdm" podUID="28b96845-6419-4df3-bff3-777d753989ce"
Jul 03 14:08:07.184017 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.183998    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 03 14:08:07.184157 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.184137    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ngmrj\""
Jul 03 14:08:07.184157 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.184151    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 03 14:08:07.184414 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.184395    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dmrfl\""
Jul 03 14:08:07.184528 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.184511    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 03 14:08:07.184721 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.184706    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 03 14:08:07.184721 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.184705    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 03 14:08:07.186107 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.186089    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:08:07.186190 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:07.186157    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sq59" podUID="c899d91f-c2fe-432e-b726-4e0f3bab3351"
Jul 03 14:08:07.186784 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.186767    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 03 14:08:07.188600 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.188582    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.191507 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.191133    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9k9sd"
Jul 03 14:08:07.191507 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.191240    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-lds9x\""
Jul 03 14:08:07.191507 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.191365    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 03 14:08:07.192853 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.192834    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.196917 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.195490    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.196917 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.196561    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 03 14:08:07.196917 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.196627    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 03 14:08:07.196917 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.196684    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7bkql\""
Jul 03 14:08:07.196917 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.196841    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 03 14:08:07.199040 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.197634    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mgdx7\""
Jul 03 14:08:07.199040 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.197632    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-system-cni-dir\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.199040 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.197748    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-multus-socket-dir-parent\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.199040 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.197770    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e70ced91-d923-4563-a8b7-7b370f727843-multus-daemon-config\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.199040 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.197806    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4883da48-1d2e-4958-99a5-d35f60236c3d-socket-dir\") pod \"aws-ebs-csi-driver-node-gq6s2\" (UID: \"4883da48-1d2e-4958-99a5-d35f60236c3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gq6s2"
Jul 03 14:08:07.199040 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.197815    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 03 14:08:07.199040 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.197824    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rr7lq\" (UniqueName: \"kubernetes.io/projected/2df513c5-3c49-4255-9106-9e36d40d4fa1-kube-api-access-rr7lq\") pod \"node-ca-7b795\" (UID: \"2df513c5-3c49-4255-9106-9e36d40d4fa1\") " pod="openshift-image-registry/node-ca-7b795"
Jul 03 14:08:07.199040 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.197845    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9sps\" (UniqueName: \"kubernetes.io/projected/c229a534-7dd6-44c1-88a9-4cd18b75369c-kube-api-access-t9sps\") pod \"iptables-alerter-pl747\" (UID: \"c229a534-7dd6-44c1-88a9-4cd18b75369c\") " pod="openshift-network-operator/iptables-alerter-pl747"
Jul 03 14:08:07.199040 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.197867    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-os-release\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.199040 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.197885    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c229a534-7dd6-44c1-88a9-4cd18b75369c-host-slash\") pod \"iptables-alerter-pl747\" (UID: \"c229a534-7dd6-44c1-88a9-4cd18b75369c\") " pod="openshift-network-operator/iptables-alerter-pl747"
Jul 03 14:08:07.199040 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.197934    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4883da48-1d2e-4958-99a5-d35f60236c3d-registration-dir\") pod \"aws-ebs-csi-driver-node-gq6s2\" (UID: \"4883da48-1d2e-4958-99a5-d35f60236c3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gq6s2"
Jul 03 14:08:07.199040 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.197963    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a5e9d1f2-6f21-48c0-8046-9f95a5218539-konnectivity-ca\") pod \"konnectivity-agent-tw77w\" (UID: \"a5e9d1f2-6f21-48c0-8046-9f95a5218539\") " pod="kube-system/konnectivity-agent-tw77w"
Jul 03 14:08:07.199040 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.198037    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8841628-0752-46c1-8dd8-850c21157563-system-cni-dir\") pod \"multus-additional-cni-plugins-pz4j6\" (UID: \"c8841628-0752-46c1-8dd8-850c21157563\") " pod="openshift-multus/multus-additional-cni-plugins-pz4j6"
Jul 03 14:08:07.199040 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.198083    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-multus-conf-dir\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.199040 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.198131    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4f4w\" (UniqueName: \"kubernetes.io/projected/e70ced91-d923-4563-a8b7-7b370f727843-kube-api-access-g4f4w\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.199040 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.198171    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c229a534-7dd6-44c1-88a9-4cd18b75369c-iptables-alerter-script\") pod \"iptables-alerter-pl747\" (UID: \"c229a534-7dd6-44c1-88a9-4cd18b75369c\") " pod="openshift-network-operator/iptables-alerter-pl747"
Jul 03 14:08:07.199040 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.198251    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-host-run-netns\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.199040 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.198278    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2df513c5-3c49-4255-9106-9e36d40d4fa1-serviceca\") pod \"node-ca-7b795\" (UID: \"2df513c5-3c49-4255-9106-9e36d40d4fa1\") " pod="openshift-image-registry/node-ca-7b795"
Jul 03 14:08:07.199824 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.198328    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-multus-cni-dir\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.199824 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.198351    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-cnibin\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.199824 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.198381    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-etc-kubernetes\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.199824 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.198411    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4883da48-1d2e-4958-99a5-d35f60236c3d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gq6s2\" (UID: \"4883da48-1d2e-4958-99a5-d35f60236c3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gq6s2"
Jul 03 14:08:07.199824 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.198462    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4883da48-1d2e-4958-99a5-d35f60236c3d-device-dir\") pod \"aws-ebs-csi-driver-node-gq6s2\" (UID: \"4883da48-1d2e-4958-99a5-d35f60236c3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gq6s2"
Jul 03 14:08:07.199824 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.198510    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4883da48-1d2e-4958-99a5-d35f60236c3d-etc-selinux\") pod \"aws-ebs-csi-driver-node-gq6s2\" (UID: \"4883da48-1d2e-4958-99a5-d35f60236c3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gq6s2"
Jul 03 14:08:07.199824 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.198539    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:07.199824 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.198555    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zhrms\""
Jul 03 14:08:07.199824 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.198561    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c8841628-0752-46c1-8dd8-850c21157563-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pz4j6\" (UID: \"c8841628-0752-46c1-8dd8-850c21157563\") " pod="openshift-multus/multus-additional-cni-plugins-pz4j6"
Jul 03 14:08:07.199824 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.198575    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 03 14:08:07.199824 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.198596    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjhmq\" (UniqueName: \"kubernetes.io/projected/c8841628-0752-46c1-8dd8-850c21157563-kube-api-access-qjhmq\") pod \"multus-additional-cni-plugins-pz4j6\" (UID: \"c8841628-0752-46c1-8dd8-850c21157563\") " pod="openshift-multus/multus-additional-cni-plugins-pz4j6"
Jul 03 14:08:07.199824 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.198628    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e70ced91-d923-4563-a8b7-7b370f727843-cni-binary-copy\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.199824 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.198666    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4883da48-1d2e-4958-99a5-d35f60236c3d-sys-fs\") pod \"aws-ebs-csi-driver-node-gq6s2\" (UID: \"4883da48-1d2e-4958-99a5-d35f60236c3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gq6s2"
Jul 03 14:08:07.199824 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.198695    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr8n6\" (UniqueName: \"kubernetes.io/projected/4883da48-1d2e-4958-99a5-d35f60236c3d-kube-api-access-pr8n6\") pod \"aws-ebs-csi-driver-node-gq6s2\" (UID: \"4883da48-1d2e-4958-99a5-d35f60236c3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gq6s2"
Jul 03 14:08:07.199824 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.198723    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28b96845-6419-4df3-bff3-777d753989ce-metrics-certs\") pod \"network-metrics-daemon-9dzdm\" (UID: \"28b96845-6419-4df3-bff3-777d753989ce\") " pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:08:07.199824 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.198771    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjtv4\" (UniqueName: \"kubernetes.io/projected/28b96845-6419-4df3-bff3-777d753989ce-kube-api-access-jjtv4\") pod \"network-metrics-daemon-9dzdm\" (UID: \"28b96845-6419-4df3-bff3-777d753989ce\") " pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:08:07.199824 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.198795    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c8841628-0752-46c1-8dd8-850c21157563-cnibin\") pod \"multus-additional-cni-plugins-pz4j6\" (UID: \"c8841628-0752-46c1-8dd8-850c21157563\") " pod="openshift-multus/multus-additional-cni-plugins-pz4j6"
Jul 03 14:08:07.199824 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.198840    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c8841628-0752-46c1-8dd8-850c21157563-cni-binary-copy\") pod \"multus-additional-cni-plugins-pz4j6\" (UID: \"c8841628-0752-46c1-8dd8-850c21157563\") " pod="openshift-multus/multus-additional-cni-plugins-pz4j6"
Jul 03 14:08:07.200631 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:07.198855    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcksp" podUID="01b1d786-bec1-48da-ace5-ec49bda1a3e3"
Jul 03 14:08:07.200631 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.198969    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c8841628-0752-46c1-8dd8-850c21157563-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pz4j6\" (UID: \"c8841628-0752-46c1-8dd8-850c21157563\") " pod="openshift-multus/multus-additional-cni-plugins-pz4j6"
Jul 03 14:08:07.200631 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.199008    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8w2v\" (UniqueName: \"kubernetes.io/projected/c899d91f-c2fe-432e-b726-4e0f3bab3351-kube-api-access-n8w2v\") pod \"network-check-target-4sq59\" (UID: \"c899d91f-c2fe-432e-b726-4e0f3bab3351\") " pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:08:07.200631 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.199026    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-host-run-k8s-cni-cncf-io\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.200631 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.199044    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-host-var-lib-cni-bin\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.200631 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.199071    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-host-var-lib-cni-multus\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.200631 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.199174    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a5e9d1f2-6f21-48c0-8046-9f95a5218539-agent-certs\") pod \"konnectivity-agent-tw77w\" (UID: \"a5e9d1f2-6f21-48c0-8046-9f95a5218539\") " pod="kube-system/konnectivity-agent-tw77w"
Jul 03 14:08:07.200631 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.199219    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c8841628-0752-46c1-8dd8-850c21157563-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pz4j6\" (UID: \"c8841628-0752-46c1-8dd8-850c21157563\") " pod="openshift-multus/multus-additional-cni-plugins-pz4j6"
Jul 03 14:08:07.200631 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.199251    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-hostroot\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.200631 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.199300    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-host-run-multus-certs\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.200631 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.199326    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-host-var-lib-kubelet\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.200631 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.199478    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2df513c5-3c49-4255-9106-9e36d40d4fa1-host\") pod \"node-ca-7b795\" (UID: \"2df513c5-3c49-4255-9106-9e36d40d4fa1\") " pod="openshift-image-registry/node-ca-7b795"
Jul 03 14:08:07.200631 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.199538    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c8841628-0752-46c1-8dd8-850c21157563-os-release\") pod \"multus-additional-cni-plugins-pz4j6\" (UID: \"c8841628-0752-46c1-8dd8-850c21157563\") " pod="openshift-multus/multus-additional-cni-plugins-pz4j6"
Jul 03 14:08:07.200631 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.200210    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 03 14:08:07.200631 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.200284    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 03 14:08:07.200631 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.200557    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 03 14:08:07.200631 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.200575    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 03 14:08:07.201483 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.201167    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 03 14:08:07.202506 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.202485    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 03 14:08:07.228654 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.228637    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-xwk4k"
Jul 03 14:08:07.237325 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.237304    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-xwk4k"
Jul 03 14:08:07.288039 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.288016    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 03 14:08:07.299885 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.299859    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-host-run-netns\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.300051 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.299908    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4883da48-1d2e-4958-99a5-d35f60236c3d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gq6s2\" (UID: \"4883da48-1d2e-4958-99a5-d35f60236c3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gq6s2"
Jul 03 14:08:07.300051 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.299934    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjtv4\" (UniqueName: \"kubernetes.io/projected/28b96845-6419-4df3-bff3-777d753989ce-kube-api-access-jjtv4\") pod \"network-metrics-daemon-9dzdm\" (UID: \"28b96845-6419-4df3-bff3-777d753989ce\") " pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:08:07.300051 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.299968    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-host-run-netns\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.300051 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.299980    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr8n6\" (UniqueName: \"kubernetes.io/projected/4883da48-1d2e-4958-99a5-d35f60236c3d-kube-api-access-pr8n6\") pod \"aws-ebs-csi-driver-node-gq6s2\" (UID: \"4883da48-1d2e-4958-99a5-d35f60236c3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gq6s2"
Jul 03 14:08:07.300051 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300008    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28b96845-6419-4df3-bff3-777d753989ce-metrics-certs\") pod \"network-metrics-daemon-9dzdm\" (UID: \"28b96845-6419-4df3-bff3-777d753989ce\") " pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:08:07.300051 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300030    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-run-openvswitch\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.300351 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300062    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4883da48-1d2e-4958-99a5-d35f60236c3d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gq6s2\" (UID: \"4883da48-1d2e-4958-99a5-d35f60236c3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gq6s2"
Jul 03 14:08:07.300351 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:07.300130    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 14:08:07.300351 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300143    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.300351 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300178    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c8841628-0752-46c1-8dd8-850c21157563-cnibin\") pod \"multus-additional-cni-plugins-pz4j6\" (UID: \"c8841628-0752-46c1-8dd8-850c21157563\") " pod="openshift-multus/multus-additional-cni-plugins-pz4j6"
Jul 03 14:08:07.300351 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:07.300206    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28b96845-6419-4df3-bff3-777d753989ce-metrics-certs podName:28b96845-6419-4df3-bff3-777d753989ce nodeName:}" failed. No retries permitted until 2026-07-03 14:08:07.800184456 +0000 UTC m=+3.021397562 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28b96845-6419-4df3-bff3-777d753989ce-metrics-certs") pod "network-metrics-daemon-9dzdm" (UID: "28b96845-6419-4df3-bff3-777d753989ce") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 14:08:07.300351 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300221    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c8841628-0752-46c1-8dd8-850c21157563-cnibin\") pod \"multus-additional-cni-plugins-pz4j6\" (UID: \"c8841628-0752-46c1-8dd8-850c21157563\") " pod="openshift-multus/multus-additional-cni-plugins-pz4j6"
Jul 03 14:08:07.300351 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300253    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-host-run-k8s-cni-cncf-io\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.300351 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300284    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-host-var-lib-cni-multus\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.300351 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300310    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/11f255a7-6652-4c64-97b8-c980552c363f-hosts-file\") pod \"node-resolver-9k9sd\" (UID: \"11f255a7-6652-4c64-97b8-c980552c363f\") " pod="openshift-dns/node-resolver-9k9sd"
Jul 03 14:08:07.300351 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300318    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-host-run-k8s-cni-cncf-io\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.300351 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300332    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-host-run-netns\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.300351 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300351    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-host-var-lib-cni-multus\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.300970 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300360    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-run-systemd\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.300970 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300385    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-node-log\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.300970 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300401    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-etc-sysctl-d\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.300970 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300443    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c8841628-0752-46c1-8dd8-850c21157563-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pz4j6\" (UID: \"c8841628-0752-46c1-8dd8-850c21157563\") " pod="openshift-multus/multus-additional-cni-plugins-pz4j6"
Jul 03 14:08:07.300970 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300464    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-host-run-multus-certs\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.300970 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300479    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-run-ovn\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.300970 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300493    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01b1d786-bec1-48da-ace5-ec49bda1a3e3-original-pull-secret\") pod \"global-pull-secret-syncer-rcksp\" (UID: \"01b1d786-bec1-48da-ace5-ec49bda1a3e3\") " pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:07.300970 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300507    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-host\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.300970 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300534    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2df513c5-3c49-4255-9106-9e36d40d4fa1-host\") pod \"node-ca-7b795\" (UID: \"2df513c5-3c49-4255-9106-9e36d40d4fa1\") " pod="openshift-image-registry/node-ca-7b795"
Jul 03 14:08:07.300970 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300556    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-host-run-multus-certs\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.300970 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300559    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-system-cni-dir\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.300970 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300589    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e70ced91-d923-4563-a8b7-7b370f727843-multus-daemon-config\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.300970 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300617    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2df513c5-3c49-4255-9106-9e36d40d4fa1-host\") pod \"node-ca-7b795\" (UID: \"2df513c5-3c49-4255-9106-9e36d40d4fa1\") " pod="openshift-image-registry/node-ca-7b795"
Jul 03 14:08:07.300970 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300610    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-system-cni-dir\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.300970 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300646    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-log-socket\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.300970 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300675    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rr7lq\" (UniqueName: \"kubernetes.io/projected/2df513c5-3c49-4255-9106-9e36d40d4fa1-kube-api-access-rr7lq\") pod \"node-ca-7b795\" (UID: \"2df513c5-3c49-4255-9106-9e36d40d4fa1\") " pod="openshift-image-registry/node-ca-7b795"
Jul 03 14:08:07.300970 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300681    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c8841628-0752-46c1-8dd8-850c21157563-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pz4j6\" (UID: \"c8841628-0752-46c1-8dd8-850c21157563\") " pod="openshift-multus/multus-additional-cni-plugins-pz4j6"
Jul 03 14:08:07.301646 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300814    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b3c91d5b-53e3-483c-a204-700513fc8648-ovnkube-config\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.301646 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300835    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/01b1d786-bec1-48da-ace5-ec49bda1a3e3-kubelet-config\") pod \"global-pull-secret-syncer-rcksp\" (UID: \"01b1d786-bec1-48da-ace5-ec49bda1a3e3\") " pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:07.301646 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300850    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/01b1d786-bec1-48da-ace5-ec49bda1a3e3-dbus\") pod \"global-pull-secret-syncer-rcksp\" (UID: \"01b1d786-bec1-48da-ace5-ec49bda1a3e3\") " pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:07.301646 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300867    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-var-lib-kubelet\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.301646 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300902    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgsjl\" (UniqueName: \"kubernetes.io/projected/1072d3c3-729b-422d-8899-509c061281b0-kube-api-access-cgsjl\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.301646 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300925    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c229a534-7dd6-44c1-88a9-4cd18b75369c-host-slash\") pod \"iptables-alerter-pl747\" (UID: \"c229a534-7dd6-44c1-88a9-4cd18b75369c\") " pod="openshift-network-operator/iptables-alerter-pl747"
Jul 03 14:08:07.301646 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300940    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-host-kubelet\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.301646 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300963    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-sys\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.301646 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.300987    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4f4w\" (UniqueName: \"kubernetes.io/projected/e70ced91-d923-4563-a8b7-7b370f727843-kube-api-access-g4f4w\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.301646 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301011    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c229a534-7dd6-44c1-88a9-4cd18b75369c-host-slash\") pod \"iptables-alerter-pl747\" (UID: \"c229a534-7dd6-44c1-88a9-4cd18b75369c\") " pod="openshift-network-operator/iptables-alerter-pl747"
Jul 03 14:08:07.301646 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301010    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-systemd-units\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.301646 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301084    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e70ced91-d923-4563-a8b7-7b370f727843-multus-daemon-config\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.301646 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301151    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b3c91d5b-53e3-483c-a204-700513fc8648-ovnkube-script-lib\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.301646 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301192    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-etc-kubernetes\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.301646 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301213    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-etc-systemd\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.301646 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301255    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2df513c5-3c49-4255-9106-9e36d40d4fa1-serviceca\") pod \"node-ca-7b795\" (UID: \"2df513c5-3c49-4255-9106-9e36d40d4fa1\") " pod="openshift-image-registry/node-ca-7b795"
Jul 03 14:08:07.301646 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301314    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-multus-cni-dir\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.302359 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301384    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-cnibin\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.302359 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301442    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-cnibin\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.302359 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301470    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-etc-kubernetes\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.302359 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301508    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-multus-cni-dir\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.302359 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301509    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-etc-kubernetes\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.302359 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301532    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4883da48-1d2e-4958-99a5-d35f60236c3d-device-dir\") pod \"aws-ebs-csi-driver-node-gq6s2\" (UID: \"4883da48-1d2e-4958-99a5-d35f60236c3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gq6s2"
Jul 03 14:08:07.302359 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301556    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4883da48-1d2e-4958-99a5-d35f60236c3d-etc-selinux\") pod \"aws-ebs-csi-driver-node-gq6s2\" (UID: \"4883da48-1d2e-4958-99a5-d35f60236c3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gq6s2"
Jul 03 14:08:07.302359 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301563    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4883da48-1d2e-4958-99a5-d35f60236c3d-device-dir\") pod \"aws-ebs-csi-driver-node-gq6s2\" (UID: \"4883da48-1d2e-4958-99a5-d35f60236c3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gq6s2"
Jul 03 14:08:07.302359 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301587    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-var-lib-openvswitch\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.302359 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301609    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c8841628-0752-46c1-8dd8-850c21157563-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pz4j6\" (UID: \"c8841628-0752-46c1-8dd8-850c21157563\") " pod="openshift-multus/multus-additional-cni-plugins-pz4j6"
Jul 03 14:08:07.302359 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301627    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjhmq\" (UniqueName: \"kubernetes.io/projected/c8841628-0752-46c1-8dd8-850c21157563-kube-api-access-qjhmq\") pod \"multus-additional-cni-plugins-pz4j6\" (UID: \"c8841628-0752-46c1-8dd8-850c21157563\") " pod="openshift-multus/multus-additional-cni-plugins-pz4j6"
Jul 03 14:08:07.302359 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301638    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2df513c5-3c49-4255-9106-9e36d40d4fa1-serviceca\") pod \"node-ca-7b795\" (UID: \"2df513c5-3c49-4255-9106-9e36d40d4fa1\") " pod="openshift-image-registry/node-ca-7b795"
Jul 03 14:08:07.302359 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301646    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4883da48-1d2e-4958-99a5-d35f60236c3d-etc-selinux\") pod \"aws-ebs-csi-driver-node-gq6s2\" (UID: \"4883da48-1d2e-4958-99a5-d35f60236c3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gq6s2"
Jul 03 14:08:07.302359 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301642    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e70ced91-d923-4563-a8b7-7b370f727843-cni-binary-copy\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.302359 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301677    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4883da48-1d2e-4958-99a5-d35f60236c3d-sys-fs\") pod \"aws-ebs-csi-driver-node-gq6s2\" (UID: \"4883da48-1d2e-4958-99a5-d35f60236c3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gq6s2"
Jul 03 14:08:07.302359 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301699    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a5e9d1f2-6f21-48c0-8046-9f95a5218539-agent-certs\") pod \"konnectivity-agent-tw77w\" (UID: \"a5e9d1f2-6f21-48c0-8046-9f95a5218539\") " pod="kube-system/konnectivity-agent-tw77w"
Jul 03 14:08:07.302359 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301728    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-host-cni-bin\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.303067 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301752    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c8841628-0752-46c1-8dd8-850c21157563-cni-binary-copy\") pod \"multus-additional-cni-plugins-pz4j6\" (UID: \"c8841628-0752-46c1-8dd8-850c21157563\") " pod="openshift-multus/multus-additional-cni-plugins-pz4j6"
Jul 03 14:08:07.303067 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301777    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c8841628-0752-46c1-8dd8-850c21157563-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pz4j6\" (UID: \"c8841628-0752-46c1-8dd8-850c21157563\") " pod="openshift-multus/multus-additional-cni-plugins-pz4j6"
Jul 03 14:08:07.303067 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301806    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8w2v\" (UniqueName: \"kubernetes.io/projected/c899d91f-c2fe-432e-b726-4e0f3bab3351-kube-api-access-n8w2v\") pod \"network-check-target-4sq59\" (UID: \"c899d91f-c2fe-432e-b726-4e0f3bab3351\") " pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:08:07.303067 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301832    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-host-var-lib-cni-bin\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.303067 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301856    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11f255a7-6652-4c64-97b8-c980552c363f-tmp-dir\") pod \"node-resolver-9k9sd\" (UID: \"11f255a7-6652-4c64-97b8-c980552c363f\") " pod="openshift-dns/node-resolver-9k9sd"
Jul 03 14:08:07.303067 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301880    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-host-run-ovn-kubernetes\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.303067 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301912    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4883da48-1d2e-4958-99a5-d35f60236c3d-sys-fs\") pod \"aws-ebs-csi-driver-node-gq6s2\" (UID: \"4883da48-1d2e-4958-99a5-d35f60236c3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gq6s2"
Jul 03 14:08:07.303067 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301925    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b3c91d5b-53e3-483c-a204-700513fc8648-env-overrides\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.303067 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301962    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-etc-modprobe-d\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.303067 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.301996    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-etc-openvswitch\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.303067 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302035    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-hostroot\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.303067 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302059    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-host-var-lib-cni-bin\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.303067 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302118    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e70ced91-d923-4563-a8b7-7b370f727843-cni-binary-copy\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.303067 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302120    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 03 14:08:07.303067 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302172    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c8841628-0752-46c1-8dd8-850c21157563-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pz4j6\" (UID: \"c8841628-0752-46c1-8dd8-850c21157563\") " pod="openshift-multus/multus-additional-cni-plugins-pz4j6"
Jul 03 14:08:07.303067 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302236    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-hostroot\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.303067 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302371    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c8841628-0752-46c1-8dd8-850c21157563-cni-binary-copy\") pod \"multus-additional-cni-plugins-pz4j6\" (UID: \"c8841628-0752-46c1-8dd8-850c21157563\") " pod="openshift-multus/multus-additional-cni-plugins-pz4j6"
Jul 03 14:08:07.303784 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302405    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c8841628-0752-46c1-8dd8-850c21157563-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pz4j6\" (UID: \"c8841628-0752-46c1-8dd8-850c21157563\") " pod="openshift-multus/multus-additional-cni-plugins-pz4j6"
Jul 03 14:08:07.303784 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302067    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgcsd\" (UniqueName: \"kubernetes.io/projected/b3c91d5b-53e3-483c-a204-700513fc8648-kube-api-access-mgcsd\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.303784 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302536    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-lib-modules\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.303784 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302578    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-host-var-lib-kubelet\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.303784 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302599    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c8841628-0752-46c1-8dd8-850c21157563-os-release\") pod \"multus-additional-cni-plugins-pz4j6\" (UID: \"c8841628-0752-46c1-8dd8-850c21157563\") " pod="openshift-multus/multus-additional-cni-plugins-pz4j6"
Jul 03 14:08:07.303784 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302633    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-multus-socket-dir-parent\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.303784 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302654    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4883da48-1d2e-4958-99a5-d35f60236c3d-socket-dir\") pod \"aws-ebs-csi-driver-node-gq6s2\" (UID: \"4883da48-1d2e-4958-99a5-d35f60236c3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gq6s2"
Jul 03 14:08:07.303784 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302669    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c8841628-0752-46c1-8dd8-850c21157563-os-release\") pod \"multus-additional-cni-plugins-pz4j6\" (UID: \"c8841628-0752-46c1-8dd8-850c21157563\") " pod="openshift-multus/multus-additional-cni-plugins-pz4j6"
Jul 03 14:08:07.303784 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302670    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-host-var-lib-kubelet\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.303784 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302678    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b3c91d5b-53e3-483c-a204-700513fc8648-ovn-node-metrics-cert\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.303784 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302704    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-etc-sysctl-conf\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.303784 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302721    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-multus-socket-dir-parent\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.303784 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302727    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-run\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.303784 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302750    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1072d3c3-729b-422d-8899-509c061281b0-tmp\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.303784 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302763    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4883da48-1d2e-4958-99a5-d35f60236c3d-socket-dir\") pod \"aws-ebs-csi-driver-node-gq6s2\" (UID: \"4883da48-1d2e-4958-99a5-d35f60236c3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gq6s2"
Jul 03 14:08:07.303784 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302776    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9sps\" (UniqueName: \"kubernetes.io/projected/c229a534-7dd6-44c1-88a9-4cd18b75369c-kube-api-access-t9sps\") pod \"iptables-alerter-pl747\" (UID: \"c229a534-7dd6-44c1-88a9-4cd18b75369c\") " pod="openshift-network-operator/iptables-alerter-pl747"
Jul 03 14:08:07.303784 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302813    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-os-release\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.304588 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302862    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4883da48-1d2e-4958-99a5-d35f60236c3d-registration-dir\") pod \"aws-ebs-csi-driver-node-gq6s2\" (UID: \"4883da48-1d2e-4958-99a5-d35f60236c3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gq6s2"
Jul 03 14:08:07.304588 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302901    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a5e9d1f2-6f21-48c0-8046-9f95a5218539-konnectivity-ca\") pod \"konnectivity-agent-tw77w\" (UID: \"a5e9d1f2-6f21-48c0-8046-9f95a5218539\") " pod="kube-system/konnectivity-agent-tw77w"
Jul 03 14:08:07.304588 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302911    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-os-release\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.304588 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302928    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkf24\" (UniqueName: \"kubernetes.io/projected/11f255a7-6652-4c64-97b8-c980552c363f-kube-api-access-rkf24\") pod \"node-resolver-9k9sd\" (UID: \"11f255a7-6652-4c64-97b8-c980552c363f\") " pod="openshift-dns/node-resolver-9k9sd"
Jul 03 14:08:07.304588 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302953    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-host-slash\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.304588 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302964    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4883da48-1d2e-4958-99a5-d35f60236c3d-registration-dir\") pod \"aws-ebs-csi-driver-node-gq6s2\" (UID: \"4883da48-1d2e-4958-99a5-d35f60236c3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gq6s2"
Jul 03 14:08:07.304588 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302973    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-host-cni-netd\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.304588 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.302998    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8841628-0752-46c1-8dd8-850c21157563-system-cni-dir\") pod \"multus-additional-cni-plugins-pz4j6\" (UID: \"c8841628-0752-46c1-8dd8-850c21157563\") " pod="openshift-multus/multus-additional-cni-plugins-pz4j6"
Jul 03 14:08:07.304588 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.303022    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-multus-conf-dir\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.304588 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.303044    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c229a534-7dd6-44c1-88a9-4cd18b75369c-iptables-alerter-script\") pod \"iptables-alerter-pl747\" (UID: \"c229a534-7dd6-44c1-88a9-4cd18b75369c\") " pod="openshift-network-operator/iptables-alerter-pl747"
Jul 03 14:08:07.304588 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.303066    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-etc-sysconfig\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.304588 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.303076    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8841628-0752-46c1-8dd8-850c21157563-system-cni-dir\") pod \"multus-additional-cni-plugins-pz4j6\" (UID: \"c8841628-0752-46c1-8dd8-850c21157563\") " pod="openshift-multus/multus-additional-cni-plugins-pz4j6"
Jul 03 14:08:07.304588 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.303092    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1072d3c3-729b-422d-8899-509c061281b0-etc-tuned\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.304588 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.303095    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e70ced91-d923-4563-a8b7-7b370f727843-multus-conf-dir\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.304588 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.303400    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a5e9d1f2-6f21-48c0-8046-9f95a5218539-konnectivity-ca\") pod \"konnectivity-agent-tw77w\" (UID: \"a5e9d1f2-6f21-48c0-8046-9f95a5218539\") " pod="kube-system/konnectivity-agent-tw77w"
Jul 03 14:08:07.304588 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.303520    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c229a534-7dd6-44c1-88a9-4cd18b75369c-iptables-alerter-script\") pod \"iptables-alerter-pl747\" (UID: \"c229a534-7dd6-44c1-88a9-4cd18b75369c\") " pod="openshift-network-operator/iptables-alerter-pl747"
Jul 03 14:08:07.305973 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.305954    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a5e9d1f2-6f21-48c0-8046-9f95a5218539-agent-certs\") pod \"konnectivity-agent-tw77w\" (UID: \"a5e9d1f2-6f21-48c0-8046-9f95a5218539\") " pod="kube-system/konnectivity-agent-tw77w"
Jul 03 14:08:07.309790 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:07.309752    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 03 14:08:07.309790 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:07.309776    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 03 14:08:07.309790 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:07.309791    2563 projected.go:196] Error preparing data for projected volume kube-api-access-n8w2v for pod openshift-network-diagnostics/network-check-target-4sq59: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 14:08:07.310093 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:07.309874    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c899d91f-c2fe-432e-b726-4e0f3bab3351-kube-api-access-n8w2v podName:c899d91f-c2fe-432e-b726-4e0f3bab3351 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:07.809854803 +0000 UTC m=+3.031067910 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-n8w2v" (UniqueName: "kubernetes.io/projected/c899d91f-c2fe-432e-b726-4e0f3bab3351-kube-api-access-n8w2v") pod "network-check-target-4sq59" (UID: "c899d91f-c2fe-432e-b726-4e0f3bab3351") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 14:08:07.310093 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.310015    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjtv4\" (UniqueName: \"kubernetes.io/projected/28b96845-6419-4df3-bff3-777d753989ce-kube-api-access-jjtv4\") pod \"network-metrics-daemon-9dzdm\" (UID: \"28b96845-6419-4df3-bff3-777d753989ce\") " pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:08:07.310405 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.310380    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rr7lq\" (UniqueName: \"kubernetes.io/projected/2df513c5-3c49-4255-9106-9e36d40d4fa1-kube-api-access-rr7lq\") pod \"node-ca-7b795\" (UID: \"2df513c5-3c49-4255-9106-9e36d40d4fa1\") " pod="openshift-image-registry/node-ca-7b795"
Jul 03 14:08:07.310503 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.310470    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4f4w\" (UniqueName: \"kubernetes.io/projected/e70ced91-d923-4563-a8b7-7b370f727843-kube-api-access-g4f4w\") pod \"multus-frdqm\" (UID: \"e70ced91-d923-4563-a8b7-7b370f727843\") " pod="openshift-multus/multus-frdqm"
Jul 03 14:08:07.310696 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.310672    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr8n6\" (UniqueName: \"kubernetes.io/projected/4883da48-1d2e-4958-99a5-d35f60236c3d-kube-api-access-pr8n6\") pod \"aws-ebs-csi-driver-node-gq6s2\" (UID: \"4883da48-1d2e-4958-99a5-d35f60236c3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gq6s2"
Jul 03 14:08:07.311560 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.311535    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9sps\" (UniqueName: \"kubernetes.io/projected/c229a534-7dd6-44c1-88a9-4cd18b75369c-kube-api-access-t9sps\") pod \"iptables-alerter-pl747\" (UID: \"c229a534-7dd6-44c1-88a9-4cd18b75369c\") " pod="openshift-network-operator/iptables-alerter-pl747"
Jul 03 14:08:07.312503 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.312479    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjhmq\" (UniqueName: \"kubernetes.io/projected/c8841628-0752-46c1-8dd8-850c21157563-kube-api-access-qjhmq\") pod \"multus-additional-cni-plugins-pz4j6\" (UID: \"c8841628-0752-46c1-8dd8-850c21157563\") " pod="openshift-multus/multus-additional-cni-plugins-pz4j6"
Jul 03 14:08:07.400144 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.400116    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 03 14:08:07.403358 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.403336    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-systemd-units\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.403485 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.403363    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b3c91d5b-53e3-483c-a204-700513fc8648-ovnkube-script-lib\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.403485 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.403381    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-etc-kubernetes\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.403485 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.403404    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-etc-systemd\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.403485 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.403435    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-systemd-units\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.403485 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.403469    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-etc-systemd\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.403725 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.403496    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-etc-kubernetes\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.403725 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.403539    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-var-lib-openvswitch\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.403725 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.403588    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-host-cni-bin\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.403725 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.403633    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11f255a7-6652-4c64-97b8-c980552c363f-tmp-dir\") pod \"node-resolver-9k9sd\" (UID: \"11f255a7-6652-4c64-97b8-c980552c363f\") " pod="openshift-dns/node-resolver-9k9sd"
Jul 03 14:08:07.403725 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.403637    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-var-lib-openvswitch\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.403725 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.403660    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-host-run-ovn-kubernetes\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.403725 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.403667    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-host-cni-bin\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.403725 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.403688    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b3c91d5b-53e3-483c-a204-700513fc8648-env-overrides\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.403725 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.403703    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-etc-modprobe-d\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.403725 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.403708    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-host-run-ovn-kubernetes\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.403725 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.403725    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-etc-openvswitch\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.404277 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.403829    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgcsd\" (UniqueName: \"kubernetes.io/projected/b3c91d5b-53e3-483c-a204-700513fc8648-kube-api-access-mgcsd\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.404277 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.403840    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-etc-modprobe-d\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.404277 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.403857    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-lib-modules\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.404277 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.403909    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b3c91d5b-53e3-483c-a204-700513fc8648-ovn-node-metrics-cert\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.404277 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.403941    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-etc-sysctl-conf\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.404277 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.403965    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-run\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.404277 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.403973    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11f255a7-6652-4c64-97b8-c980552c363f-tmp-dir\") pod \"node-resolver-9k9sd\" (UID: \"11f255a7-6652-4c64-97b8-c980552c363f\") " pod="openshift-dns/node-resolver-9k9sd"
Jul 03 14:08:07.404277 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.403983    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-lib-modules\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.404277 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.403990    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1072d3c3-729b-422d-8899-509c061281b0-tmp\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.404277 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404033    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-etc-openvswitch\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.404277 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404043    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkf24\" (UniqueName: \"kubernetes.io/projected/11f255a7-6652-4c64-97b8-c980552c363f-kube-api-access-rkf24\") pod \"node-resolver-9k9sd\" (UID: \"11f255a7-6652-4c64-97b8-c980552c363f\") " pod="openshift-dns/node-resolver-9k9sd"
Jul 03 14:08:07.404277 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404064    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b3c91d5b-53e3-483c-a204-700513fc8648-ovnkube-script-lib\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.404277 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404071    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-host-slash\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.404277 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404096    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-host-cni-netd\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.404277 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404123    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-etc-sysconfig\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.404277 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404146    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1072d3c3-729b-422d-8899-509c061281b0-etc-tuned\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.404277 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404165    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b3c91d5b-53e3-483c-a204-700513fc8648-env-overrides\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.404277 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404186    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-run-openvswitch\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.405120 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404222    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-host-cni-netd\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.405120 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404212    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.405120 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404260    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/11f255a7-6652-4c64-97b8-c980552c363f-hosts-file\") pod \"node-resolver-9k9sd\" (UID: \"11f255a7-6652-4c64-97b8-c980552c363f\") " pod="openshift-dns/node-resolver-9k9sd"
Jul 03 14:08:07.405120 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404265    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-host-slash\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.405120 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404282    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-host-run-netns\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.405120 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404306    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-run-systemd\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.405120 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404319    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-etc-sysconfig\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.405120 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404329    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-node-log\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.405120 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404354    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-etc-sysctl-d\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.405120 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404380    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-run-ovn\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.405120 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404405    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01b1d786-bec1-48da-ace5-ec49bda1a3e3-original-pull-secret\") pod \"global-pull-secret-syncer-rcksp\" (UID: \"01b1d786-bec1-48da-ace5-ec49bda1a3e3\") " pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:07.405120 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404420    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-etc-sysctl-conf\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.405120 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404429    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-host\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.405120 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404431    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-run\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.405120 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404433    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-host-run-netns\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.405120 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404458    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-log-socket\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.405120 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404466    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-run-openvswitch\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.405120 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404483    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b3c91d5b-53e3-483c-a204-700513fc8648-ovnkube-config\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.405711 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404512    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/01b1d786-bec1-48da-ace5-ec49bda1a3e3-kubelet-config\") pod \"global-pull-secret-syncer-rcksp\" (UID: \"01b1d786-bec1-48da-ace5-ec49bda1a3e3\") " pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:07.405711 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404527    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-run-ovn\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.405711 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404527    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-host\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.405711 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404536    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/01b1d786-bec1-48da-ace5-ec49bda1a3e3-dbus\") pod \"global-pull-secret-syncer-rcksp\" (UID: \"01b1d786-bec1-48da-ace5-ec49bda1a3e3\") " pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:07.405711 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404554    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-log-socket\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.405711 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404546    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.405711 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404587    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/11f255a7-6652-4c64-97b8-c980552c363f-hosts-file\") pod \"node-resolver-9k9sd\" (UID: \"11f255a7-6652-4c64-97b8-c980552c363f\") " pod="openshift-dns/node-resolver-9k9sd"
Jul 03 14:08:07.405711 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404596    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-var-lib-kubelet\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.405711 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404623    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgsjl\" (UniqueName: \"kubernetes.io/projected/1072d3c3-729b-422d-8899-509c061281b0-kube-api-access-cgsjl\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.405711 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404636    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/01b1d786-bec1-48da-ace5-ec49bda1a3e3-dbus\") pod \"global-pull-secret-syncer-rcksp\" (UID: \"01b1d786-bec1-48da-ace5-ec49bda1a3e3\") " pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:07.405711 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:07.404642    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 03 14:08:07.405711 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404487    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-node-log\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.405711 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404652    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-host-kubelet\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.405711 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404655    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-etc-sysctl-d\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.405711 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404485    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-run-systemd\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.405711 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404675    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-sys\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.405711 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:07.404695    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01b1d786-bec1-48da-ace5-ec49bda1a3e3-original-pull-secret podName:01b1d786-bec1-48da-ace5-ec49bda1a3e3 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:07.904678151 +0000 UTC m=+3.125891240 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01b1d786-bec1-48da-ace5-ec49bda1a3e3-original-pull-secret") pod "global-pull-secret-syncer-rcksp" (UID: "01b1d786-bec1-48da-ace5-ec49bda1a3e3") : object "kube-system"/"original-pull-secret" not registered
Jul 03 14:08:07.406302 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404697    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/01b1d786-bec1-48da-ace5-ec49bda1a3e3-kubelet-config\") pod \"global-pull-secret-syncer-rcksp\" (UID: \"01b1d786-bec1-48da-ace5-ec49bda1a3e3\") " pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:07.406302 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404753    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-var-lib-kubelet\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.406302 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404761    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1072d3c3-729b-422d-8899-509c061281b0-sys\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.406302 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.404793    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3c91d5b-53e3-483c-a204-700513fc8648-host-kubelet\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.406302 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.405114    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b3c91d5b-53e3-483c-a204-700513fc8648-ovnkube-config\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.406795 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.406772    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1072d3c3-729b-422d-8899-509c061281b0-tmp\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.406878 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.406807    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b3c91d5b-53e3-483c-a204-700513fc8648-ovn-node-metrics-cert\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.406878 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.406855    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1072d3c3-729b-422d-8899-509c061281b0-etc-tuned\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:07.415797 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.415709    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgcsd\" (UniqueName: \"kubernetes.io/projected/b3c91d5b-53e3-483c-a204-700513fc8648-kube-api-access-mgcsd\") pod \"ovnkube-node-xk86w\" (UID: \"b3c91d5b-53e3-483c-a204-700513fc8648\") " pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:07.416043 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.416022    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkf24\" (UniqueName: \"kubernetes.io/projected/11f255a7-6652-4c64-97b8-c980552c363f-kube-api-access-rkf24\") pod \"node-resolver-9k9sd\" (UID: \"11f255a7-6652-4c64-97b8-c980552c363f\") " pod="openshift-dns/node-resolver-9k9sd"
Jul 03 14:08:07.417236 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:07.417217    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgsjl\" (UniqueName: \"kubernetes.io/projected/1072d3c3-729b-422d-8899-509c061281b0-kube-api-access-cgsjl\") pod \"tuned-m4t7h\" (UID: \"1072d3c3-729b-422d-8899-509c061281b0\") " pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:09.258862 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:09.258814    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pl747"
Jul 03 14:08:09.265833 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:09.265810    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tw77w"
Jul 03 14:08:09.275574 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:09.275552    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7b795"
Jul 03 14:08:09.281780 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:09.281757    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gq6s2"
Jul 03 14:08:09.287663 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:09.287640    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pz4j6"
Jul 03 14:08:09.295513 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:09.295500    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-frdqm"
Jul 03 14:08:09.301542 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:09.301526    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9k9sd"
Jul 03 14:08:09.308124 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:09.308106    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m4t7h"
Jul 03 14:08:09.314032 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:09.314008    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:09.578713 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:09.578642    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28b96845-6419-4df3-bff3-777d753989ce-metrics-certs\") pod \"network-metrics-daemon-9dzdm\" (UID: \"28b96845-6419-4df3-bff3-777d753989ce\") " pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:08:09.578887 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:09.578820    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 14:08:09.578948 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:09.578902    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28b96845-6419-4df3-bff3-777d753989ce-metrics-certs podName:28b96845-6419-4df3-bff3-777d753989ce nodeName:}" failed. No retries permitted until 2026-07-03 14:08:10.578881208 +0000 UTC m=+4.029014625 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28b96845-6419-4df3-bff3-777d753989ce-metrics-certs") pod "network-metrics-daemon-9dzdm" (UID: "28b96845-6419-4df3-bff3-777d753989ce") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 14:08:09.679041 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:09.678999    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8w2v\" (UniqueName: \"kubernetes.io/projected/c899d91f-c2fe-432e-b726-4e0f3bab3351-kube-api-access-n8w2v\") pod \"network-check-target-4sq59\" (UID: \"c899d91f-c2fe-432e-b726-4e0f3bab3351\") " pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:08:09.679226 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:09.679179    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 03 14:08:09.679226 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:09.679190    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01b1d786-bec1-48da-ace5-ec49bda1a3e3-original-pull-secret\") pod \"global-pull-secret-syncer-rcksp\" (UID: \"01b1d786-bec1-48da-ace5-ec49bda1a3e3\") " pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:09.679306 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:09.679205    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 03 14:08:09.679306 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:09.679251    2563 projected.go:196] Error preparing data for projected volume kube-api-access-n8w2v for pod openshift-network-diagnostics/network-check-target-4sq59: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 14:08:09.679393 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:09.679316    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c899d91f-c2fe-432e-b726-4e0f3bab3351-kube-api-access-n8w2v podName:c899d91f-c2fe-432e-b726-4e0f3bab3351 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:10.679299772 +0000 UTC m=+4.129433210 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-n8w2v" (UniqueName: "kubernetes.io/projected/c899d91f-c2fe-432e-b726-4e0f3bab3351-kube-api-access-n8w2v") pod "network-check-target-4sq59" (UID: "c899d91f-c2fe-432e-b726-4e0f3bab3351") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 14:08:09.679393 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:09.679333    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 03 14:08:09.679393 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:09.679370    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01b1d786-bec1-48da-ace5-ec49bda1a3e3-original-pull-secret podName:01b1d786-bec1-48da-ace5-ec49bda1a3e3 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:10.679359664 +0000 UTC m=+4.129493087 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01b1d786-bec1-48da-ace5-ec49bda1a3e3-original-pull-secret") pod "global-pull-secret-syncer-rcksp" (UID: "01b1d786-bec1-48da-ace5-ec49bda1a3e3") : object "kube-system"/"original-pull-secret" not registered
Jul 03 14:08:09.957279 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:09.957206    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11f255a7_6652_4c64_97b8_c980552c363f.slice/crio-ea2e24d6007b629068893f7554ac77ef6057b9a712edf85e58aa622191e9117b WatchSource:0}: Error finding container ea2e24d6007b629068893f7554ac77ef6057b9a712edf85e58aa622191e9117b: Status 404 returned error can't find the container with id ea2e24d6007b629068893f7554ac77ef6057b9a712edf85e58aa622191e9117b
Jul 03 14:08:09.958155 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:09.958110    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4883da48_1d2e_4958_99a5_d35f60236c3d.slice/crio-6ae436bbcad5872832c8a9008f971e4dbe2c311b76ab36aa774a2a3cce7b6a8d WatchSource:0}: Error finding container 6ae436bbcad5872832c8a9008f971e4dbe2c311b76ab36aa774a2a3cce7b6a8d: Status 404 returned error can't find the container with id 6ae436bbcad5872832c8a9008f971e4dbe2c311b76ab36aa774a2a3cce7b6a8d
Jul 03 14:08:09.959426 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:09.959410    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc229a534_7dd6_44c1_88a9_4cd18b75369c.slice/crio-0bad49facf889c3129601ae02c02d2b374479f4b6bb354b82c472f53a253075c WatchSource:0}: Error finding container 0bad49facf889c3129601ae02c02d2b374479f4b6bb354b82c472f53a253075c: Status 404 returned error can't find the container with id 0bad49facf889c3129601ae02c02d2b374479f4b6bb354b82c472f53a253075c
Jul 03 14:08:09.960298 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:09.960276    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8841628_0752_46c1_8dd8_850c21157563.slice/crio-84d118874138949828aa65b97cca53cb276619e0422de32e44f67a14b08af0fb WatchSource:0}: Error finding container 84d118874138949828aa65b97cca53cb276619e0422de32e44f67a14b08af0fb: Status 404 returned error can't find the container with id 84d118874138949828aa65b97cca53cb276619e0422de32e44f67a14b08af0fb
Jul 03 14:08:09.962540 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:09.962517    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5e9d1f2_6f21_48c0_8046_9f95a5218539.slice/crio-9f66818747e8a3530146769af443417b02c2ddaeffc4737e369bea87a8a32b0e WatchSource:0}: Error finding container 9f66818747e8a3530146769af443417b02c2ddaeffc4737e369bea87a8a32b0e: Status 404 returned error can't find the container with id 9f66818747e8a3530146769af443417b02c2ddaeffc4737e369bea87a8a32b0e
Jul 03 14:08:09.963472 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:09.963452    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3c91d5b_53e3_483c_a204_700513fc8648.slice/crio-7f020b463436cbb79a90a9e125e8d3a5884aa424e9c60f73dbd229ce5549c468 WatchSource:0}: Error finding container 7f020b463436cbb79a90a9e125e8d3a5884aa424e9c60f73dbd229ce5549c468: Status 404 returned error can't find the container with id 7f020b463436cbb79a90a9e125e8d3a5884aa424e9c60f73dbd229ce5549c468
Jul 03 14:08:09.965719 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:09.965701    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2df513c5_3c49_4255_9106_9e36d40d4fa1.slice/crio-c910d3fcb22188648c5e7bb3fa06b89d755e039f0da5664555c5f4554f20d0e7 WatchSource:0}: Error finding container c910d3fcb22188648c5e7bb3fa06b89d755e039f0da5664555c5f4554f20d0e7: Status 404 returned error can't find the container with id c910d3fcb22188648c5e7bb3fa06b89d755e039f0da5664555c5f4554f20d0e7
Jul 03 14:08:09.966583 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:09.966071    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode70ced91_d923_4563_a8b7_7b370f727843.slice/crio-d8b900f48ebad2e32f33afd8bfb0364664931b98f6af8d53eb9cfb419ae71686 WatchSource:0}: Error finding container d8b900f48ebad2e32f33afd8bfb0364664931b98f6af8d53eb9cfb419ae71686: Status 404 returned error can't find the container with id d8b900f48ebad2e32f33afd8bfb0364664931b98f6af8d53eb9cfb419ae71686
Jul 03 14:08:09.968567 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:09.968063    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1072d3c3_729b_422d_8899_509c061281b0.slice/crio-1ce8b728033fa035cc5abdbab939d1d8921ceab6c9da1be4da9c31e0117d4a60 WatchSource:0}: Error finding container 1ce8b728033fa035cc5abdbab939d1d8921ceab6c9da1be4da9c31e0117d4a60: Status 404 returned error can't find the container with id 1ce8b728033fa035cc5abdbab939d1d8921ceab6c9da1be4da9c31e0117d4a60
Jul 03 14:08:10.011440 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:10.011385    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-02 14:03:08 +0000 UTC" deadline="2028-03-13 14:19:09.873429488 +0000 UTC"
Jul 03 14:08:10.011440 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:10.011439    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14856h10m59.861994392s"
Jul 03 14:08:10.052661 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:10.052523    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:10.052775 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:10.052730    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcksp" podUID="01b1d786-bec1-48da-ace5-ec49bda1a3e3"
Jul 03 14:08:10.059935 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:10.059903    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-frdqm" event={"ID":"e70ced91-d923-4563-a8b7-7b370f727843","Type":"ContainerStarted","Data":"d8b900f48ebad2e32f33afd8bfb0364664931b98f6af8d53eb9cfb419ae71686"}
Jul 03 14:08:10.061160 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:10.061121    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xk86w" event={"ID":"b3c91d5b-53e3-483c-a204-700513fc8648","Type":"ContainerStarted","Data":"7f020b463436cbb79a90a9e125e8d3a5884aa424e9c60f73dbd229ce5549c468"}
Jul 03 14:08:10.062133 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:10.062111    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tw77w" event={"ID":"a5e9d1f2-6f21-48c0-8046-9f95a5218539","Type":"ContainerStarted","Data":"9f66818747e8a3530146769af443417b02c2ddaeffc4737e369bea87a8a32b0e"}
Jul 03 14:08:10.063121 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:10.063098    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pz4j6" event={"ID":"c8841628-0752-46c1-8dd8-850c21157563","Type":"ContainerStarted","Data":"84d118874138949828aa65b97cca53cb276619e0422de32e44f67a14b08af0fb"}
Jul 03 14:08:10.064180 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:10.064155    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pl747" event={"ID":"c229a534-7dd6-44c1-88a9-4cd18b75369c","Type":"ContainerStarted","Data":"0bad49facf889c3129601ae02c02d2b374479f4b6bb354b82c472f53a253075c"}
Jul 03 14:08:10.067396 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:10.067373    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9k9sd" event={"ID":"11f255a7-6652-4c64-97b8-c980552c363f","Type":"ContainerStarted","Data":"ea2e24d6007b629068893f7554ac77ef6057b9a712edf85e58aa622191e9117b"}
Jul 03 14:08:10.069479 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:10.069448    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-233.ec2.internal" event={"ID":"61e1c42ac9a08ecfb3c8cc5e93139196","Type":"ContainerStarted","Data":"eac05f169924041b901769d3bb4b30a4ee9401348f269138a508b29c88a7917c"}
Jul 03 14:08:10.070524 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:10.070504    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m4t7h" event={"ID":"1072d3c3-729b-422d-8899-509c061281b0","Type":"ContainerStarted","Data":"1ce8b728033fa035cc5abdbab939d1d8921ceab6c9da1be4da9c31e0117d4a60"}
Jul 03 14:08:10.074589 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:10.074570    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7b795" event={"ID":"2df513c5-3c49-4255-9106-9e36d40d4fa1","Type":"ContainerStarted","Data":"c910d3fcb22188648c5e7bb3fa06b89d755e039f0da5664555c5f4554f20d0e7"}
Jul 03 14:08:10.077139 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:10.077117    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gq6s2" event={"ID":"4883da48-1d2e-4958-99a5-d35f60236c3d","Type":"ContainerStarted","Data":"6ae436bbcad5872832c8a9008f971e4dbe2c311b76ab36aa774a2a3cce7b6a8d"}
Jul 03 14:08:10.085713 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:10.085674    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-233.ec2.internal" podStartSLOduration=2.085661472 podStartE2EDuration="2.085661472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:08:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:08:10.085006469 +0000 UTC m=+3.535139920" watchObservedRunningTime="2026-07-03 14:08:10.085661472 +0000 UTC m=+3.535794911"
Jul 03 14:08:10.587435 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:10.586740    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28b96845-6419-4df3-bff3-777d753989ce-metrics-certs\") pod \"network-metrics-daemon-9dzdm\" (UID: \"28b96845-6419-4df3-bff3-777d753989ce\") " pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:08:10.587435 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:10.586950    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 14:08:10.587435 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:10.587034    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28b96845-6419-4df3-bff3-777d753989ce-metrics-certs podName:28b96845-6419-4df3-bff3-777d753989ce nodeName:}" failed. No retries permitted until 2026-07-03 14:08:12.587012285 +0000 UTC m=+6.037145710 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28b96845-6419-4df3-bff3-777d753989ce-metrics-certs") pod "network-metrics-daemon-9dzdm" (UID: "28b96845-6419-4df3-bff3-777d753989ce") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 14:08:10.688652 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:10.687863    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8w2v\" (UniqueName: \"kubernetes.io/projected/c899d91f-c2fe-432e-b726-4e0f3bab3351-kube-api-access-n8w2v\") pod \"network-check-target-4sq59\" (UID: \"c899d91f-c2fe-432e-b726-4e0f3bab3351\") " pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:08:10.688652 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:10.687945    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01b1d786-bec1-48da-ace5-ec49bda1a3e3-original-pull-secret\") pod \"global-pull-secret-syncer-rcksp\" (UID: \"01b1d786-bec1-48da-ace5-ec49bda1a3e3\") " pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:10.688652 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:10.688080    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 03 14:08:10.688652 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:10.688145    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01b1d786-bec1-48da-ace5-ec49bda1a3e3-original-pull-secret podName:01b1d786-bec1-48da-ace5-ec49bda1a3e3 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:12.68812396 +0000 UTC m=+6.138257380 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01b1d786-bec1-48da-ace5-ec49bda1a3e3-original-pull-secret") pod "global-pull-secret-syncer-rcksp" (UID: "01b1d786-bec1-48da-ace5-ec49bda1a3e3") : object "kube-system"/"original-pull-secret" not registered
Jul 03 14:08:10.688652 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:10.688539    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 03 14:08:10.688652 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:10.688561    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 03 14:08:10.688652 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:10.688574    2563 projected.go:196] Error preparing data for projected volume kube-api-access-n8w2v for pod openshift-network-diagnostics/network-check-target-4sq59: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 14:08:10.688652 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:10.688618    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c899d91f-c2fe-432e-b726-4e0f3bab3351-kube-api-access-n8w2v podName:c899d91f-c2fe-432e-b726-4e0f3bab3351 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:12.68860308 +0000 UTC m=+6.138736513 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-n8w2v" (UniqueName: "kubernetes.io/projected/c899d91f-c2fe-432e-b726-4e0f3bab3351-kube-api-access-n8w2v") pod "network-check-target-4sq59" (UID: "c899d91f-c2fe-432e-b726-4e0f3bab3351") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 14:08:11.011792 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:11.011692    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-02 14:03:08 +0000 UTC" deadline="2028-02-05 15:38:27.638310929 +0000 UTC"
Jul 03 14:08:11.011792 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:11.011736    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13969h30m16.626579832s"
Jul 03 14:08:11.054728 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:11.054615    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:08:11.054899 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:11.054736    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sq59" podUID="c899d91f-c2fe-432e-b726-4e0f3bab3351"
Jul 03 14:08:11.055167 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:11.055141    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:08:11.055272 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:11.055253    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dzdm" podUID="28b96845-6419-4df3-bff3-777d753989ce"
Jul 03 14:08:11.101761 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:11.101725    2563 generic.go:356] "Generic (PLEG): container finished" podID="d3d20294c54eebf1cdd45bd4f32b1e5e" containerID="c1e5b0ea78e437dce55faa1d6ce9b78d68bce189aff4ca1e138c59d8156425c0" exitCode=0
Jul 03 14:08:11.102339 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:11.102312    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-233.ec2.internal" event={"ID":"d3d20294c54eebf1cdd45bd4f32b1e5e","Type":"ContainerDied","Data":"c1e5b0ea78e437dce55faa1d6ce9b78d68bce189aff4ca1e138c59d8156425c0"}
Jul 03 14:08:12.053014 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:12.052983    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:12.053484 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:12.053141    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcksp" podUID="01b1d786-bec1-48da-ace5-ec49bda1a3e3"
Jul 03 14:08:12.119645 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:12.119609    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-233.ec2.internal" event={"ID":"d3d20294c54eebf1cdd45bd4f32b1e5e","Type":"ContainerStarted","Data":"29cdd42b5ea799e4095fea8afeec5db2a6c01394e2309e14d648aa40d1c7fdeb"}
Jul 03 14:08:12.605511 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:12.605470    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28b96845-6419-4df3-bff3-777d753989ce-metrics-certs\") pod \"network-metrics-daemon-9dzdm\" (UID: \"28b96845-6419-4df3-bff3-777d753989ce\") " pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:08:12.605708 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:12.605641    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 14:08:12.605773 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:12.605709    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28b96845-6419-4df3-bff3-777d753989ce-metrics-certs podName:28b96845-6419-4df3-bff3-777d753989ce nodeName:}" failed. No retries permitted until 2026-07-03 14:08:16.60569113 +0000 UTC m=+10.055824551 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28b96845-6419-4df3-bff3-777d753989ce-metrics-certs") pod "network-metrics-daemon-9dzdm" (UID: "28b96845-6419-4df3-bff3-777d753989ce") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 14:08:12.706614 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:12.706571    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8w2v\" (UniqueName: \"kubernetes.io/projected/c899d91f-c2fe-432e-b726-4e0f3bab3351-kube-api-access-n8w2v\") pod \"network-check-target-4sq59\" (UID: \"c899d91f-c2fe-432e-b726-4e0f3bab3351\") " pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:08:12.706614 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:12.706661    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01b1d786-bec1-48da-ace5-ec49bda1a3e3-original-pull-secret\") pod \"global-pull-secret-syncer-rcksp\" (UID: \"01b1d786-bec1-48da-ace5-ec49bda1a3e3\") " pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:12.707097 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:12.706788    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 03 14:08:12.707097 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:12.706850    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01b1d786-bec1-48da-ace5-ec49bda1a3e3-original-pull-secret podName:01b1d786-bec1-48da-ace5-ec49bda1a3e3 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:16.706832796 +0000 UTC m=+10.156966216 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01b1d786-bec1-48da-ace5-ec49bda1a3e3-original-pull-secret") pod "global-pull-secret-syncer-rcksp" (UID: "01b1d786-bec1-48da-ace5-ec49bda1a3e3") : object "kube-system"/"original-pull-secret" not registered
Jul 03 14:08:12.707499 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:12.707287    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 03 14:08:12.707499 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:12.707309    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 03 14:08:12.707499 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:12.707321    2563 projected.go:196] Error preparing data for projected volume kube-api-access-n8w2v for pod openshift-network-diagnostics/network-check-target-4sq59: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 14:08:12.707499 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:12.707367    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c899d91f-c2fe-432e-b726-4e0f3bab3351-kube-api-access-n8w2v podName:c899d91f-c2fe-432e-b726-4e0f3bab3351 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:16.707352237 +0000 UTC m=+10.157485661 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-n8w2v" (UniqueName: "kubernetes.io/projected/c899d91f-c2fe-432e-b726-4e0f3bab3351-kube-api-access-n8w2v") pod "network-check-target-4sq59" (UID: "c899d91f-c2fe-432e-b726-4e0f3bab3351") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 14:08:13.053039 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:13.052665    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:08:13.053039 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:13.052795    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sq59" podUID="c899d91f-c2fe-432e-b726-4e0f3bab3351"
Jul 03 14:08:13.053878 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:13.053659    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:08:13.053878 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:13.053807    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dzdm" podUID="28b96845-6419-4df3-bff3-777d753989ce"
Jul 03 14:08:14.052606 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:14.052545    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:14.052807 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:14.052673    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcksp" podUID="01b1d786-bec1-48da-ace5-ec49bda1a3e3"
Jul 03 14:08:15.052716 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:15.052681    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:08:15.053122 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:15.052681    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:08:15.053122 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:15.052828    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dzdm" podUID="28b96845-6419-4df3-bff3-777d753989ce"
Jul 03 14:08:15.053122 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:15.052905    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sq59" podUID="c899d91f-c2fe-432e-b726-4e0f3bab3351"
Jul 03 14:08:16.052436 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:16.052356    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:16.052602 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:16.052490    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcksp" podUID="01b1d786-bec1-48da-ace5-ec49bda1a3e3"
Jul 03 14:08:16.637167 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:16.637132    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28b96845-6419-4df3-bff3-777d753989ce-metrics-certs\") pod \"network-metrics-daemon-9dzdm\" (UID: \"28b96845-6419-4df3-bff3-777d753989ce\") " pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:08:16.637617 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:16.637269    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 14:08:16.637617 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:16.637351    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28b96845-6419-4df3-bff3-777d753989ce-metrics-certs podName:28b96845-6419-4df3-bff3-777d753989ce nodeName:}" failed. No retries permitted until 2026-07-03 14:08:24.637330229 +0000 UTC m=+18.087463651 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28b96845-6419-4df3-bff3-777d753989ce-metrics-certs") pod "network-metrics-daemon-9dzdm" (UID: "28b96845-6419-4df3-bff3-777d753989ce") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 14:08:16.737765 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:16.737518    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01b1d786-bec1-48da-ace5-ec49bda1a3e3-original-pull-secret\") pod \"global-pull-secret-syncer-rcksp\" (UID: \"01b1d786-bec1-48da-ace5-ec49bda1a3e3\") " pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:16.737765 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:16.737596    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8w2v\" (UniqueName: \"kubernetes.io/projected/c899d91f-c2fe-432e-b726-4e0f3bab3351-kube-api-access-n8w2v\") pod \"network-check-target-4sq59\" (UID: \"c899d91f-c2fe-432e-b726-4e0f3bab3351\") " pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:08:16.737765 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:16.737696    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 03 14:08:16.737765 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:16.737712    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 03 14:08:16.737765 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:16.737730    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 03 14:08:16.737765 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:16.737743    2563 projected.go:196] Error preparing data for projected volume kube-api-access-n8w2v for pod openshift-network-diagnostics/network-check-target-4sq59: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 14:08:16.737765 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:16.737770    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01b1d786-bec1-48da-ace5-ec49bda1a3e3-original-pull-secret podName:01b1d786-bec1-48da-ace5-ec49bda1a3e3 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:24.73774885 +0000 UTC m=+18.187882274 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01b1d786-bec1-48da-ace5-ec49bda1a3e3-original-pull-secret") pod "global-pull-secret-syncer-rcksp" (UID: "01b1d786-bec1-48da-ace5-ec49bda1a3e3") : object "kube-system"/"original-pull-secret" not registered
Jul 03 14:08:16.738260 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:16.737791    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c899d91f-c2fe-432e-b726-4e0f3bab3351-kube-api-access-n8w2v podName:c899d91f-c2fe-432e-b726-4e0f3bab3351 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:24.737777744 +0000 UTC m=+18.187911185 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-n8w2v" (UniqueName: "kubernetes.io/projected/c899d91f-c2fe-432e-b726-4e0f3bab3351-kube-api-access-n8w2v") pod "network-check-target-4sq59" (UID: "c899d91f-c2fe-432e-b726-4e0f3bab3351") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 14:08:17.053948 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:17.053874    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:08:17.054128 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:17.054005    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sq59" podUID="c899d91f-c2fe-432e-b726-4e0f3bab3351"
Jul 03 14:08:17.054239 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:17.054219    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:08:17.054356 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:17.054335    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dzdm" podUID="28b96845-6419-4df3-bff3-777d753989ce"
Jul 03 14:08:18.052867 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:18.052786    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:18.053362 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:18.052924    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcksp" podUID="01b1d786-bec1-48da-ace5-ec49bda1a3e3"
Jul 03 14:08:19.052761 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:19.052722    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:08:19.052941 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:19.052893    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sq59" podUID="c899d91f-c2fe-432e-b726-4e0f3bab3351"
Jul 03 14:08:19.053311 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:19.053269    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:08:19.053405 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:19.053380    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dzdm" podUID="28b96845-6419-4df3-bff3-777d753989ce"
Jul 03 14:08:20.053229 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:20.053188    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:20.053691 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:20.053328    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcksp" podUID="01b1d786-bec1-48da-ace5-ec49bda1a3e3"
Jul 03 14:08:21.052261 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:21.052225    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:08:21.052446 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:21.052342    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sq59" podUID="c899d91f-c2fe-432e-b726-4e0f3bab3351"
Jul 03 14:08:21.052446 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:21.052411    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:08:21.052568 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:21.052542    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dzdm" podUID="28b96845-6419-4df3-bff3-777d753989ce"
Jul 03 14:08:22.053183 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:22.053153    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:22.053680 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:22.053270    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcksp" podUID="01b1d786-bec1-48da-ace5-ec49bda1a3e3"
Jul 03 14:08:23.052669 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:23.052636    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:08:23.052842 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:23.052636    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:08:23.052842 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:23.052773    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dzdm" podUID="28b96845-6419-4df3-bff3-777d753989ce"
Jul 03 14:08:23.052988 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:23.052839    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sq59" podUID="c899d91f-c2fe-432e-b726-4e0f3bab3351"
Jul 03 14:08:24.052531 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:24.052490    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:24.053137 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:24.052625    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcksp" podUID="01b1d786-bec1-48da-ace5-ec49bda1a3e3"
Jul 03 14:08:24.698473 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:24.698437    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28b96845-6419-4df3-bff3-777d753989ce-metrics-certs\") pod \"network-metrics-daemon-9dzdm\" (UID: \"28b96845-6419-4df3-bff3-777d753989ce\") " pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:08:24.698655 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:24.698613    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 14:08:24.698716 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:24.698692    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28b96845-6419-4df3-bff3-777d753989ce-metrics-certs podName:28b96845-6419-4df3-bff3-777d753989ce nodeName:}" failed. No retries permitted until 2026-07-03 14:08:40.698670926 +0000 UTC m=+34.148804345 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28b96845-6419-4df3-bff3-777d753989ce-metrics-certs") pod "network-metrics-daemon-9dzdm" (UID: "28b96845-6419-4df3-bff3-777d753989ce") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 14:08:24.799173 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:24.799140    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01b1d786-bec1-48da-ace5-ec49bda1a3e3-original-pull-secret\") pod \"global-pull-secret-syncer-rcksp\" (UID: \"01b1d786-bec1-48da-ace5-ec49bda1a3e3\") " pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:24.799329 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:24.799194    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8w2v\" (UniqueName: \"kubernetes.io/projected/c899d91f-c2fe-432e-b726-4e0f3bab3351-kube-api-access-n8w2v\") pod \"network-check-target-4sq59\" (UID: \"c899d91f-c2fe-432e-b726-4e0f3bab3351\") " pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:08:24.799329 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:24.799308    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 03 14:08:24.799425 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:24.799336    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 03 14:08:24.799425 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:24.799354    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 03 14:08:24.799425 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:24.799368    2563 projected.go:196] Error preparing data for projected volume kube-api-access-n8w2v for pod openshift-network-diagnostics/network-check-target-4sq59: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 14:08:24.799425 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:24.799380    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01b1d786-bec1-48da-ace5-ec49bda1a3e3-original-pull-secret podName:01b1d786-bec1-48da-ace5-ec49bda1a3e3 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:40.79935923 +0000 UTC m=+34.249492649 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01b1d786-bec1-48da-ace5-ec49bda1a3e3-original-pull-secret") pod "global-pull-secret-syncer-rcksp" (UID: "01b1d786-bec1-48da-ace5-ec49bda1a3e3") : object "kube-system"/"original-pull-secret" not registered
Jul 03 14:08:24.799425 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:24.799412    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c899d91f-c2fe-432e-b726-4e0f3bab3351-kube-api-access-n8w2v podName:c899d91f-c2fe-432e-b726-4e0f3bab3351 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:40.7993954 +0000 UTC m=+34.249528820 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-n8w2v" (UniqueName: "kubernetes.io/projected/c899d91f-c2fe-432e-b726-4e0f3bab3351-kube-api-access-n8w2v") pod "network-check-target-4sq59" (UID: "c899d91f-c2fe-432e-b726-4e0f3bab3351") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 14:08:25.052989 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:25.052893    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:08:25.053424 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:25.052893    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:08:25.053424 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:25.053043    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dzdm" podUID="28b96845-6419-4df3-bff3-777d753989ce"
Jul 03 14:08:25.053424 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:25.053101    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sq59" podUID="c899d91f-c2fe-432e-b726-4e0f3bab3351"
Jul 03 14:08:26.052889 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:26.052854    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:26.053099 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:26.052994    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcksp" podUID="01b1d786-bec1-48da-ace5-ec49bda1a3e3"
Jul 03 14:08:27.054001 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:27.053952    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:08:27.054435 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:27.054095    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dzdm" podUID="28b96845-6419-4df3-bff3-777d753989ce"
Jul 03 14:08:27.054435 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:27.054170    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:08:27.054435 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:27.054256    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sq59" podUID="c899d91f-c2fe-432e-b726-4e0f3bab3351"
Jul 03 14:08:28.052241 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:28.052207    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:28.052434 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:28.052334    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcksp" podUID="01b1d786-bec1-48da-ace5-ec49bda1a3e3"
Jul 03 14:08:29.053119 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:29.052788    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:08:29.053909 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:29.052849    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:08:29.053909 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:29.053218    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dzdm" podUID="28b96845-6419-4df3-bff3-777d753989ce"
Jul 03 14:08:29.053909 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:29.053280    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sq59" podUID="c899d91f-c2fe-432e-b726-4e0f3bab3351"
Jul 03 14:08:29.156202 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:29.156172    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-frdqm" event={"ID":"e70ced91-d923-4563-a8b7-7b370f727843","Type":"ContainerStarted","Data":"4207b3059c67523ed8cba6c81188ab224d8225f83294faafbf17d0f23204c9ef"}
Jul 03 14:08:29.157907 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:29.157884    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xk86w" event={"ID":"b3c91d5b-53e3-483c-a204-700513fc8648","Type":"ContainerStarted","Data":"d742adb4880e38073156c33990e51f428491782259d70f73ef8bedc159494a73"}
Jul 03 14:08:29.158025 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:29.157957    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xk86w" event={"ID":"b3c91d5b-53e3-483c-a204-700513fc8648","Type":"ContainerStarted","Data":"b4b72c6b46986d751f0c4a7f2cecc34f18709dc99bc42da94c33178cb2b5e6fe"}
Jul 03 14:08:29.158092 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:29.158024    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xk86w" event={"ID":"b3c91d5b-53e3-483c-a204-700513fc8648","Type":"ContainerStarted","Data":"d4b70973274441d8660eef9515ca9059e898cd116a59e69f63850011e1dca83b"}
Jul 03 14:08:29.159152 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:29.159132    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tw77w" event={"ID":"a5e9d1f2-6f21-48c0-8046-9f95a5218539","Type":"ContainerStarted","Data":"54aaaeb738df12dbabc14f5133b6102012f4d92ef0eec484bee513379215bdae"}
Jul 03 14:08:29.160441 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:29.160420    2563 generic.go:356] "Generic (PLEG): container finished" podID="c8841628-0752-46c1-8dd8-850c21157563" containerID="5593f53a37358b61d6d622f0a42e080a6f3831a5895d3a461a1fc0716a0e23ab" exitCode=0
Jul 03 14:08:29.160535 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:29.160498    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pz4j6" event={"ID":"c8841628-0752-46c1-8dd8-850c21157563","Type":"ContainerDied","Data":"5593f53a37358b61d6d622f0a42e080a6f3831a5895d3a461a1fc0716a0e23ab"}
Jul 03 14:08:29.161991 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:29.161919    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9k9sd" event={"ID":"11f255a7-6652-4c64-97b8-c980552c363f","Type":"ContainerStarted","Data":"f9600efd1a460646c5aa3d352f1a698af68b5f2ece6da7414dc582bd387a1e3b"}
Jul 03 14:08:29.163307 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:29.163287    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m4t7h" event={"ID":"1072d3c3-729b-422d-8899-509c061281b0","Type":"ContainerStarted","Data":"cf6450c5e7359b33e504a6ff7e8f4a80dc94e991953fdcf8aa40fbc5aac1a8cf"}
Jul 03 14:08:29.164901 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:29.164875    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7b795" event={"ID":"2df513c5-3c49-4255-9106-9e36d40d4fa1","Type":"ContainerStarted","Data":"f312e791d5aeea1db90fd51a2b1ae78455cbdcbdaf228a3ce262ad9c7c8e9e51"}
Jul 03 14:08:29.166426 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:29.166405    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gq6s2" event={"ID":"4883da48-1d2e-4958-99a5-d35f60236c3d","Type":"ContainerStarted","Data":"3cb13e6412a0de66ed65515e92b012f4744d109c010118379a5d59fadbcb29c4"}
Jul 03 14:08:29.185387 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:29.185344    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-233.ec2.internal" podStartSLOduration=21.185330745999998 podStartE2EDuration="21.185330746s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:08:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:08:12.13892451 +0000 UTC m=+5.589057951" watchObservedRunningTime="2026-07-03 14:08:29.185330746 +0000 UTC m=+22.635464185"
Jul 03 14:08:29.203198 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:29.203154    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-9k9sd" podStartSLOduration=3.802960717 podStartE2EDuration="22.203137577s" totalImagesPullingTime="18.40017686s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:08:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:08:29.203051431 +0000 UTC m=+22.653184889" watchObservedRunningTime="2026-07-03 14:08:29.203137577 +0000 UTC m=+22.653271016"
Jul 03 14:08:29.257267 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:29.257221    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-m4t7h" podStartSLOduration=3.507567286 podStartE2EDuration="22.25720668s" totalImagesPullingTime="18.749639394s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:08:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:08:29.256767109 +0000 UTC m=+22.706900549" watchObservedRunningTime="2026-07-03 14:08:29.25720668 +0000 UTC m=+22.707340120"
Jul 03 14:08:29.257480 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:29.257463    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-frdqm" podStartSLOduration=3.458390156 podStartE2EDuration="22.257457839s" totalImagesPullingTime="18.799067683s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:08:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:08:29.230796884 +0000 UTC m=+22.680930332" watchObservedRunningTime="2026-07-03 14:08:29.257457839 +0000 UTC m=+22.707591279"
Jul 03 14:08:29.302678 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:29.302632    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-tw77w" podStartSLOduration=7.690468715 podStartE2EDuration="22.302615791s" totalImagesPullingTime="14.612147076s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:08:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:08:29.280122742 +0000 UTC m=+22.730256182" watchObservedRunningTime="2026-07-03 14:08:29.302615791 +0000 UTC m=+22.752749231"
Jul 03 14:08:30.052367 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:30.052333    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:30.052554 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:30.052443    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcksp" podUID="01b1d786-bec1-48da-ace5-ec49bda1a3e3"
Jul 03 14:08:30.171035 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:30.171004    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xk86w" event={"ID":"b3c91d5b-53e3-483c-a204-700513fc8648","Type":"ContainerStarted","Data":"3381057487b93a7f1e05326fb379d3489d425c8748c45031c426394b51f4150c"}
Jul 03 14:08:30.171506 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:30.171068    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xk86w" event={"ID":"b3c91d5b-53e3-483c-a204-700513fc8648","Type":"ContainerStarted","Data":"491d677b85f442d06ba86f42905e674f3c443c184589962da77426bc55ec025b"}
Jul 03 14:08:30.171506 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:30.171098    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xk86w" event={"ID":"b3c91d5b-53e3-483c-a204-700513fc8648","Type":"ContainerStarted","Data":"600eb83efa8807de6faffefc02fc0c85dc334afabe20392ab8630d6b3cc74866"}
Jul 03 14:08:30.359525 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:30.359497    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 03 14:08:31.029843 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:31.029682    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-03T14:08:30.359519552Z","UUID":"2b41b33e-5996-450d-a17f-b8fef549b65e","Handler":null,"Name":"","Endpoint":""}
Jul 03 14:08:31.031318 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:31.031295    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 03 14:08:31.031448 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:31.031334    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 03 14:08:31.052584 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:31.052550    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:08:31.052756 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:31.052542    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:08:31.052756 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:31.052690    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sq59" podUID="c899d91f-c2fe-432e-b726-4e0f3bab3351"
Jul 03 14:08:31.052851 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:31.052762    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dzdm" podUID="28b96845-6419-4df3-bff3-777d753989ce"
Jul 03 14:08:31.175506 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:31.175464    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gq6s2" event={"ID":"4883da48-1d2e-4958-99a5-d35f60236c3d","Type":"ContainerStarted","Data":"28064386e46f138f14be82bb7d94e0885b44c3ef948fd850dfff88254a11aa3a"}
Jul 03 14:08:31.177017 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:31.176986    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pl747" event={"ID":"c229a534-7dd6-44c1-88a9-4cd18b75369c","Type":"ContainerStarted","Data":"c4aeeec7940f89aa78a0c04e422e93e7e62acee7d7016c626ec21cf19aa9176c"}
Jul 03 14:08:31.193840 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:31.193799    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7b795" podStartSLOduration=5.4465315610000005 podStartE2EDuration="24.193781664s" totalImagesPullingTime="18.747250103s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:08:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:08:29.3037358 +0000 UTC m=+22.753869241" watchObservedRunningTime="2026-07-03 14:08:31.193781664 +0000 UTC m=+24.643915124"
Jul 03 14:08:31.194200 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:31.194172    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-pl747" podStartSLOduration=5.4414291630000005 podStartE2EDuration="24.194160935s" totalImagesPullingTime="18.752731772s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:08:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:08:31.193623911 +0000 UTC m=+24.643757351" watchObservedRunningTime="2026-07-03 14:08:31.194160935 +0000 UTC m=+24.644294375"
Jul 03 14:08:31.222924 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:31.222882    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-tw77w"
Jul 03 14:08:31.223596 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:31.223575    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-tw77w"
Jul 03 14:08:32.052580 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:32.052353    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:32.052852 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:32.052693    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcksp" podUID="01b1d786-bec1-48da-ace5-ec49bda1a3e3"
Jul 03 14:08:32.181745 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:32.181705    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gq6s2" event={"ID":"4883da48-1d2e-4958-99a5-d35f60236c3d","Type":"ContainerStarted","Data":"b6b1b823f74ce0a72de292382c38309829e0289d5463a223d0ccaa6db4d32190"}
Jul 03 14:08:32.185597 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:32.185572    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xk86w" event={"ID":"b3c91d5b-53e3-483c-a204-700513fc8648","Type":"ContainerStarted","Data":"9c63ac4479325acda43bb7c0bba20002d1fa84d230419be6fc86c0c323f28b0e"}
Jul 03 14:08:32.185912 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:32.185868    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-tw77w"
Jul 03 14:08:32.186351 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:32.186332    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-tw77w"
Jul 03 14:08:32.202347 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:32.202308    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gq6s2" podStartSLOduration=4.327267896 podStartE2EDuration="25.202294867s" totalImagesPullingTime="20.875026971s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:08:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:08:32.202183172 +0000 UTC m=+25.652316613" watchObservedRunningTime="2026-07-03 14:08:32.202294867 +0000 UTC m=+25.652428306"
Jul 03 14:08:33.053247 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:33.053211    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:08:33.053505 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:33.053336    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sq59" podUID="c899d91f-c2fe-432e-b726-4e0f3bab3351"
Jul 03 14:08:33.053505 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:33.053391    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:08:33.053505 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:33.053496    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dzdm" podUID="28b96845-6419-4df3-bff3-777d753989ce"
Jul 03 14:08:34.053166 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:34.052915    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:34.053953 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:34.053230    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcksp" podUID="01b1d786-bec1-48da-ace5-ec49bda1a3e3"
Jul 03 14:08:34.193577 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:34.193536    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xk86w" event={"ID":"b3c91d5b-53e3-483c-a204-700513fc8648","Type":"ContainerStarted","Data":"733af1a5d34ded672f01a302b40219544ab95ccbd0843d57433c4820ec67afbd"}
Jul 03 14:08:34.224509 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:34.224455    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xk86w" podStartSLOduration=8.415123137 podStartE2EDuration="27.224438244s" totalImagesPullingTime="18.809315107s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:08:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:08:34.223878629 +0000 UTC m=+27.674012082" watchObservedRunningTime="2026-07-03 14:08:34.224438244 +0000 UTC m=+27.674571683"
Jul 03 14:08:35.055193 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:35.054548    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:08:35.055193 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:35.054679    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dzdm" podUID="28b96845-6419-4df3-bff3-777d753989ce"
Jul 03 14:08:35.055193 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:35.055059    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:08:35.055193 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:35.055141    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sq59" podUID="c899d91f-c2fe-432e-b726-4e0f3bab3351"
Jul 03 14:08:35.195549 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:35.195510    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:35.195710 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:35.195589    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:35.195710 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:35.195640    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:35.211602 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:35.211571    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:35.211733 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:35.211721    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:08:35.907013 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:35.906896    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rcksp"]
Jul 03 14:08:35.907166 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:35.907109    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:35.907241 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:35.907219    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcksp" podUID="01b1d786-bec1-48da-ace5-ec49bda1a3e3"
Jul 03 14:08:35.909751 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:35.909726    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9dzdm"]
Jul 03 14:08:35.909857 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:35.909836    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:08:35.909960 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:35.909933    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dzdm" podUID="28b96845-6419-4df3-bff3-777d753989ce"
Jul 03 14:08:35.915476 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:35.915451    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4sq59"]
Jul 03 14:08:35.915586 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:35.915538    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:08:35.915641 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:35.915616    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sq59" podUID="c899d91f-c2fe-432e-b726-4e0f3bab3351"
Jul 03 14:08:38.053228 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:38.053141    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:08:38.053228 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:38.053183    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:08:38.053228 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:38.053183    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:38.053681 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:38.053248    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dzdm" podUID="28b96845-6419-4df3-bff3-777d753989ce"
Jul 03 14:08:38.053681 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:38.053362    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sq59" podUID="c899d91f-c2fe-432e-b726-4e0f3bab3351"
Jul 03 14:08:38.053681 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:38.053450    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcksp" podUID="01b1d786-bec1-48da-ace5-ec49bda1a3e3"
Jul 03 14:08:38.204215 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:38.204030    2563 generic.go:356] "Generic (PLEG): container finished" podID="c8841628-0752-46c1-8dd8-850c21157563" containerID="1c420ebcbf641a4355ed61339c9826e87b419a2dacf831f1432079744a15b702" exitCode=0
Jul 03 14:08:38.204215 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:38.204080    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pz4j6" event={"ID":"c8841628-0752-46c1-8dd8-850c21157563","Type":"ContainerDied","Data":"1c420ebcbf641a4355ed61339c9826e87b419a2dacf831f1432079744a15b702"}
Jul 03 14:08:39.208287 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:39.208196    2563 generic.go:356] "Generic (PLEG): container finished" podID="c8841628-0752-46c1-8dd8-850c21157563" containerID="af1b02a8350a1d7a9010ebee738aa1772c9dbe6143fd887039878529d5e1cf1c" exitCode=0
Jul 03 14:08:39.208287 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:39.208243    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pz4j6" event={"ID":"c8841628-0752-46c1-8dd8-850c21157563","Type":"ContainerDied","Data":"af1b02a8350a1d7a9010ebee738aa1772c9dbe6143fd887039878529d5e1cf1c"}
Jul 03 14:08:40.053214 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:40.053125    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:40.053383 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:40.053218    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:08:40.053383 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:40.053221    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcksp" podUID="01b1d786-bec1-48da-ace5-ec49bda1a3e3"
Jul 03 14:08:40.053383 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:40.053308    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4sq59" podUID="c899d91f-c2fe-432e-b726-4e0f3bab3351"
Jul 03 14:08:40.053383 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:40.053224    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:08:40.053545 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:40.053413    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dzdm" podUID="28b96845-6419-4df3-bff3-777d753989ce"
Jul 03 14:08:40.213033 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:40.212999    2563 generic.go:356] "Generic (PLEG): container finished" podID="c8841628-0752-46c1-8dd8-850c21157563" containerID="e82b4bde1ef5b0d169337145b851758ccff27411e0298c4afb748ef4e0295e44" exitCode=0
Jul 03 14:08:40.213405 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:40.213048    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pz4j6" event={"ID":"c8841628-0752-46c1-8dd8-850c21157563","Type":"ContainerDied","Data":"e82b4bde1ef5b0d169337145b851758ccff27411e0298c4afb748ef4e0295e44"}
Jul 03 14:08:40.715270 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:40.715232    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28b96845-6419-4df3-bff3-777d753989ce-metrics-certs\") pod \"network-metrics-daemon-9dzdm\" (UID: \"28b96845-6419-4df3-bff3-777d753989ce\") " pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:08:40.715453 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:40.715393    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 14:08:40.715499 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:40.715470    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28b96845-6419-4df3-bff3-777d753989ce-metrics-certs podName:28b96845-6419-4df3-bff3-777d753989ce nodeName:}" failed. No retries permitted until 2026-07-03 14:09:12.71545125 +0000 UTC m=+66.165584668 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28b96845-6419-4df3-bff3-777d753989ce-metrics-certs") pod "network-metrics-daemon-9dzdm" (UID: "28b96845-6419-4df3-bff3-777d753989ce") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 03 14:08:40.816492 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:40.816455    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01b1d786-bec1-48da-ace5-ec49bda1a3e3-original-pull-secret\") pod \"global-pull-secret-syncer-rcksp\" (UID: \"01b1d786-bec1-48da-ace5-ec49bda1a3e3\") " pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:40.816783 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:40.816512    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8w2v\" (UniqueName: \"kubernetes.io/projected/c899d91f-c2fe-432e-b726-4e0f3bab3351-kube-api-access-n8w2v\") pod \"network-check-target-4sq59\" (UID: \"c899d91f-c2fe-432e-b726-4e0f3bab3351\") " pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:08:40.816783 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:40.816603    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 03 14:08:40.816783 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:40.816637    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 03 14:08:40.816783 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:40.816657    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 03 14:08:40.816783 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:40.816668    2563 projected.go:196] Error preparing data for projected volume kube-api-access-n8w2v for pod openshift-network-diagnostics/network-check-target-4sq59: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 14:08:40.816783 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:40.816669    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01b1d786-bec1-48da-ace5-ec49bda1a3e3-original-pull-secret podName:01b1d786-bec1-48da-ace5-ec49bda1a3e3 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:12.816654968 +0000 UTC m=+66.266788389 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01b1d786-bec1-48da-ace5-ec49bda1a3e3-original-pull-secret") pod "global-pull-secret-syncer-rcksp" (UID: "01b1d786-bec1-48da-ace5-ec49bda1a3e3") : object "kube-system"/"original-pull-secret" not registered
Jul 03 14:08:40.816783 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:40.816716    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c899d91f-c2fe-432e-b726-4e0f3bab3351-kube-api-access-n8w2v podName:c899d91f-c2fe-432e-b726-4e0f3bab3351 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:12.816699291 +0000 UTC m=+66.266832712 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-n8w2v" (UniqueName: "kubernetes.io/projected/c899d91f-c2fe-432e-b726-4e0f3bab3351-kube-api-access-n8w2v") pod "network-check-target-4sq59" (UID: "c899d91f-c2fe-432e-b726-4e0f3bab3351") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 03 14:08:41.359882 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.359852    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-233.ec2.internal" event="NodeReady"
Jul 03 14:08:41.360334 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.360005    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 03 14:08:41.403534 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.403498    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"]
Jul 03 14:08:41.406304 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.406284    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:08:41.412929 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.412906    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 03 14:08:41.413746 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.413589    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 03 14:08:41.413746 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.413621    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 03 14:08:41.413746 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.413663    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-scvmw\""
Jul 03 14:08:41.418326 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.418306    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 03 14:08:41.430300 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.430272    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"]
Jul 03 14:08:41.436233 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.433803    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bf7t2"]
Jul 03 14:08:41.439639 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.439379    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bf7t2"
Jul 03 14:08:41.440190 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.440171    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-srkff"]
Jul 03 14:08:41.442840 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.442825    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-srkff"
Jul 03 14:08:41.445387 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.445361    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 03 14:08:41.445727 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.445708    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wf6pg\""
Jul 03 14:08:41.445838 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.445769    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 03 14:08:41.446218 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.446098    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 03 14:08:41.446384 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.446360    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 03 14:08:41.446723 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.446627    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 03 14:08:41.446723 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.446642    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7fg4h\""
Jul 03 14:08:41.450490 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.450473    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bf7t2"]
Jul 03 14:08:41.456059 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.456037    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-srkff"]
Jul 03 14:08:41.520223 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.520136    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-trusted-ca\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:08:41.520223 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.520180    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-bound-sa-token\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:08:41.520223 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.520209    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e58a995c-7842-4473-b890-7c93a46980f3-metrics-tls\") pod \"dns-default-bf7t2\" (UID: \"e58a995c-7842-4473-b890-7c93a46980f3\") " pod="openshift-dns/dns-default-bf7t2"
Jul 03 14:08:41.520493 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.520267    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-ca-trust-extracted\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:08:41.520493 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.520307    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-certificates\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:08:41.520493 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.520330    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/620719fe-f6f2-4310-9f6e-0f351f68264c-cert\") pod \"ingress-canary-srkff\" (UID: \"620719fe-f6f2-4310-9f6e-0f351f68264c\") " pod="openshift-ingress-canary/ingress-canary-srkff"
Jul 03 14:08:41.520493 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.520344    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e58a995c-7842-4473-b890-7c93a46980f3-config-volume\") pod \"dns-default-bf7t2\" (UID: \"e58a995c-7842-4473-b890-7c93a46980f3\") " pod="openshift-dns/dns-default-bf7t2"
Jul 03 14:08:41.520493 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.520384    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvrgl\" (UniqueName: \"kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-kube-api-access-wvrgl\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:08:41.520493 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.520464    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-installation-pull-secrets\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:08:41.520845 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.520513    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-image-registry-private-configuration\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:08:41.520845 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.520546    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbg24\" (UniqueName: \"kubernetes.io/projected/620719fe-f6f2-4310-9f6e-0f351f68264c-kube-api-access-mbg24\") pod \"ingress-canary-srkff\" (UID: \"620719fe-f6f2-4310-9f6e-0f351f68264c\") " pod="openshift-ingress-canary/ingress-canary-srkff"
Jul 03 14:08:41.520845 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.520573    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-tls\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:08:41.520845 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.520599    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e58a995c-7842-4473-b890-7c93a46980f3-tmp-dir\") pod \"dns-default-bf7t2\" (UID: \"e58a995c-7842-4473-b890-7c93a46980f3\") " pod="openshift-dns/dns-default-bf7t2"
Jul 03 14:08:41.520845 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.520746    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhk4f\" (UniqueName: \"kubernetes.io/projected/e58a995c-7842-4473-b890-7c93a46980f3-kube-api-access-mhk4f\") pod \"dns-default-bf7t2\" (UID: \"e58a995c-7842-4473-b890-7c93a46980f3\") " pod="openshift-dns/dns-default-bf7t2"
Jul 03 14:08:41.621422 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.621382    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-trusted-ca\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:08:41.621422 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.621422    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-bound-sa-token\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:08:41.621663 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.621442    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e58a995c-7842-4473-b890-7c93a46980f3-metrics-tls\") pod \"dns-default-bf7t2\" (UID: \"e58a995c-7842-4473-b890-7c93a46980f3\") " pod="openshift-dns/dns-default-bf7t2"
Jul 03 14:08:41.621663 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.621463    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-ca-trust-extracted\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:08:41.621663 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.621480    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-certificates\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:08:41.621663 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.621498    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/620719fe-f6f2-4310-9f6e-0f351f68264c-cert\") pod \"ingress-canary-srkff\" (UID: \"620719fe-f6f2-4310-9f6e-0f351f68264c\") " pod="openshift-ingress-canary/ingress-canary-srkff"
Jul 03 14:08:41.621663 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:41.621567    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 03 14:08:41.621663 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:41.621644    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e58a995c-7842-4473-b890-7c93a46980f3-metrics-tls podName:e58a995c-7842-4473-b890-7c93a46980f3 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:42.121627595 +0000 UTC m=+35.571761016 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e58a995c-7842-4473-b890-7c93a46980f3-metrics-tls") pod "dns-default-bf7t2" (UID: "e58a995c-7842-4473-b890-7c93a46980f3") : secret "dns-default-metrics-tls" not found
Jul 03 14:08:41.621997 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.621711    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e58a995c-7842-4473-b890-7c93a46980f3-config-volume\") pod \"dns-default-bf7t2\" (UID: \"e58a995c-7842-4473-b890-7c93a46980f3\") " pod="openshift-dns/dns-default-bf7t2"
Jul 03 14:08:41.621997 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.621764    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvrgl\" (UniqueName: \"kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-kube-api-access-wvrgl\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:08:41.621997 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.621819    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-installation-pull-secrets\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:08:41.621997 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.621859    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-image-registry-private-configuration\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:08:41.621997 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.621893    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbg24\" (UniqueName: \"kubernetes.io/projected/620719fe-f6f2-4310-9f6e-0f351f68264c-kube-api-access-mbg24\") pod \"ingress-canary-srkff\" (UID: \"620719fe-f6f2-4310-9f6e-0f351f68264c\") " pod="openshift-ingress-canary/ingress-canary-srkff"
Jul 03 14:08:41.621997 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.621902    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-ca-trust-extracted\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:08:41.621997 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.621917    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-tls\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:08:41.621997 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.621956    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e58a995c-7842-4473-b890-7c93a46980f3-tmp-dir\") pod \"dns-default-bf7t2\" (UID: \"e58a995c-7842-4473-b890-7c93a46980f3\") " pod="openshift-dns/dns-default-bf7t2"
Jul 03 14:08:41.622390 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:41.622028    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 03 14:08:41.622390 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.622036    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhk4f\" (UniqueName: \"kubernetes.io/projected/e58a995c-7842-4473-b890-7c93a46980f3-kube-api-access-mhk4f\") pod \"dns-default-bf7t2\" (UID: \"e58a995c-7842-4473-b890-7c93a46980f3\") " pod="openshift-dns/dns-default-bf7t2"
Jul 03 14:08:41.622390 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:41.622048    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cbb57ffb8-2ccdw: secret "image-registry-tls" not found
Jul 03 14:08:41.622390 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:41.622114    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-tls podName:cb3f129d-0917-4c5b-a369-ac9a0f9e63a2 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:42.122097574 +0000 UTC m=+35.572231007 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-tls") pod "image-registry-6cbb57ffb8-2ccdw" (UID: "cb3f129d-0917-4c5b-a369-ac9a0f9e63a2") : secret "image-registry-tls" not found
Jul 03 14:08:41.622390 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.622252    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-certificates\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:08:41.622390 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.622297    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e58a995c-7842-4473-b890-7c93a46980f3-config-volume\") pod \"dns-default-bf7t2\" (UID: \"e58a995c-7842-4473-b890-7c93a46980f3\") " pod="openshift-dns/dns-default-bf7t2"
Jul 03 14:08:41.622638 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.622463    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e58a995c-7842-4473-b890-7c93a46980f3-tmp-dir\") pod \"dns-default-bf7t2\" (UID: \"e58a995c-7842-4473-b890-7c93a46980f3\") " pod="openshift-dns/dns-default-bf7t2"
Jul 03 14:08:41.622638 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.622491    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-trusted-ca\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:08:41.625611 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.625588    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-installation-pull-secrets\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:08:41.625714 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.625643    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-image-registry-private-configuration\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:08:41.625714 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.625670    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/620719fe-f6f2-4310-9f6e-0f351f68264c-cert\") pod \"ingress-canary-srkff\" (UID: \"620719fe-f6f2-4310-9f6e-0f351f68264c\") " pod="openshift-ingress-canary/ingress-canary-srkff"
Jul 03 14:08:41.649434 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.649374    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-bound-sa-token\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:08:41.649575 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.649546    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhk4f\" (UniqueName: \"kubernetes.io/projected/e58a995c-7842-4473-b890-7c93a46980f3-kube-api-access-mhk4f\") pod \"dns-default-bf7t2\" (UID: \"e58a995c-7842-4473-b890-7c93a46980f3\") " pod="openshift-dns/dns-default-bf7t2"
Jul 03 14:08:41.650099 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.650081    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbg24\" (UniqueName: \"kubernetes.io/projected/620719fe-f6f2-4310-9f6e-0f351f68264c-kube-api-access-mbg24\") pod \"ingress-canary-srkff\" (UID: \"620719fe-f6f2-4310-9f6e-0f351f68264c\") " pod="openshift-ingress-canary/ingress-canary-srkff"
Jul 03 14:08:41.651190 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.651168    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvrgl\" (UniqueName: \"kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-kube-api-access-wvrgl\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:08:41.672200 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.672167    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tw77w_a5e9d1f2-6f21-48c0-8046-9f95a5218539/konnectivity-agent/0.log" limit=1024
Jul 03 14:08:41.758537 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.758495    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-srkff"
Jul 03 14:08:41.907065 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:41.906788    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-srkff"]
Jul 03 14:08:41.910879 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:08:41.910848    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod620719fe_f6f2_4310_9f6e_0f351f68264c.slice/crio-d56f8da583531898a6aa320323b13eeb799bcfb95184adc98db92afd75e9d486 WatchSource:0}: Error finding container d56f8da583531898a6aa320323b13eeb799bcfb95184adc98db92afd75e9d486: Status 404 returned error can't find the container with id d56f8da583531898a6aa320323b13eeb799bcfb95184adc98db92afd75e9d486
Jul 03 14:08:42.052411 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:42.052377    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:08:42.052594 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:42.052388    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:08:42.052594 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:42.052521    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:08:42.056824 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:42.056799    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 03 14:08:42.057010 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:42.056799    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 03 14:08:42.057080 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:42.057023    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 03 14:08:42.057286 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:42.057273    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-472cc\""
Jul 03 14:08:42.057688 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:42.057672    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 03 14:08:42.057798 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:42.057781    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tlbvf\""
Jul 03 14:08:42.126667 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:42.126636    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-tls\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:08:42.126842 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:42.126707    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e58a995c-7842-4473-b890-7c93a46980f3-metrics-tls\") pod \"dns-default-bf7t2\" (UID: \"e58a995c-7842-4473-b890-7c93a46980f3\") " pod="openshift-dns/dns-default-bf7t2"
Jul 03 14:08:42.126842 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:42.126802    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 03 14:08:42.126842 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:42.126825    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cbb57ffb8-2ccdw: secret "image-registry-tls" not found
Jul 03 14:08:42.127016 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:42.126849    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 03 14:08:42.127016 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:42.126886    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-tls podName:cb3f129d-0917-4c5b-a369-ac9a0f9e63a2 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:43.126869747 +0000 UTC m=+36.577003184 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-tls") pod "image-registry-6cbb57ffb8-2ccdw" (UID: "cb3f129d-0917-4c5b-a369-ac9a0f9e63a2") : secret "image-registry-tls" not found
Jul 03 14:08:42.127016 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:42.126908    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e58a995c-7842-4473-b890-7c93a46980f3-metrics-tls podName:e58a995c-7842-4473-b890-7c93a46980f3 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:43.126894882 +0000 UTC m=+36.577028300 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e58a995c-7842-4473-b890-7c93a46980f3-metrics-tls") pod "dns-default-bf7t2" (UID: "e58a995c-7842-4473-b890-7c93a46980f3") : secret "dns-default-metrics-tls" not found
Jul 03 14:08:42.219587 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:42.219557    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-srkff" event={"ID":"620719fe-f6f2-4310-9f6e-0f351f68264c","Type":"ContainerStarted","Data":"d56f8da583531898a6aa320323b13eeb799bcfb95184adc98db92afd75e9d486"}
Jul 03 14:08:43.134088 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:43.134055    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-tls\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:08:43.134579 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:43.134121    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e58a995c-7842-4473-b890-7c93a46980f3-metrics-tls\") pod \"dns-default-bf7t2\" (UID: \"e58a995c-7842-4473-b890-7c93a46980f3\") " pod="openshift-dns/dns-default-bf7t2"
Jul 03 14:08:43.134579 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:43.134250    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 03 14:08:43.134579 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:43.134312    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e58a995c-7842-4473-b890-7c93a46980f3-metrics-tls podName:e58a995c-7842-4473-b890-7c93a46980f3 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:45.1342984 +0000 UTC m=+38.584431818 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e58a995c-7842-4473-b890-7c93a46980f3-metrics-tls") pod "dns-default-bf7t2" (UID: "e58a995c-7842-4473-b890-7c93a46980f3") : secret "dns-default-metrics-tls" not found
Jul 03 14:08:43.134579 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:43.134255    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 03 14:08:43.134579 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:43.134356    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cbb57ffb8-2ccdw: secret "image-registry-tls" not found
Jul 03 14:08:43.134579 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:43.134403    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-tls podName:cb3f129d-0917-4c5b-a369-ac9a0f9e63a2 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:45.13439109 +0000 UTC m=+38.584524508 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-tls") pod "image-registry-6cbb57ffb8-2ccdw" (UID: "cb3f129d-0917-4c5b-a369-ac9a0f9e63a2") : secret "image-registry-tls" not found
Jul 03 14:08:45.150384 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:45.150337    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e58a995c-7842-4473-b890-7c93a46980f3-metrics-tls\") pod \"dns-default-bf7t2\" (UID: \"e58a995c-7842-4473-b890-7c93a46980f3\") " pod="openshift-dns/dns-default-bf7t2"
Jul 03 14:08:45.150864 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:45.150521    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-tls\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:08:45.150864 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:45.150535    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 03 14:08:45.150864 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:45.150616    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e58a995c-7842-4473-b890-7c93a46980f3-metrics-tls podName:e58a995c-7842-4473-b890-7c93a46980f3 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:49.150595546 +0000 UTC m=+42.600728963 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e58a995c-7842-4473-b890-7c93a46980f3-metrics-tls") pod "dns-default-bf7t2" (UID: "e58a995c-7842-4473-b890-7c93a46980f3") : secret "dns-default-metrics-tls" not found
Jul 03 14:08:45.150864 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:45.150672    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 03 14:08:45.150864 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:45.150689    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cbb57ffb8-2ccdw: secret "image-registry-tls" not found
Jul 03 14:08:45.150864 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:45.150738    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-tls podName:cb3f129d-0917-4c5b-a369-ac9a0f9e63a2 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:49.150725664 +0000 UTC m=+42.600859083 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-tls") pod "image-registry-6cbb57ffb8-2ccdw" (UID: "cb3f129d-0917-4c5b-a369-ac9a0f9e63a2") : secret "image-registry-tls" not found
Jul 03 14:08:46.232630 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:46.232597    2563 generic.go:356] "Generic (PLEG): container finished" podID="c8841628-0752-46c1-8dd8-850c21157563" containerID="4af75f53d0e28ace37261213ec70821d379c6b3d84571a608ee2034dd4bda3b3" exitCode=0
Jul 03 14:08:46.233079 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:46.232671    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pz4j6" event={"ID":"c8841628-0752-46c1-8dd8-850c21157563","Type":"ContainerDied","Data":"4af75f53d0e28ace37261213ec70821d379c6b3d84571a608ee2034dd4bda3b3"}
Jul 03 14:08:46.234187 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:46.234162    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-srkff" event={"ID":"620719fe-f6f2-4310-9f6e-0f351f68264c","Type":"ContainerStarted","Data":"d2279a65bc39d999f806a502a152e5159ca138f9c06bfd813d95499474f1181c"}
Jul 03 14:08:46.281630 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:46.281576    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-srkff" podStartSLOduration=1.340235131 podStartE2EDuration="5.28155882s" totalImagesPullingTime="3.941323689s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:08:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:08:46.280806517 +0000 UTC m=+39.730939958" watchObservedRunningTime="2026-07-03 14:08:46.28155882 +0000 UTC m=+39.731692259"
Jul 03 14:08:47.239206 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:47.239031    2563 generic.go:356] "Generic (PLEG): container finished" podID="c8841628-0752-46c1-8dd8-850c21157563" containerID="4441f8b40693b421736c4b70bf20e0dc4315c30ee820c14e016e44ccc1f53731" exitCode=0
Jul 03 14:08:47.239580 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:47.239109    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pz4j6" event={"ID":"c8841628-0752-46c1-8dd8-850c21157563","Type":"ContainerDied","Data":"4441f8b40693b421736c4b70bf20e0dc4315c30ee820c14e016e44ccc1f53731"}
Jul 03 14:08:48.244588 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:48.244543    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pz4j6" event={"ID":"c8841628-0752-46c1-8dd8-850c21157563","Type":"ContainerStarted","Data":"90d4bfede50b42be6d112268567b9d6f7a197ceb46ab2fe4351d4ee9b41dd1e8"}
Jul 03 14:08:48.277736 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:48.277685    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-pz4j6" podStartSLOduration=7.128056333 podStartE2EDuration="41.277669368s" totalImagesPullingTime="34.149613035s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:08:07 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:08:48.276774646 +0000 UTC m=+41.726908085" watchObservedRunningTime="2026-07-03 14:08:48.277669368 +0000 UTC m=+41.727802809"
Jul 03 14:08:49.185864 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:49.185823    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-tls\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:08:49.186091 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:49.185890    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e58a995c-7842-4473-b890-7c93a46980f3-metrics-tls\") pod \"dns-default-bf7t2\" (UID: \"e58a995c-7842-4473-b890-7c93a46980f3\") " pod="openshift-dns/dns-default-bf7t2"
Jul 03 14:08:49.186091 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:49.186013    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 03 14:08:49.186091 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:49.186038    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cbb57ffb8-2ccdw: secret "image-registry-tls" not found
Jul 03 14:08:49.186256 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:49.186100    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-tls podName:cb3f129d-0917-4c5b-a369-ac9a0f9e63a2 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:57.186079608 +0000 UTC m=+50.636213040 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-tls") pod "image-registry-6cbb57ffb8-2ccdw" (UID: "cb3f129d-0917-4c5b-a369-ac9a0f9e63a2") : secret "image-registry-tls" not found
Jul 03 14:08:49.186256 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:49.186021    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 03 14:08:49.186256 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:49.186144    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e58a995c-7842-4473-b890-7c93a46980f3-metrics-tls podName:e58a995c-7842-4473-b890-7c93a46980f3 nodeName:}" failed. No retries permitted until 2026-07-03 14:08:57.186132763 +0000 UTC m=+50.636266181 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e58a995c-7842-4473-b890-7c93a46980f3-metrics-tls") pod "dns-default-bf7t2" (UID: "e58a995c-7842-4473-b890-7c93a46980f3") : secret "dns-default-metrics-tls" not found
Jul 03 14:08:57.244318 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:57.244272    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e58a995c-7842-4473-b890-7c93a46980f3-metrics-tls\") pod \"dns-default-bf7t2\" (UID: \"e58a995c-7842-4473-b890-7c93a46980f3\") " pod="openshift-dns/dns-default-bf7t2"
Jul 03 14:08:57.244828 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:08:57.244375    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-tls\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:08:57.244828 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:57.244420    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 03 14:08:57.244828 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:57.244499    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e58a995c-7842-4473-b890-7c93a46980f3-metrics-tls podName:e58a995c-7842-4473-b890-7c93a46980f3 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:13.244481112 +0000 UTC m=+66.694614529 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e58a995c-7842-4473-b890-7c93a46980f3-metrics-tls") pod "dns-default-bf7t2" (UID: "e58a995c-7842-4473-b890-7c93a46980f3") : secret "dns-default-metrics-tls" not found
Jul 03 14:08:57.244828 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:57.244533    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 03 14:08:57.244828 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:57.244550    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cbb57ffb8-2ccdw: secret "image-registry-tls" not found
Jul 03 14:08:57.244828 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:08:57.244607    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-tls podName:cb3f129d-0917-4c5b-a369-ac9a0f9e63a2 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:13.244594779 +0000 UTC m=+66.694728196 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-tls") pod "image-registry-6cbb57ffb8-2ccdw" (UID: "cb3f129d-0917-4c5b-a369-ac9a0f9e63a2") : secret "image-registry-tls" not found
Jul 03 14:09:07.210423 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:07.210394    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xk86w"
Jul 03 14:09:12.747589 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:12.747549    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28b96845-6419-4df3-bff3-777d753989ce-metrics-certs\") pod \"network-metrics-daemon-9dzdm\" (UID: \"28b96845-6419-4df3-bff3-777d753989ce\") " pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:09:12.750775 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:12.750756    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 03 14:09:12.758520 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:12.758493    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 03 14:09:12.758618 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:12.758574    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28b96845-6419-4df3-bff3-777d753989ce-metrics-certs podName:28b96845-6419-4df3-bff3-777d753989ce nodeName:}" failed. No retries permitted until 2026-07-03 14:10:16.758559012 +0000 UTC m=+130.208692430 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28b96845-6419-4df3-bff3-777d753989ce-metrics-certs") pod "network-metrics-daemon-9dzdm" (UID: "28b96845-6419-4df3-bff3-777d753989ce") : secret "metrics-daemon-secret" not found
Jul 03 14:09:12.848628 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:12.848592    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8w2v\" (UniqueName: \"kubernetes.io/projected/c899d91f-c2fe-432e-b726-4e0f3bab3351-kube-api-access-n8w2v\") pod \"network-check-target-4sq59\" (UID: \"c899d91f-c2fe-432e-b726-4e0f3bab3351\") " pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:09:12.848738 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:12.848677    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01b1d786-bec1-48da-ace5-ec49bda1a3e3-original-pull-secret\") pod \"global-pull-secret-syncer-rcksp\" (UID: \"01b1d786-bec1-48da-ace5-ec49bda1a3e3\") " pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:09:12.851445 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:12.851425    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 03 14:09:12.851544 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:12.851496    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 03 14:09:12.861902 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:12.861875    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 03 14:09:12.863068 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:12.863048    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01b1d786-bec1-48da-ace5-ec49bda1a3e3-original-pull-secret\") pod \"global-pull-secret-syncer-rcksp\" (UID: \"01b1d786-bec1-48da-ace5-ec49bda1a3e3\") " pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:09:12.872078 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:12.872056    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8w2v\" (UniqueName: \"kubernetes.io/projected/c899d91f-c2fe-432e-b726-4e0f3bab3351-kube-api-access-n8w2v\") pod \"network-check-target-4sq59\" (UID: \"c899d91f-c2fe-432e-b726-4e0f3bab3351\") " pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:09:12.964859 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:12.964822    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcksp"
Jul 03 14:09:12.975236 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:12.975204    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tlbvf\""
Jul 03 14:09:12.983186 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:12.983165    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:09:13.096432 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:13.096403    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rcksp"]
Jul 03 14:09:13.100156 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:09:13.100120    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01b1d786_bec1_48da_ace5_ec49bda1a3e3.slice/crio-4560dcf4ef8d9c36fa29deb2d9c12534d8fc88295977f4c7ffe7478dd640648b WatchSource:0}: Error finding container 4560dcf4ef8d9c36fa29deb2d9c12534d8fc88295977f4c7ffe7478dd640648b: Status 404 returned error can't find the container with id 4560dcf4ef8d9c36fa29deb2d9c12534d8fc88295977f4c7ffe7478dd640648b
Jul 03 14:09:13.111921 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:13.111897    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4sq59"]
Jul 03 14:09:13.114832 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:09:13.114809    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc899d91f_c2fe_432e_b726_4e0f3bab3351.slice/crio-ccac03c76ce56b051c50dd26388222af0ced78291f80d549bd5a93e6c8674df6 WatchSource:0}: Error finding container ccac03c76ce56b051c50dd26388222af0ced78291f80d549bd5a93e6c8674df6: Status 404 returned error can't find the container with id ccac03c76ce56b051c50dd26388222af0ced78291f80d549bd5a93e6c8674df6
Jul 03 14:09:13.252760 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:13.252725    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e58a995c-7842-4473-b890-7c93a46980f3-metrics-tls\") pod \"dns-default-bf7t2\" (UID: \"e58a995c-7842-4473-b890-7c93a46980f3\") " pod="openshift-dns/dns-default-bf7t2"
Jul 03 14:09:13.252934 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:13.252785    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-tls\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:09:13.252934 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:13.252878    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 03 14:09:13.252934 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:13.252893    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cbb57ffb8-2ccdw: secret "image-registry-tls" not found
Jul 03 14:09:13.253073 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:13.252944    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-tls podName:cb3f129d-0917-4c5b-a369-ac9a0f9e63a2 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:45.252932131 +0000 UTC m=+98.703065549 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-tls") pod "image-registry-6cbb57ffb8-2ccdw" (UID: "cb3f129d-0917-4c5b-a369-ac9a0f9e63a2") : secret "image-registry-tls" not found
Jul 03 14:09:13.253073 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:13.252876    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 03 14:09:13.253073 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:13.253026    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e58a995c-7842-4473-b890-7c93a46980f3-metrics-tls podName:e58a995c-7842-4473-b890-7c93a46980f3 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:45.253010689 +0000 UTC m=+98.703144111 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e58a995c-7842-4473-b890-7c93a46980f3-metrics-tls") pod "dns-default-bf7t2" (UID: "e58a995c-7842-4473-b890-7c93a46980f3") : secret "dns-default-metrics-tls" not found
Jul 03 14:09:13.304381 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:13.304297    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4sq59" event={"ID":"c899d91f-c2fe-432e-b726-4e0f3bab3351","Type":"ContainerStarted","Data":"ccac03c76ce56b051c50dd26388222af0ced78291f80d549bd5a93e6c8674df6"}
Jul 03 14:09:13.305394 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:13.305366    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rcksp" event={"ID":"01b1d786-bec1-48da-ace5-ec49bda1a3e3","Type":"ContainerStarted","Data":"4560dcf4ef8d9c36fa29deb2d9c12534d8fc88295977f4c7ffe7478dd640648b"}
Jul 03 14:09:17.318527 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:17.318483    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4sq59" event={"ID":"c899d91f-c2fe-432e-b726-4e0f3bab3351","Type":"ContainerStarted","Data":"88fb246f73ed92de9e62a4fb17b52d4ed788ab1dce3a9b1968fd1bf7f49c463c"}
Jul 03 14:09:17.319028 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:17.318653    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:09:18.051007 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.050922    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4sq59" podStartSLOduration=67.643070598 podStartE2EDuration="1m11.050901827s" totalImagesPullingTime="3.407831229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:08:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:09:17.343000661 +0000 UTC m=+70.793134127" watchObservedRunningTime="2026-07-03 14:09:18.050901827 +0000 UTC m=+71.501035267"
Jul 03 14:09:18.051451 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.051431    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-88rx9"]
Jul 03 14:09:18.054607 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.054587    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-88rx9"
Jul 03 14:09:18.057572 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.057538    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 03 14:09:18.057572 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.057558    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-vl5nh\""
Jul 03 14:09:18.058529 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.058508    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 03 14:09:18.059513 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.059497    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 03 14:09:18.059631 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.059531    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 03 14:09:18.072115 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.072088    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-88rx9"]
Jul 03 14:09:18.094280 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.094249    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d85062a4-19c6-4cc0-9488-19b48622c627-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-88rx9\" (UID: \"d85062a4-19c6-4cc0-9488-19b48622c627\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-88rx9"
Jul 03 14:09:18.094417 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.094320    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d85062a4-19c6-4cc0-9488-19b48622c627-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-88rx9\" (UID: \"d85062a4-19c6-4cc0-9488-19b48622c627\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-88rx9"
Jul 03 14:09:18.094481 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.094447    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x87rf\" (UniqueName: \"kubernetes.io/projected/d85062a4-19c6-4cc0-9488-19b48622c627-kube-api-access-x87rf\") pod \"cluster-monitoring-operator-64786d9cbb-88rx9\" (UID: \"d85062a4-19c6-4cc0-9488-19b48622c627\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-88rx9"
Jul 03 14:09:18.195159 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.195125    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x87rf\" (UniqueName: \"kubernetes.io/projected/d85062a4-19c6-4cc0-9488-19b48622c627-kube-api-access-x87rf\") pod \"cluster-monitoring-operator-64786d9cbb-88rx9\" (UID: \"d85062a4-19c6-4cc0-9488-19b48622c627\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-88rx9"
Jul 03 14:09:18.195159 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.195166    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d85062a4-19c6-4cc0-9488-19b48622c627-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-88rx9\" (UID: \"d85062a4-19c6-4cc0-9488-19b48622c627\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-88rx9"
Jul 03 14:09:18.195401 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.195196    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d85062a4-19c6-4cc0-9488-19b48622c627-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-88rx9\" (UID: \"d85062a4-19c6-4cc0-9488-19b48622c627\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-88rx9"
Jul 03 14:09:18.195401 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:18.195290    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 03 14:09:18.195401 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:18.195361    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d85062a4-19c6-4cc0-9488-19b48622c627-cluster-monitoring-operator-tls podName:d85062a4-19c6-4cc0-9488-19b48622c627 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:18.695345382 +0000 UTC m=+72.145478821 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d85062a4-19c6-4cc0-9488-19b48622c627-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-88rx9" (UID: "d85062a4-19c6-4cc0-9488-19b48622c627") : secret "cluster-monitoring-operator-tls" not found
Jul 03 14:09:18.195792 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.195771    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d85062a4-19c6-4cc0-9488-19b48622c627-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-88rx9\" (UID: \"d85062a4-19c6-4cc0-9488-19b48622c627\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-88rx9"
Jul 03 14:09:18.208539 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.208512    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x87rf\" (UniqueName: \"kubernetes.io/projected/d85062a4-19c6-4cc0-9488-19b48622c627-kube-api-access-x87rf\") pod \"cluster-monitoring-operator-64786d9cbb-88rx9\" (UID: \"d85062a4-19c6-4cc0-9488-19b48622c627\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-88rx9"
Jul 03 14:09:18.699806 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.699753    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d85062a4-19c6-4cc0-9488-19b48622c627-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-88rx9\" (UID: \"d85062a4-19c6-4cc0-9488-19b48622c627\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-88rx9"
Jul 03 14:09:18.700215 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:18.699910    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 03 14:09:18.700215 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:18.699995    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d85062a4-19c6-4cc0-9488-19b48622c627-cluster-monitoring-operator-tls podName:d85062a4-19c6-4cc0-9488-19b48622c627 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:19.699963617 +0000 UTC m=+73.150097040 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d85062a4-19c6-4cc0-9488-19b48622c627-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-88rx9" (UID: "d85062a4-19c6-4cc0-9488-19b48622c627") : secret "cluster-monitoring-operator-tls" not found
Jul 03 14:09:18.729923 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.729895    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-l9qcj"]
Jul 03 14:09:18.736158 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.736132    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-l9qcj"
Jul 03 14:09:18.738702 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.738670    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 03 14:09:18.738868 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.738847    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-wdz9w\""
Jul 03 14:09:18.738948 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.738932    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 03 14:09:18.739391 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.739377    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 03 14:09:18.739640 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.739624    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 03 14:09:18.746259 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.746240    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-l9qcj"]
Jul 03 14:09:18.800456 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.800425    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8a71b7ee-121f-474e-a3b6-dbbbaa61daa9-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-l9qcj\" (UID: \"8a71b7ee-121f-474e-a3b6-dbbbaa61daa9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-l9qcj"
Jul 03 14:09:18.800624 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.800478    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8a71b7ee-121f-474e-a3b6-dbbbaa61daa9-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-l9qcj\" (UID: \"8a71b7ee-121f-474e-a3b6-dbbbaa61daa9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-l9qcj"
Jul 03 14:09:18.800624 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.800528    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rffvl\" (UniqueName: \"kubernetes.io/projected/8a71b7ee-121f-474e-a3b6-dbbbaa61daa9-kube-api-access-rffvl\") pod \"kube-storage-version-migrator-operator-846bf668f7-l9qcj\" (UID: \"8a71b7ee-121f-474e-a3b6-dbbbaa61daa9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-l9qcj"
Jul 03 14:09:18.901311 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.901280    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8a71b7ee-121f-474e-a3b6-dbbbaa61daa9-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-l9qcj\" (UID: \"8a71b7ee-121f-474e-a3b6-dbbbaa61daa9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-l9qcj"
Jul 03 14:09:18.901378 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.901339    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rffvl\" (UniqueName: \"kubernetes.io/projected/8a71b7ee-121f-474e-a3b6-dbbbaa61daa9-kube-api-access-rffvl\") pod \"kube-storage-version-migrator-operator-846bf668f7-l9qcj\" (UID: \"8a71b7ee-121f-474e-a3b6-dbbbaa61daa9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-l9qcj"
Jul 03 14:09:18.901416 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.901395    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8a71b7ee-121f-474e-a3b6-dbbbaa61daa9-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-l9qcj\" (UID: \"8a71b7ee-121f-474e-a3b6-dbbbaa61daa9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-l9qcj"
Jul 03 14:09:18.901933 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.901904    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8a71b7ee-121f-474e-a3b6-dbbbaa61daa9-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-l9qcj\" (UID: \"8a71b7ee-121f-474e-a3b6-dbbbaa61daa9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-l9qcj"
Jul 03 14:09:18.903640 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.903618    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8a71b7ee-121f-474e-a3b6-dbbbaa61daa9-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-l9qcj\" (UID: \"8a71b7ee-121f-474e-a3b6-dbbbaa61daa9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-l9qcj"
Jul 03 14:09:18.913189 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:18.913164    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rffvl\" (UniqueName: \"kubernetes.io/projected/8a71b7ee-121f-474e-a3b6-dbbbaa61daa9-kube-api-access-rffvl\") pod \"kube-storage-version-migrator-operator-846bf668f7-l9qcj\" (UID: \"8a71b7ee-121f-474e-a3b6-dbbbaa61daa9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-l9qcj"
Jul 03 14:09:19.044683 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:19.044578    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-l9qcj"
Jul 03 14:09:19.164559 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:19.164528    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-l9qcj"]
Jul 03 14:09:19.167523 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:09:19.167495    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a71b7ee_121f_474e_a3b6_dbbbaa61daa9.slice/crio-5837e1a9358d0fba52081910b6825e4b51243b58b108e1bfe7829f7d78c0cee6 WatchSource:0}: Error finding container 5837e1a9358d0fba52081910b6825e4b51243b58b108e1bfe7829f7d78c0cee6: Status 404 returned error can't find the container with id 5837e1a9358d0fba52081910b6825e4b51243b58b108e1bfe7829f7d78c0cee6
Jul 03 14:09:19.324108 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:19.324014    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-l9qcj" event={"ID":"8a71b7ee-121f-474e-a3b6-dbbbaa61daa9","Type":"ContainerStarted","Data":"5837e1a9358d0fba52081910b6825e4b51243b58b108e1bfe7829f7d78c0cee6"}
Jul 03 14:09:19.325410 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:19.325384    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rcksp" event={"ID":"01b1d786-bec1-48da-ace5-ec49bda1a3e3","Type":"ContainerStarted","Data":"65219ced5b5b281024ed1e2883d3890707eafdaf3005e04ae9cf75f76bca6058"}
Jul 03 14:09:19.707752 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:19.707710    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d85062a4-19c6-4cc0-9488-19b48622c627-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-88rx9\" (UID: \"d85062a4-19c6-4cc0-9488-19b48622c627\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-88rx9"
Jul 03 14:09:19.708226 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:19.707831    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 03 14:09:19.708226 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:19.707917    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d85062a4-19c6-4cc0-9488-19b48622c627-cluster-monitoring-operator-tls podName:d85062a4-19c6-4cc0-9488-19b48622c627 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:21.70789645 +0000 UTC m=+75.158029872 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d85062a4-19c6-4cc0-9488-19b48622c627-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-88rx9" (UID: "d85062a4-19c6-4cc0-9488-19b48622c627") : secret "cluster-monitoring-operator-tls" not found
Jul 03 14:09:21.721808 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:21.721767    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d85062a4-19c6-4cc0-9488-19b48622c627-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-88rx9\" (UID: \"d85062a4-19c6-4cc0-9488-19b48622c627\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-88rx9"
Jul 03 14:09:21.722232 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:21.721919    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 03 14:09:21.722232 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:21.722003    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d85062a4-19c6-4cc0-9488-19b48622c627-cluster-monitoring-operator-tls podName:d85062a4-19c6-4cc0-9488-19b48622c627 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:25.721988419 +0000 UTC m=+79.172121837 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d85062a4-19c6-4cc0-9488-19b48622c627-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-88rx9" (UID: "d85062a4-19c6-4cc0-9488-19b48622c627") : secret "cluster-monitoring-operator-tls" not found
Jul 03 14:09:21.846434 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:21.846380    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-rcksp" podStartSLOduration=69.707230167 podStartE2EDuration="1m14.84635875s" totalImagesPullingTime="5.139128583s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:08:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:09:19.352182042 +0000 UTC m=+72.802315483" watchObservedRunningTime="2026-07-03 14:09:21.84635875 +0000 UTC m=+75.296492192"
Jul 03 14:09:21.846804 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:21.846788    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-v6t5n"]
Jul 03 14:09:21.849913 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:21.849895    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v6t5n"
Jul 03 14:09:21.854662 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:21.854643    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 03 14:09:21.854916 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:21.854894    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 03 14:09:21.855255 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:21.855237    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 03 14:09:21.855781 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:21.855767    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kwmd4\""
Jul 03 14:09:21.860449 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:21.860432    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 03 14:09:21.882353 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:21.882327    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v6t5n"]
Jul 03 14:09:21.920418 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:21.920384    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9kjx7"]
Jul 03 14:09:21.923404 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:21.923379    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9kjx7"
Jul 03 14:09:21.923674 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:21.923547    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f499c073-70d1-4178-8c52-1e64babb8a37-crio-socket\") pod \"insights-runtime-extractor-v6t5n\" (UID: \"f499c073-70d1-4178-8c52-1e64babb8a37\") " pod="openshift-insights/insights-runtime-extractor-v6t5n"
Jul 03 14:09:21.923674 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:21.923597    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f499c073-70d1-4178-8c52-1e64babb8a37-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v6t5n\" (UID: \"f499c073-70d1-4178-8c52-1e64babb8a37\") " pod="openshift-insights/insights-runtime-extractor-v6t5n"
Jul 03 14:09:21.923674 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:21.923637    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f499c073-70d1-4178-8c52-1e64babb8a37-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v6t5n\" (UID: \"f499c073-70d1-4178-8c52-1e64babb8a37\") " pod="openshift-insights/insights-runtime-extractor-v6t5n"
Jul 03 14:09:21.923858 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:21.923715    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd8br\" (UniqueName: \"kubernetes.io/projected/f499c073-70d1-4178-8c52-1e64babb8a37-kube-api-access-cd8br\") pod \"insights-runtime-extractor-v6t5n\" (UID: \"f499c073-70d1-4178-8c52-1e64babb8a37\") " pod="openshift-insights/insights-runtime-extractor-v6t5n"
Jul 03 14:09:21.923858 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:21.923801    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f499c073-70d1-4178-8c52-1e64babb8a37-data-volume\") pod \"insights-runtime-extractor-v6t5n\" (UID: \"f499c073-70d1-4178-8c52-1e64babb8a37\") " pod="openshift-insights/insights-runtime-extractor-v6t5n"
Jul 03 14:09:21.931298 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:21.930987    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 03 14:09:21.931298 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:21.931110    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-bnj5v\""
Jul 03 14:09:21.931490 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:21.931305    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 03 14:09:21.931490 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:21.931469    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 03 14:09:21.934595 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:21.931721    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 03 14:09:21.941665 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:21.941643    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9kjx7"]
Jul 03 14:09:22.025056 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:22.024926    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a258d061-bd67-45af-b0a1-4440e92a1258-config\") pod \"service-ca-operator-5d46d9ffd4-9kjx7\" (UID: \"a258d061-bd67-45af-b0a1-4440e92a1258\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9kjx7"
Jul 03 14:09:22.025056 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:22.024993    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f499c073-70d1-4178-8c52-1e64babb8a37-crio-socket\") pod \"insights-runtime-extractor-v6t5n\" (UID: \"f499c073-70d1-4178-8c52-1e64babb8a37\") " pod="openshift-insights/insights-runtime-extractor-v6t5n"
Jul 03 14:09:22.025056 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:22.025031    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f499c073-70d1-4178-8c52-1e64babb8a37-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v6t5n\" (UID: \"f499c073-70d1-4178-8c52-1e64babb8a37\") " pod="openshift-insights/insights-runtime-extractor-v6t5n"
Jul 03 14:09:22.025056 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:22.025052    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd8br\" (UniqueName: \"kubernetes.io/projected/f499c073-70d1-4178-8c52-1e64babb8a37-kube-api-access-cd8br\") pod \"insights-runtime-extractor-v6t5n\" (UID: \"f499c073-70d1-4178-8c52-1e64babb8a37\") " pod="openshift-insights/insights-runtime-extractor-v6t5n"
Jul 03 14:09:22.025380 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:22.025076    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f499c073-70d1-4178-8c52-1e64babb8a37-crio-socket\") pod \"insights-runtime-extractor-v6t5n\" (UID: \"f499c073-70d1-4178-8c52-1e64babb8a37\") " pod="openshift-insights/insights-runtime-extractor-v6t5n"
Jul 03 14:09:22.025380 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:22.025130    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f499c073-70d1-4178-8c52-1e64babb8a37-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v6t5n\" (UID: \"f499c073-70d1-4178-8c52-1e64babb8a37\") " pod="openshift-insights/insights-runtime-extractor-v6t5n"
Jul 03 14:09:22.025380 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:22.025154    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a258d061-bd67-45af-b0a1-4440e92a1258-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-9kjx7\" (UID: \"a258d061-bd67-45af-b0a1-4440e92a1258\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9kjx7"
Jul 03 14:09:22.025380 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:22.025172    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 03 14:09:22.025380 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:22.025253    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f499c073-70d1-4178-8c52-1e64babb8a37-kube-rbac-proxy-cm podName:f499c073-70d1-4178-8c52-1e64babb8a37 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:22.525234084 +0000 UTC m=+75.975367518 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f499c073-70d1-4178-8c52-1e64babb8a37-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v6t5n" (UID: "f499c073-70d1-4178-8c52-1e64babb8a37") : configmap "kube-rbac-proxy" not found
Jul 03 14:09:22.025380 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:22.025253    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 03 14:09:22.025380 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:22.025281    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjm8m\" (UniqueName: \"kubernetes.io/projected/a258d061-bd67-45af-b0a1-4440e92a1258-kube-api-access-kjm8m\") pod \"service-ca-operator-5d46d9ffd4-9kjx7\" (UID: \"a258d061-bd67-45af-b0a1-4440e92a1258\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9kjx7"
Jul 03 14:09:22.025380 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:22.025288    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f499c073-70d1-4178-8c52-1e64babb8a37-insights-runtime-extractor-tls podName:f499c073-70d1-4178-8c52-1e64babb8a37 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:22.525278898 +0000 UTC m=+75.975412317 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f499c073-70d1-4178-8c52-1e64babb8a37-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v6t5n" (UID: "f499c073-70d1-4178-8c52-1e64babb8a37") : secret "insights-runtime-extractor-tls" not found
Jul 03 14:09:22.025380 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:22.025328    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f499c073-70d1-4178-8c52-1e64babb8a37-data-volume\") pod \"insights-runtime-extractor-v6t5n\" (UID: \"f499c073-70d1-4178-8c52-1e64babb8a37\") " pod="openshift-insights/insights-runtime-extractor-v6t5n"
Jul 03 14:09:22.025842 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:22.025602    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f499c073-70d1-4178-8c52-1e64babb8a37-data-volume\") pod \"insights-runtime-extractor-v6t5n\" (UID: \"f499c073-70d1-4178-8c52-1e64babb8a37\") " pod="openshift-insights/insights-runtime-extractor-v6t5n"
Jul 03 14:09:22.037903 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:22.037878    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd8br\" (UniqueName: \"kubernetes.io/projected/f499c073-70d1-4178-8c52-1e64babb8a37-kube-api-access-cd8br\") pod \"insights-runtime-extractor-v6t5n\" (UID: \"f499c073-70d1-4178-8c52-1e64babb8a37\") " pod="openshift-insights/insights-runtime-extractor-v6t5n"
Jul 03 14:09:22.126236 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:22.126197    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a258d061-bd67-45af-b0a1-4440e92a1258-config\") pod \"service-ca-operator-5d46d9ffd4-9kjx7\" (UID: \"a258d061-bd67-45af-b0a1-4440e92a1258\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9kjx7"
Jul 03 14:09:22.126425 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:22.126294    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a258d061-bd67-45af-b0a1-4440e92a1258-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-9kjx7\" (UID: \"a258d061-bd67-45af-b0a1-4440e92a1258\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9kjx7"
Jul 03 14:09:22.126425 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:22.126318    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjm8m\" (UniqueName: \"kubernetes.io/projected/a258d061-bd67-45af-b0a1-4440e92a1258-kube-api-access-kjm8m\") pod \"service-ca-operator-5d46d9ffd4-9kjx7\" (UID: \"a258d061-bd67-45af-b0a1-4440e92a1258\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9kjx7"
Jul 03 14:09:22.126909 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:22.126873    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a258d061-bd67-45af-b0a1-4440e92a1258-config\") pod \"service-ca-operator-5d46d9ffd4-9kjx7\" (UID: \"a258d061-bd67-45af-b0a1-4440e92a1258\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9kjx7"
Jul 03 14:09:22.128532 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:22.128515    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a258d061-bd67-45af-b0a1-4440e92a1258-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-9kjx7\" (UID: \"a258d061-bd67-45af-b0a1-4440e92a1258\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9kjx7"
Jul 03 14:09:22.136092 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:22.136069    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjm8m\" (UniqueName: \"kubernetes.io/projected/a258d061-bd67-45af-b0a1-4440e92a1258-kube-api-access-kjm8m\") pod \"service-ca-operator-5d46d9ffd4-9kjx7\" (UID: \"a258d061-bd67-45af-b0a1-4440e92a1258\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9kjx7"
Jul 03 14:09:22.235412 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:22.235360    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9kjx7"
Jul 03 14:09:22.334663 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:22.334623    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-l9qcj" event={"ID":"8a71b7ee-121f-474e-a3b6-dbbbaa61daa9","Type":"ContainerStarted","Data":"748df83199c6502863d008fb5c6331cfa50c98c3f91d38d89fb34f35d47fcdd1"}
Jul 03 14:09:22.386493 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:22.386436    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-l9qcj" podStartSLOduration=2.265383493 podStartE2EDuration="4.386412413s" totalImagesPullingTime="2.12102892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:09:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:09:22.359575651 +0000 UTC m=+75.809709091" watchObservedRunningTime="2026-07-03 14:09:22.386412413 +0000 UTC m=+75.836545856"
Jul 03 14:09:22.387023 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:22.387006    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9kjx7"]
Jul 03 14:09:22.390367 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:09:22.390343    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda258d061_bd67_45af_b0a1_4440e92a1258.slice/crio-098b04937e891f193041a4b33cbbef4783ee7835d65405a626cb3ae5f3bb7c6e WatchSource:0}: Error finding container 098b04937e891f193041a4b33cbbef4783ee7835d65405a626cb3ae5f3bb7c6e: Status 404 returned error can't find the container with id 098b04937e891f193041a4b33cbbef4783ee7835d65405a626cb3ae5f3bb7c6e
Jul 03 14:09:22.529950 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:22.529912    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f499c073-70d1-4178-8c52-1e64babb8a37-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v6t5n\" (UID: \"f499c073-70d1-4178-8c52-1e64babb8a37\") " pod="openshift-insights/insights-runtime-extractor-v6t5n"
Jul 03 14:09:22.530138 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:22.530059    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 03 14:09:22.530138 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:22.530097    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f499c073-70d1-4178-8c52-1e64babb8a37-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v6t5n\" (UID: \"f499c073-70d1-4178-8c52-1e64babb8a37\") " pod="openshift-insights/insights-runtime-extractor-v6t5n"
Jul 03 14:09:22.530138 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:22.530136    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f499c073-70d1-4178-8c52-1e64babb8a37-kube-rbac-proxy-cm podName:f499c073-70d1-4178-8c52-1e64babb8a37 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:23.530117546 +0000 UTC m=+76.980250971 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f499c073-70d1-4178-8c52-1e64babb8a37-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v6t5n" (UID: "f499c073-70d1-4178-8c52-1e64babb8a37") : configmap "kube-rbac-proxy" not found
Jul 03 14:09:22.530269 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:22.530184    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 03 14:09:22.530269 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:22.530239    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f499c073-70d1-4178-8c52-1e64babb8a37-insights-runtime-extractor-tls podName:f499c073-70d1-4178-8c52-1e64babb8a37 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:23.530228264 +0000 UTC m=+76.980361681 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f499c073-70d1-4178-8c52-1e64babb8a37-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v6t5n" (UID: "f499c073-70d1-4178-8c52-1e64babb8a37") : secret "insights-runtime-extractor-tls" not found
Jul 03 14:09:23.338507 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:23.338459    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9kjx7" event={"ID":"a258d061-bd67-45af-b0a1-4440e92a1258","Type":"ContainerStarted","Data":"098b04937e891f193041a4b33cbbef4783ee7835d65405a626cb3ae5f3bb7c6e"}
Jul 03 14:09:23.538275 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:23.538235    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f499c073-70d1-4178-8c52-1e64babb8a37-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v6t5n\" (UID: \"f499c073-70d1-4178-8c52-1e64babb8a37\") " pod="openshift-insights/insights-runtime-extractor-v6t5n"
Jul 03 14:09:23.538441 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:23.538312    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f499c073-70d1-4178-8c52-1e64babb8a37-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v6t5n\" (UID: \"f499c073-70d1-4178-8c52-1e64babb8a37\") " pod="openshift-insights/insights-runtime-extractor-v6t5n"
Jul 03 14:09:23.538441 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:23.538359    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 03 14:09:23.538441 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:23.538393    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 03 14:09:23.538441 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:23.538435    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f499c073-70d1-4178-8c52-1e64babb8a37-kube-rbac-proxy-cm podName:f499c073-70d1-4178-8c52-1e64babb8a37 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:25.538418746 +0000 UTC m=+78.988552167 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f499c073-70d1-4178-8c52-1e64babb8a37-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v6t5n" (UID: "f499c073-70d1-4178-8c52-1e64babb8a37") : configmap "kube-rbac-proxy" not found
Jul 03 14:09:23.538579 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:23.538452    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f499c073-70d1-4178-8c52-1e64babb8a37-insights-runtime-extractor-tls podName:f499c073-70d1-4178-8c52-1e64babb8a37 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:25.538444424 +0000 UTC m=+78.988577842 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f499c073-70d1-4178-8c52-1e64babb8a37-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v6t5n" (UID: "f499c073-70d1-4178-8c52-1e64babb8a37") : secret "insights-runtime-extractor-tls" not found
Jul 03 14:09:23.802093 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:23.802063    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9k9sd_11f255a7-6652-4c64-97b8-c980552c363f/dns-node-resolver/0.log"
Jul 03 14:09:24.602674 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:24.602651    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7b795_2df513c5-3c49-4255-9106-9e36d40d4fa1/node-ca/0.log"
Jul 03 14:09:25.345158 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:25.345122    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9kjx7" event={"ID":"a258d061-bd67-45af-b0a1-4440e92a1258","Type":"ContainerStarted","Data":"5e6c6d893591ebbb6d883f3f2837a98cc5b5d7f7dde8f35d1c079bad9b3d94a5"}
Jul 03 14:09:25.362628 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:25.362574    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9kjx7" podStartSLOduration=2.106191846 podStartE2EDuration="4.362558181s" totalImagesPullingTime="2.256366335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:09:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:09:25.362033735 +0000 UTC m=+78.812167194" watchObservedRunningTime="2026-07-03 14:09:25.362558181 +0000 UTC m=+78.812691620"
Jul 03 14:09:25.498091 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:25.498055    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-fc745"]
Jul 03 14:09:25.501374 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:25.501354    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-fc745"
Jul 03 14:09:25.503724 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:25.503692    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 03 14:09:25.503847 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:25.503826    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-cncrv\""
Jul 03 14:09:25.503903 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:25.503861    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 03 14:09:25.511436 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:25.511413    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-fc745"]
Jul 03 14:09:25.556381 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:25.556343    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f499c073-70d1-4178-8c52-1e64babb8a37-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v6t5n\" (UID: \"f499c073-70d1-4178-8c52-1e64babb8a37\") " pod="openshift-insights/insights-runtime-extractor-v6t5n"
Jul 03 14:09:25.556560 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:25.556407    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f499c073-70d1-4178-8c52-1e64babb8a37-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v6t5n\" (UID: \"f499c073-70d1-4178-8c52-1e64babb8a37\") " pod="openshift-insights/insights-runtime-extractor-v6t5n"
Jul 03 14:09:25.556560 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:25.556444    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e3af58c5-0172-407c-814e-1169456d4bda-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-fc745\" (UID: \"e3af58c5-0172-407c-814e-1169456d4bda\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fc745"
Jul 03 14:09:25.556560 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:25.556468    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e3af58c5-0172-407c-814e-1169456d4bda-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-fc745\" (UID: \"e3af58c5-0172-407c-814e-1169456d4bda\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fc745"
Jul 03 14:09:25.556560 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:25.556513    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 03 14:09:25.556560 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:25.556535    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 03 14:09:25.556761 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:25.556604    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f499c073-70d1-4178-8c52-1e64babb8a37-kube-rbac-proxy-cm podName:f499c073-70d1-4178-8c52-1e64babb8a37 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:29.55658557 +0000 UTC m=+83.006718989 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f499c073-70d1-4178-8c52-1e64babb8a37-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v6t5n" (UID: "f499c073-70d1-4178-8c52-1e64babb8a37") : configmap "kube-rbac-proxy" not found
Jul 03 14:09:25.556761 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:25.556656    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f499c073-70d1-4178-8c52-1e64babb8a37-insights-runtime-extractor-tls podName:f499c073-70d1-4178-8c52-1e64babb8a37 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:29.556635457 +0000 UTC m=+83.006768890 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f499c073-70d1-4178-8c52-1e64babb8a37-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v6t5n" (UID: "f499c073-70d1-4178-8c52-1e64babb8a37") : secret "insights-runtime-extractor-tls" not found
Jul 03 14:09:25.602514 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:25.600859    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-srkff_620719fe-f6f2-4310-9f6e-0f351f68264c/serve-healthcheck-canary/0.log"
Jul 03 14:09:25.657951 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:25.657906    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e3af58c5-0172-407c-814e-1169456d4bda-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-fc745\" (UID: \"e3af58c5-0172-407c-814e-1169456d4bda\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fc745"
Jul 03 14:09:25.658439 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:25.657989    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e3af58c5-0172-407c-814e-1169456d4bda-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-fc745\" (UID: \"e3af58c5-0172-407c-814e-1169456d4bda\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fc745"
Jul 03 14:09:25.658439 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:25.658091    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 03 14:09:25.658439 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:25.658175    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3af58c5-0172-407c-814e-1169456d4bda-networking-console-plugin-cert podName:e3af58c5-0172-407c-814e-1169456d4bda nodeName:}" failed. No retries permitted until 2026-07-03 14:09:26.158153105 +0000 UTC m=+79.608286526 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e3af58c5-0172-407c-814e-1169456d4bda-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-fc745" (UID: "e3af58c5-0172-407c-814e-1169456d4bda") : secret "networking-console-plugin-cert" not found
Jul 03 14:09:25.658756 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:25.658737    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e3af58c5-0172-407c-814e-1169456d4bda-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-fc745\" (UID: \"e3af58c5-0172-407c-814e-1169456d4bda\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fc745"
Jul 03 14:09:25.758600 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:25.758563    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d85062a4-19c6-4cc0-9488-19b48622c627-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-88rx9\" (UID: \"d85062a4-19c6-4cc0-9488-19b48622c627\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-88rx9"
Jul 03 14:09:25.758804 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:25.758734    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 03 14:09:25.758864 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:25.758813    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d85062a4-19c6-4cc0-9488-19b48622c627-cluster-monitoring-operator-tls podName:d85062a4-19c6-4cc0-9488-19b48622c627 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:33.758792569 +0000 UTC m=+87.208925990 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d85062a4-19c6-4cc0-9488-19b48622c627-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-88rx9" (UID: "d85062a4-19c6-4cc0-9488-19b48622c627") : secret "cluster-monitoring-operator-tls" not found
Jul 03 14:09:26.162781 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:26.162745    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e3af58c5-0172-407c-814e-1169456d4bda-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-fc745\" (UID: \"e3af58c5-0172-407c-814e-1169456d4bda\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fc745"
Jul 03 14:09:26.163004 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:26.162951    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 03 14:09:26.163081 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:26.163062    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3af58c5-0172-407c-814e-1169456d4bda-networking-console-plugin-cert podName:e3af58c5-0172-407c-814e-1169456d4bda nodeName:}" failed. No retries permitted until 2026-07-03 14:09:27.163038518 +0000 UTC m=+80.613171944 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e3af58c5-0172-407c-814e-1169456d4bda-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-fc745" (UID: "e3af58c5-0172-407c-814e-1169456d4bda") : secret "networking-console-plugin-cert" not found
Jul 03 14:09:27.170635 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:27.170602    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e3af58c5-0172-407c-814e-1169456d4bda-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-fc745\" (UID: \"e3af58c5-0172-407c-814e-1169456d4bda\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fc745"
Jul 03 14:09:27.171051 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:27.170741    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 03 14:09:27.171051 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:27.170799    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3af58c5-0172-407c-814e-1169456d4bda-networking-console-plugin-cert podName:e3af58c5-0172-407c-814e-1169456d4bda nodeName:}" failed. No retries permitted until 2026-07-03 14:09:29.170784189 +0000 UTC m=+82.620917607 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e3af58c5-0172-407c-814e-1169456d4bda-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-fc745" (UID: "e3af58c5-0172-407c-814e-1169456d4bda") : secret "networking-console-plugin-cert" not found
Jul 03 14:09:29.187933 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:29.187818    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e3af58c5-0172-407c-814e-1169456d4bda-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-fc745\" (UID: \"e3af58c5-0172-407c-814e-1169456d4bda\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fc745"
Jul 03 14:09:29.188292 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:29.187952    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 03 14:09:29.188292 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:29.188032    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3af58c5-0172-407c-814e-1169456d4bda-networking-console-plugin-cert podName:e3af58c5-0172-407c-814e-1169456d4bda nodeName:}" failed. No retries permitted until 2026-07-03 14:09:33.18801636 +0000 UTC m=+86.638149794 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e3af58c5-0172-407c-814e-1169456d4bda-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-fc745" (UID: "e3af58c5-0172-407c-814e-1169456d4bda") : secret "networking-console-plugin-cert" not found
Jul 03 14:09:29.591887 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:29.591807    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f499c073-70d1-4178-8c52-1e64babb8a37-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v6t5n\" (UID: \"f499c073-70d1-4178-8c52-1e64babb8a37\") " pod="openshift-insights/insights-runtime-extractor-v6t5n"
Jul 03 14:09:29.591887 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:29.591868    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f499c073-70d1-4178-8c52-1e64babb8a37-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v6t5n\" (UID: \"f499c073-70d1-4178-8c52-1e64babb8a37\") " pod="openshift-insights/insights-runtime-extractor-v6t5n"
Jul 03 14:09:29.592094 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:29.591955    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 03 14:09:29.592094 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:29.591962    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 03 14:09:29.592094 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:29.592021    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f499c073-70d1-4178-8c52-1e64babb8a37-kube-rbac-proxy-cm podName:f499c073-70d1-4178-8c52-1e64babb8a37 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:37.592008837 +0000 UTC m=+91.042142255 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f499c073-70d1-4178-8c52-1e64babb8a37-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v6t5n" (UID: "f499c073-70d1-4178-8c52-1e64babb8a37") : configmap "kube-rbac-proxy" not found
Jul 03 14:09:29.592094 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:29.592072    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f499c073-70d1-4178-8c52-1e64babb8a37-insights-runtime-extractor-tls podName:f499c073-70d1-4178-8c52-1e64babb8a37 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:37.592055138 +0000 UTC m=+91.042188556 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f499c073-70d1-4178-8c52-1e64babb8a37-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v6t5n" (UID: "f499c073-70d1-4178-8c52-1e64babb8a37") : secret "insights-runtime-extractor-tls" not found
Jul 03 14:09:33.221692 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:33.221646    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e3af58c5-0172-407c-814e-1169456d4bda-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-fc745\" (UID: \"e3af58c5-0172-407c-814e-1169456d4bda\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fc745"
Jul 03 14:09:33.222246 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:33.221821    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 03 14:09:33.222246 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:33.221920    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3af58c5-0172-407c-814e-1169456d4bda-networking-console-plugin-cert podName:e3af58c5-0172-407c-814e-1169456d4bda nodeName:}" failed. No retries permitted until 2026-07-03 14:09:41.221898273 +0000 UTC m=+94.672031691 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e3af58c5-0172-407c-814e-1169456d4bda-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-fc745" (UID: "e3af58c5-0172-407c-814e-1169456d4bda") : secret "networking-console-plugin-cert" not found
Jul 03 14:09:33.827377 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:33.827324    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d85062a4-19c6-4cc0-9488-19b48622c627-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-88rx9\" (UID: \"d85062a4-19c6-4cc0-9488-19b48622c627\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-88rx9"
Jul 03 14:09:33.827561 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:33.827498    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 03 14:09:33.827603 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:33.827576    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d85062a4-19c6-4cc0-9488-19b48622c627-cluster-monitoring-operator-tls podName:d85062a4-19c6-4cc0-9488-19b48622c627 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:49.827557471 +0000 UTC m=+103.277690906 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d85062a4-19c6-4cc0-9488-19b48622c627-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-88rx9" (UID: "d85062a4-19c6-4cc0-9488-19b48622c627") : secret "cluster-monitoring-operator-tls" not found
Jul 03 14:09:37.660133 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:37.660096    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f499c073-70d1-4178-8c52-1e64babb8a37-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v6t5n\" (UID: \"f499c073-70d1-4178-8c52-1e64babb8a37\") " pod="openshift-insights/insights-runtime-extractor-v6t5n"
Jul 03 14:09:37.660497 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:37.660180    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f499c073-70d1-4178-8c52-1e64babb8a37-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v6t5n\" (UID: \"f499c073-70d1-4178-8c52-1e64babb8a37\") " pod="openshift-insights/insights-runtime-extractor-v6t5n"
Jul 03 14:09:37.660497 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:37.660278    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 03 14:09:37.660497 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:09:37.660329    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f499c073-70d1-4178-8c52-1e64babb8a37-insights-runtime-extractor-tls podName:f499c073-70d1-4178-8c52-1e64babb8a37 nodeName:}" failed. No retries permitted until 2026-07-03 14:09:53.660314971 +0000 UTC m=+107.110448390 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f499c073-70d1-4178-8c52-1e64babb8a37-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v6t5n" (UID: "f499c073-70d1-4178-8c52-1e64babb8a37") : secret "insights-runtime-extractor-tls" not found
Jul 03 14:09:37.660676 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:37.660654    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f499c073-70d1-4178-8c52-1e64babb8a37-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v6t5n\" (UID: \"f499c073-70d1-4178-8c52-1e64babb8a37\") " pod="openshift-insights/insights-runtime-extractor-v6t5n"
Jul 03 14:09:41.289741 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:41.289689    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e3af58c5-0172-407c-814e-1169456d4bda-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-fc745\" (UID: \"e3af58c5-0172-407c-814e-1169456d4bda\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fc745"
Jul 03 14:09:41.292055 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:41.292035    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e3af58c5-0172-407c-814e-1169456d4bda-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-fc745\" (UID: \"e3af58c5-0172-407c-814e-1169456d4bda\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-fc745"
Jul 03 14:09:41.410074 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:41.410038    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-fc745"
Jul 03 14:09:41.544572 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:41.544493    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-fc745"]
Jul 03 14:09:41.547681 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:09:41.547654    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3af58c5_0172_407c_814e_1169456d4bda.slice/crio-c8cc0b36a59371383918d51fb19692c538eadcae5a8fd3c223f5079a246b49b5 WatchSource:0}: Error finding container c8cc0b36a59371383918d51fb19692c538eadcae5a8fd3c223f5079a246b49b5: Status 404 returned error can't find the container with id c8cc0b36a59371383918d51fb19692c538eadcae5a8fd3c223f5079a246b49b5
Jul 03 14:09:42.389879 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:42.389835    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-fc745" event={"ID":"e3af58c5-0172-407c-814e-1169456d4bda","Type":"ContainerStarted","Data":"c8cc0b36a59371383918d51fb19692c538eadcae5a8fd3c223f5079a246b49b5"}
Jul 03 14:09:43.394451 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:43.394415    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-fc745" event={"ID":"e3af58c5-0172-407c-814e-1169456d4bda","Type":"ContainerStarted","Data":"1ea4550bf0991f9c6c4f35bfffcbd6110a09d074b593f9650ef38fa1a9cb97b8"}
Jul 03 14:09:43.415145 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:43.415092    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-fc745" podStartSLOduration=17.233828948 podStartE2EDuration="18.415077098s" totalImagesPullingTime="1.18124815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:09:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:09:43.414087661 +0000 UTC m=+96.864221111" watchObservedRunningTime="2026-07-03 14:09:43.415077098 +0000 UTC m=+96.865210537"
Jul 03 14:09:45.324481 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:45.324430    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e58a995c-7842-4473-b890-7c93a46980f3-metrics-tls\") pod \"dns-default-bf7t2\" (UID: \"e58a995c-7842-4473-b890-7c93a46980f3\") " pod="openshift-dns/dns-default-bf7t2"
Jul 03 14:09:45.324836 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:45.324513    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-tls\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:09:45.326833 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:45.326811    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e58a995c-7842-4473-b890-7c93a46980f3-metrics-tls\") pod \"dns-default-bf7t2\" (UID: \"e58a995c-7842-4473-b890-7c93a46980f3\") " pod="openshift-dns/dns-default-bf7t2"
Jul 03 14:09:45.326943 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:45.326838    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-tls\") pod \"image-registry-6cbb57ffb8-2ccdw\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") " pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:09:45.354841 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:45.354818    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wf6pg\""
Jul 03 14:09:45.362119 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:45.362103    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bf7t2"
Jul 03 14:09:45.491207 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:45.491171    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bf7t2"]
Jul 03 14:09:45.494647 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:09:45.494622    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode58a995c_7842_4473_b890_7c93a46980f3.slice/crio-a93c0f4350172dd0c4855c7bdd1b529298b559860869d81454288edc7f4d1032 WatchSource:0}: Error finding container a93c0f4350172dd0c4855c7bdd1b529298b559860869d81454288edc7f4d1032: Status 404 returned error can't find the container with id a93c0f4350172dd0c4855c7bdd1b529298b559860869d81454288edc7f4d1032
Jul 03 14:09:45.619360 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:45.619326    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-scvmw\""
Jul 03 14:09:45.627357 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:45.627335    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:09:45.755023 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:45.755002    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"]
Jul 03 14:09:45.757369 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:09:45.757347    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb3f129d_0917_4c5b_a369_ac9a0f9e63a2.slice/crio-8f6999e21114d524001a09790ff8f8a565465d2c9eaa381f5514bbc94c082359 WatchSource:0}: Error finding container 8f6999e21114d524001a09790ff8f8a565465d2c9eaa381f5514bbc94c082359: Status 404 returned error can't find the container with id 8f6999e21114d524001a09790ff8f8a565465d2c9eaa381f5514bbc94c082359
Jul 03 14:09:46.407611 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:46.407574    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw" event={"ID":"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2","Type":"ContainerStarted","Data":"227304c1a93973836a67ca2043540a5d609ad798269cf8eb9ec64edd292d1f42"}
Jul 03 14:09:46.408104 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:46.407738    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:09:46.408104 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:46.407779    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw" event={"ID":"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2","Type":"ContainerStarted","Data":"8f6999e21114d524001a09790ff8f8a565465d2c9eaa381f5514bbc94c082359"}
Jul 03 14:09:46.408773 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:46.408749    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bf7t2" event={"ID":"e58a995c-7842-4473-b890-7c93a46980f3","Type":"ContainerStarted","Data":"a93c0f4350172dd0c4855c7bdd1b529298b559860869d81454288edc7f4d1032"}
Jul 03 14:09:46.434529 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:46.434476    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw" podStartSLOduration=99.434455347 podStartE2EDuration="1m39.434455347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:08:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:09:46.433670611 +0000 UTC m=+99.883804050" watchObservedRunningTime="2026-07-03 14:09:46.434455347 +0000 UTC m=+99.884588786"
Jul 03 14:09:47.414394 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:47.414345    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bf7t2" event={"ID":"e58a995c-7842-4473-b890-7c93a46980f3","Type":"ContainerStarted","Data":"44af5d19d29a4b04aaf535dcf2c5a03b43e0d4e98743a75c8a11494eab46654e"}
Jul 03 14:09:47.414768 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:47.414405    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bf7t2" event={"ID":"e58a995c-7842-4473-b890-7c93a46980f3","Type":"ContainerStarted","Data":"251b39e82ae185f887d30bab7bd4d3971c6ad2643ee57da0b806adfd1504091d"}
Jul 03 14:09:47.434277 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:47.434220    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-bf7t2" podStartSLOduration=64.912872928 podStartE2EDuration="1m6.434202413s" totalImagesPullingTime="1.521329485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:08:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:09:47.434019873 +0000 UTC m=+100.884153340" watchObservedRunningTime="2026-07-03 14:09:47.434202413 +0000 UTC m=+100.884335854"
Jul 03 14:09:48.323881 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:48.323849    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4sq59"
Jul 03 14:09:48.418185 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:48.418156    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bf7t2"
Jul 03 14:09:49.860549 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:49.860506    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d85062a4-19c6-4cc0-9488-19b48622c627-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-88rx9\" (UID: \"d85062a4-19c6-4cc0-9488-19b48622c627\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-88rx9"
Jul 03 14:09:49.862824 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:49.862807    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d85062a4-19c6-4cc0-9488-19b48622c627-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-88rx9\" (UID: \"d85062a4-19c6-4cc0-9488-19b48622c627\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-88rx9"
Jul 03 14:09:49.865744 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:49.865726    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-88rx9"
Jul 03 14:09:50.025049 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:50.025019    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-88rx9"]
Jul 03 14:09:50.028161 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:09:50.028127    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd85062a4_19c6_4cc0_9488_19b48622c627.slice/crio-55d497422334ad15647245550162c862c0fcfdada16a9efc701266b1953fc5c2 WatchSource:0}: Error finding container 55d497422334ad15647245550162c862c0fcfdada16a9efc701266b1953fc5c2: Status 404 returned error can't find the container with id 55d497422334ad15647245550162c862c0fcfdada16a9efc701266b1953fc5c2
Jul 03 14:09:50.427657 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:50.427626    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-88rx9" event={"ID":"d85062a4-19c6-4cc0-9488-19b48622c627","Type":"ContainerStarted","Data":"55d497422334ad15647245550162c862c0fcfdada16a9efc701266b1953fc5c2"}
Jul 03 14:09:51.633543 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:51.633504    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bc54959-vrqhx"]
Jul 03 14:09:51.670931 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:51.670896    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bc54959-vrqhx"
Jul 03 14:09:51.676717 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:51.676688    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bc54959-vrqhx"]
Jul 03 14:09:51.681065 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:51.680836    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 03 14:09:51.681065 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:51.680928    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 03 14:09:51.681613 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:51.681581    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 03 14:09:51.690730 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:51.690711    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 03 14:09:51.775644 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:51.775607    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c0ab2fcc-4e82-4890-adec-d0d650d3fb4e-tmp\") pod \"klusterlet-addon-workmgr-76bc54959-vrqhx\" (UID: \"c0ab2fcc-4e82-4890-adec-d0d650d3fb4e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bc54959-vrqhx"
Jul 03 14:09:51.775810 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:51.775672    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkkfl\" (UniqueName: \"kubernetes.io/projected/c0ab2fcc-4e82-4890-adec-d0d650d3fb4e-kube-api-access-bkkfl\") pod \"klusterlet-addon-workmgr-76bc54959-vrqhx\" (UID: \"c0ab2fcc-4e82-4890-adec-d0d650d3fb4e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bc54959-vrqhx"
Jul 03 14:09:51.775810 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:51.775739    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c0ab2fcc-4e82-4890-adec-d0d650d3fb4e-klusterlet-config\") pod \"klusterlet-addon-workmgr-76bc54959-vrqhx\" (UID: \"c0ab2fcc-4e82-4890-adec-d0d650d3fb4e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bc54959-vrqhx"
Jul 03 14:09:51.780570 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:51.780541    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-s7j5g"]
Jul 03 14:09:51.803357 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:51.803313    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-s7j5g"
Jul 03 14:09:51.807671 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:51.807638    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-c2czc\""
Jul 03 14:09:51.809016 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:51.808993    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 03 14:09:51.810177 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:51.810140    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 03 14:09:51.815312 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:51.815288    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-s7j5g"]
Jul 03 14:09:51.876336 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:51.876296    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c0ab2fcc-4e82-4890-adec-d0d650d3fb4e-tmp\") pod \"klusterlet-addon-workmgr-76bc54959-vrqhx\" (UID: \"c0ab2fcc-4e82-4890-adec-d0d650d3fb4e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bc54959-vrqhx"
Jul 03 14:09:51.876517 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:51.876366    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkkfl\" (UniqueName: \"kubernetes.io/projected/c0ab2fcc-4e82-4890-adec-d0d650d3fb4e-kube-api-access-bkkfl\") pod \"klusterlet-addon-workmgr-76bc54959-vrqhx\" (UID: \"c0ab2fcc-4e82-4890-adec-d0d650d3fb4e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bc54959-vrqhx"
Jul 03 14:09:51.876517 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:51.876418    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c0ab2fcc-4e82-4890-adec-d0d650d3fb4e-klusterlet-config\") pod \"klusterlet-addon-workmgr-76bc54959-vrqhx\" (UID: \"c0ab2fcc-4e82-4890-adec-d0d650d3fb4e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bc54959-vrqhx"
Jul 03 14:09:51.876780 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:51.876758    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c0ab2fcc-4e82-4890-adec-d0d650d3fb4e-tmp\") pod \"klusterlet-addon-workmgr-76bc54959-vrqhx\" (UID: \"c0ab2fcc-4e82-4890-adec-d0d650d3fb4e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bc54959-vrqhx"
Jul 03 14:09:51.879253 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:51.879221    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c0ab2fcc-4e82-4890-adec-d0d650d3fb4e-klusterlet-config\") pod \"klusterlet-addon-workmgr-76bc54959-vrqhx\" (UID: \"c0ab2fcc-4e82-4890-adec-d0d650d3fb4e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bc54959-vrqhx"
Jul 03 14:09:51.920865 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:51.920780    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkkfl\" (UniqueName: \"kubernetes.io/projected/c0ab2fcc-4e82-4890-adec-d0d650d3fb4e-kube-api-access-bkkfl\") pod \"klusterlet-addon-workmgr-76bc54959-vrqhx\" (UID: \"c0ab2fcc-4e82-4890-adec-d0d650d3fb4e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bc54959-vrqhx"
Jul 03 14:09:51.977756 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:51.977715    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7gk2\" (UniqueName: \"kubernetes.io/projected/ee8e766e-8e65-4d2d-abc5-7b7a0dc21b33-kube-api-access-c7gk2\") pod \"downloads-685bfb5f64-s7j5g\" (UID: \"ee8e766e-8e65-4d2d-abc5-7b7a0dc21b33\") " pod="openshift-console/downloads-685bfb5f64-s7j5g"
Jul 03 14:09:51.983614 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:51.983586    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bc54959-vrqhx"
Jul 03 14:09:52.079008 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:52.078949    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7gk2\" (UniqueName: \"kubernetes.io/projected/ee8e766e-8e65-4d2d-abc5-7b7a0dc21b33-kube-api-access-c7gk2\") pod \"downloads-685bfb5f64-s7j5g\" (UID: \"ee8e766e-8e65-4d2d-abc5-7b7a0dc21b33\") " pod="openshift-console/downloads-685bfb5f64-s7j5g"
Jul 03 14:09:52.097407 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:52.097379    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7gk2\" (UniqueName: \"kubernetes.io/projected/ee8e766e-8e65-4d2d-abc5-7b7a0dc21b33-kube-api-access-c7gk2\") pod \"downloads-685bfb5f64-s7j5g\" (UID: \"ee8e766e-8e65-4d2d-abc5-7b7a0dc21b33\") " pod="openshift-console/downloads-685bfb5f64-s7j5g"
Jul 03 14:09:52.114791 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:52.114761    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-s7j5g"
Jul 03 14:09:52.384602 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:52.384537    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-s7j5g"]
Jul 03 14:09:52.387652 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:52.387629    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bc54959-vrqhx"]
Jul 03 14:09:52.551220 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:09:52.551147    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0ab2fcc_4e82_4890_adec_d0d650d3fb4e.slice/crio-f376f7af5f169d42b2cfb5a61fbd862fa4b71a02597fca0a4602848b38cb3f21 WatchSource:0}: Error finding container f376f7af5f169d42b2cfb5a61fbd862fa4b71a02597fca0a4602848b38cb3f21: Status 404 returned error can't find the container with id f376f7af5f169d42b2cfb5a61fbd862fa4b71a02597fca0a4602848b38cb3f21
Jul 03 14:09:52.551732 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:09:52.551714    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee8e766e_8e65_4d2d_abc5_7b7a0dc21b33.slice/crio-1596a28de959e59c041f8be79f60d691a3ef8be790c5485f4c4ab024af142dfc WatchSource:0}: Error finding container 1596a28de959e59c041f8be79f60d691a3ef8be790c5485f4c4ab024af142dfc: Status 404 returned error can't find the container with id 1596a28de959e59c041f8be79f60d691a3ef8be790c5485f4c4ab024af142dfc
Jul 03 14:09:53.205566 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:53.205518    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-77xcg"]
Jul 03 14:09:53.207721 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:53.207687    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-77xcg"
Jul 03 14:09:53.210099 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:53.210080    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 03 14:09:53.210221 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:53.210195    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-vkcrw\""
Jul 03 14:09:53.219125 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:53.219070    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-77xcg"]
Jul 03 14:09:53.388695 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:53.388656    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/815607d1-89de-4e62-9a6f-301217b897ef-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-77xcg\" (UID: \"815607d1-89de-4e62-9a6f-301217b897ef\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-77xcg"
Jul 03 14:09:53.439810 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:53.439745    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bc54959-vrqhx" event={"ID":"c0ab2fcc-4e82-4890-adec-d0d650d3fb4e","Type":"ContainerStarted","Data":"f376f7af5f169d42b2cfb5a61fbd862fa4b71a02597fca0a4602848b38cb3f21"}
Jul 03 14:09:53.441418 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:53.441384    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-88rx9" event={"ID":"d85062a4-19c6-4cc0-9488-19b48622c627","Type":"ContainerStarted","Data":"19a4590c19c7868d84e97a676195876ba5b4e85a5d9e5b54404f9570238e4ae9"}
Jul 03 14:09:53.442705 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:53.442678    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-s7j5g" event={"ID":"ee8e766e-8e65-4d2d-abc5-7b7a0dc21b33","Type":"ContainerStarted","Data":"1596a28de959e59c041f8be79f60d691a3ef8be790c5485f4c4ab024af142dfc"}
Jul 03 14:09:53.462231 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:53.462035    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-88rx9" podStartSLOduration=32.907043093 podStartE2EDuration="35.462016768s" totalImagesPullingTime="2.554973675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:09:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:09:53.461899037 +0000 UTC m=+106.912032480" watchObservedRunningTime="2026-07-03 14:09:53.462016768 +0000 UTC m=+106.912150208"
Jul 03 14:09:53.489458 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:53.489396    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/815607d1-89de-4e62-9a6f-301217b897ef-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-77xcg\" (UID: \"815607d1-89de-4e62-9a6f-301217b897ef\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-77xcg"
Jul 03 14:09:53.492434 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:53.492384    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/815607d1-89de-4e62-9a6f-301217b897ef-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-77xcg\" (UID: \"815607d1-89de-4e62-9a6f-301217b897ef\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-77xcg"
Jul 03 14:09:53.521003 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:53.520719    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-77xcg"
Jul 03 14:09:53.692600 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:53.692120    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f499c073-70d1-4178-8c52-1e64babb8a37-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v6t5n\" (UID: \"f499c073-70d1-4178-8c52-1e64babb8a37\") " pod="openshift-insights/insights-runtime-extractor-v6t5n"
Jul 03 14:09:53.693853 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:53.693781    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-77xcg"]
Jul 03 14:09:53.696254 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:53.696224    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f499c073-70d1-4178-8c52-1e64babb8a37-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v6t5n\" (UID: \"f499c073-70d1-4178-8c52-1e64babb8a37\") " pod="openshift-insights/insights-runtime-extractor-v6t5n"
Jul 03 14:09:53.698588 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:09:53.698540    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod815607d1_89de_4e62_9a6f_301217b897ef.slice/crio-1420f514a2dca2d05f0685a9435c3489cfdb2fe02f5897a929135c4b94c407a0 WatchSource:0}: Error finding container 1420f514a2dca2d05f0685a9435c3489cfdb2fe02f5897a929135c4b94c407a0: Status 404 returned error can't find the container with id 1420f514a2dca2d05f0685a9435c3489cfdb2fe02f5897a929135c4b94c407a0
Jul 03 14:09:53.959026 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:53.958987    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v6t5n"
Jul 03 14:09:54.139334 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:54.139239    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v6t5n"]
Jul 03 14:09:54.142937 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:09:54.142906    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf499c073_70d1_4178_8c52_1e64babb8a37.slice/crio-4c9a0f3f7e207f02a2dfdb3505a54e50b860f7b8a1e9281d9f1f8b2459db7505 WatchSource:0}: Error finding container 4c9a0f3f7e207f02a2dfdb3505a54e50b860f7b8a1e9281d9f1f8b2459db7505: Status 404 returned error can't find the container with id 4c9a0f3f7e207f02a2dfdb3505a54e50b860f7b8a1e9281d9f1f8b2459db7505
Jul 03 14:09:54.449921 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:54.448721    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-77xcg" event={"ID":"815607d1-89de-4e62-9a6f-301217b897ef","Type":"ContainerStarted","Data":"1420f514a2dca2d05f0685a9435c3489cfdb2fe02f5897a929135c4b94c407a0"}
Jul 03 14:09:54.452586 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:54.452554    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v6t5n" event={"ID":"f499c073-70d1-4178-8c52-1e64babb8a37","Type":"ContainerStarted","Data":"4c9a0f3f7e207f02a2dfdb3505a54e50b860f7b8a1e9281d9f1f8b2459db7505"}
Jul 03 14:09:56.615619 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:56.615552    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b94d7cd68-wnlbl"]
Jul 03 14:09:56.625196 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:56.625165    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b94d7cd68-wnlbl"
Jul 03 14:09:56.628274 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:56.628237    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 03 14:09:56.628439 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:56.628417    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 03 14:09:56.629383 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:56.629361    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 03 14:09:56.629493 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:56.629466    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 03 14:09:56.629871 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:56.629658    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-svd8d\""
Jul 03 14:09:56.629871 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:56.629700    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 03 14:09:56.637215 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:56.637153    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b94d7cd68-wnlbl"]
Jul 03 14:09:56.721599 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:56.721276    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31c13a3d-642e-4dae-b3f4-0d8596014470-console-oauth-config\") pod \"console-7b94d7cd68-wnlbl\" (UID: \"31c13a3d-642e-4dae-b3f4-0d8596014470\") " pod="openshift-console/console-7b94d7cd68-wnlbl"
Jul 03 14:09:56.721599 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:56.721332    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppmfj\" (UniqueName: \"kubernetes.io/projected/31c13a3d-642e-4dae-b3f4-0d8596014470-kube-api-access-ppmfj\") pod \"console-7b94d7cd68-wnlbl\" (UID: \"31c13a3d-642e-4dae-b3f4-0d8596014470\") " pod="openshift-console/console-7b94d7cd68-wnlbl"
Jul 03 14:09:56.721599 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:56.721393    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31c13a3d-642e-4dae-b3f4-0d8596014470-console-serving-cert\") pod \"console-7b94d7cd68-wnlbl\" (UID: \"31c13a3d-642e-4dae-b3f4-0d8596014470\") " pod="openshift-console/console-7b94d7cd68-wnlbl"
Jul 03 14:09:56.721599 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:56.721426    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31c13a3d-642e-4dae-b3f4-0d8596014470-oauth-serving-cert\") pod \"console-7b94d7cd68-wnlbl\" (UID: \"31c13a3d-642e-4dae-b3f4-0d8596014470\") " pod="openshift-console/console-7b94d7cd68-wnlbl"
Jul 03 14:09:56.721599 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:56.721483    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31c13a3d-642e-4dae-b3f4-0d8596014470-console-config\") pod \"console-7b94d7cd68-wnlbl\" (UID: \"31c13a3d-642e-4dae-b3f4-0d8596014470\") " pod="openshift-console/console-7b94d7cd68-wnlbl"
Jul 03 14:09:56.721599 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:56.721539    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31c13a3d-642e-4dae-b3f4-0d8596014470-service-ca\") pod \"console-7b94d7cd68-wnlbl\" (UID: \"31c13a3d-642e-4dae-b3f4-0d8596014470\") " pod="openshift-console/console-7b94d7cd68-wnlbl"
Jul 03 14:09:56.822392 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:56.822356    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31c13a3d-642e-4dae-b3f4-0d8596014470-service-ca\") pod \"console-7b94d7cd68-wnlbl\" (UID: \"31c13a3d-642e-4dae-b3f4-0d8596014470\") " pod="openshift-console/console-7b94d7cd68-wnlbl"
Jul 03 14:09:56.822575 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:56.822404    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31c13a3d-642e-4dae-b3f4-0d8596014470-console-oauth-config\") pod \"console-7b94d7cd68-wnlbl\" (UID: \"31c13a3d-642e-4dae-b3f4-0d8596014470\") " pod="openshift-console/console-7b94d7cd68-wnlbl"
Jul 03 14:09:56.822640 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:56.822596    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppmfj\" (UniqueName: \"kubernetes.io/projected/31c13a3d-642e-4dae-b3f4-0d8596014470-kube-api-access-ppmfj\") pod \"console-7b94d7cd68-wnlbl\" (UID: \"31c13a3d-642e-4dae-b3f4-0d8596014470\") " pod="openshift-console/console-7b94d7cd68-wnlbl"
Jul 03 14:09:56.822693 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:56.822642    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31c13a3d-642e-4dae-b3f4-0d8596014470-console-serving-cert\") pod \"console-7b94d7cd68-wnlbl\" (UID: \"31c13a3d-642e-4dae-b3f4-0d8596014470\") " pod="openshift-console/console-7b94d7cd68-wnlbl"
Jul 03 14:09:56.822693 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:56.822685    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31c13a3d-642e-4dae-b3f4-0d8596014470-oauth-serving-cert\") pod \"console-7b94d7cd68-wnlbl\" (UID: \"31c13a3d-642e-4dae-b3f4-0d8596014470\") " pod="openshift-console/console-7b94d7cd68-wnlbl"
Jul 03 14:09:56.822755 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:56.822743    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31c13a3d-642e-4dae-b3f4-0d8596014470-console-config\") pod \"console-7b94d7cd68-wnlbl\" (UID: \"31c13a3d-642e-4dae-b3f4-0d8596014470\") " pod="openshift-console/console-7b94d7cd68-wnlbl"
Jul 03 14:09:56.823224 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:56.823198    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31c13a3d-642e-4dae-b3f4-0d8596014470-service-ca\") pod \"console-7b94d7cd68-wnlbl\" (UID: \"31c13a3d-642e-4dae-b3f4-0d8596014470\") " pod="openshift-console/console-7b94d7cd68-wnlbl"
Jul 03 14:09:56.823455 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:56.823393    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31c13a3d-642e-4dae-b3f4-0d8596014470-console-config\") pod \"console-7b94d7cd68-wnlbl\" (UID: \"31c13a3d-642e-4dae-b3f4-0d8596014470\") " pod="openshift-console/console-7b94d7cd68-wnlbl"
Jul 03 14:09:56.823573 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:56.823471    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31c13a3d-642e-4dae-b3f4-0d8596014470-oauth-serving-cert\") pod \"console-7b94d7cd68-wnlbl\" (UID: \"31c13a3d-642e-4dae-b3f4-0d8596014470\") " pod="openshift-console/console-7b94d7cd68-wnlbl"
Jul 03 14:09:56.825713 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:56.825691    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31c13a3d-642e-4dae-b3f4-0d8596014470-console-oauth-config\") pod \"console-7b94d7cd68-wnlbl\" (UID: \"31c13a3d-642e-4dae-b3f4-0d8596014470\") " pod="openshift-console/console-7b94d7cd68-wnlbl"
Jul 03 14:09:56.826729 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:56.826707    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31c13a3d-642e-4dae-b3f4-0d8596014470-console-serving-cert\") pod \"console-7b94d7cd68-wnlbl\" (UID: \"31c13a3d-642e-4dae-b3f4-0d8596014470\") " pod="openshift-console/console-7b94d7cd68-wnlbl"
Jul 03 14:09:56.834130 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:56.834105    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppmfj\" (UniqueName: \"kubernetes.io/projected/31c13a3d-642e-4dae-b3f4-0d8596014470-kube-api-access-ppmfj\") pod \"console-7b94d7cd68-wnlbl\" (UID: \"31c13a3d-642e-4dae-b3f4-0d8596014470\") " pod="openshift-console/console-7b94d7cd68-wnlbl"
Jul 03 14:09:56.937236 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:56.937130    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b94d7cd68-wnlbl"
Jul 03 14:09:58.423624 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:09:58.423593    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bf7t2"
Jul 03 14:10:00.335764 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:00.335653    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b94d7cd68-wnlbl"]
Jul 03 14:10:00.341264 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:10:00.341228    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31c13a3d_642e_4dae_b3f4_0d8596014470.slice/crio-c217c1f2bbcc6fdcc403ff69a4926c7ae598eef0cf8978b9b83916ae30556e0d WatchSource:0}: Error finding container c217c1f2bbcc6fdcc403ff69a4926c7ae598eef0cf8978b9b83916ae30556e0d: Status 404 returned error can't find the container with id c217c1f2bbcc6fdcc403ff69a4926c7ae598eef0cf8978b9b83916ae30556e0d
Jul 03 14:10:00.480689 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:00.480592    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-77xcg" event={"ID":"815607d1-89de-4e62-9a6f-301217b897ef","Type":"ContainerStarted","Data":"d3703d2bf3d289ec487ae61c7af7c2df70eedf32e5535a16311d51997a996918"}
Jul 03 14:10:00.480866 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:00.480791    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-77xcg"
Jul 03 14:10:00.482403 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:00.482343    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v6t5n" event={"ID":"f499c073-70d1-4178-8c52-1e64babb8a37","Type":"ContainerStarted","Data":"e70f6d8761449211abef4cd6edf1dc81113d9494856ae5857fec8407e9e89729"}
Jul 03 14:10:00.483891 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:00.483867    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b94d7cd68-wnlbl" event={"ID":"31c13a3d-642e-4dae-b3f4-0d8596014470","Type":"ContainerStarted","Data":"c217c1f2bbcc6fdcc403ff69a4926c7ae598eef0cf8978b9b83916ae30556e0d"}
Jul 03 14:10:00.485494 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:00.485443    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bc54959-vrqhx" event={"ID":"c0ab2fcc-4e82-4890-adec-d0d650d3fb4e","Type":"ContainerStarted","Data":"4440b17814e0a8e4fcb47abd07fd11d142606e9fb101da19760dce1ee64575a3"}
Jul 03 14:10:00.485701 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:00.485681    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bc54959-vrqhx"
Jul 03 14:10:00.487138 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:00.487068    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bc54959-vrqhx"
Jul 03 14:10:00.487138 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:00.487125    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-77xcg"
Jul 03 14:10:00.529595 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:00.529444    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-77xcg" podStartSLOduration=1.059110029 podStartE2EDuration="7.529423058s" totalImagesPullingTime="6.470313029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:09:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:10:00.529355774 +0000 UTC m=+113.979489215" watchObservedRunningTime="2026-07-03 14:10:00.529423058 +0000 UTC m=+113.979556494"
Jul 03 14:10:00.594094 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:00.594040    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76bc54959-vrqhx" podStartSLOduration=2.001248101 podStartE2EDuration="9.594022039s" totalImagesPullingTime="7.592773938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:09:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:10:00.592826577 +0000 UTC m=+114.042960016" watchObservedRunningTime="2026-07-03 14:10:00.594022039 +0000 UTC m=+114.044155478"
Jul 03 14:10:01.491410 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:01.491370    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v6t5n" event={"ID":"f499c073-70d1-4178-8c52-1e64babb8a37","Type":"ContainerStarted","Data":"18c834e2afbd3fff5382b5025db556ba70aa18ec0be4fb6d5ca46da4948b9cfb"}
Jul 03 14:10:05.633916 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:05.633875    2563 patch_prober.go:28] interesting pod/image-registry-6cbb57ffb8-2ccdw container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 03 14:10:05.634399 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:05.633944    2563 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw" podUID="cb3f129d-0917-4c5b-a369-ac9a0f9e63a2" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 03 14:10:06.766574 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.766532    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-d77cl"]
Jul 03 14:10:06.771269 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.771248    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-d77cl"
Jul 03 14:10:06.777937 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.777844    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 03 14:10:06.779000 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.778948    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 03 14:10:06.779813 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.779784    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-qdn6l\""
Jul 03 14:10:06.779986 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.779944    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 03 14:10:06.782881 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.782859    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-d77cl"]
Jul 03 14:10:06.809203 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.809163    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af196fa9-9734-4b49-a5cd-43c50cf3e54e-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-d77cl\" (UID: \"af196fa9-9734-4b49-a5cd-43c50cf3e54e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-d77cl"
Jul 03 14:10:06.809387 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.809303    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pfpg\" (UniqueName: \"kubernetes.io/projected/af196fa9-9734-4b49-a5cd-43c50cf3e54e-kube-api-access-9pfpg\") pod \"openshift-state-metrics-54cf96dfcf-d77cl\" (UID: \"af196fa9-9734-4b49-a5cd-43c50cf3e54e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-d77cl"
Jul 03 14:10:06.809387 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.809347    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af196fa9-9734-4b49-a5cd-43c50cf3e54e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-d77cl\" (UID: \"af196fa9-9734-4b49-a5cd-43c50cf3e54e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-d77cl"
Jul 03 14:10:06.809499 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.809457    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/af196fa9-9734-4b49-a5cd-43c50cf3e54e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-d77cl\" (UID: \"af196fa9-9734-4b49-a5cd-43c50cf3e54e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-d77cl"
Jul 03 14:10:06.810370 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.810344    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-b59zs"]
Jul 03 14:10:06.813785 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.813764    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:06.816933 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.816906    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 03 14:10:06.816933 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.816922    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 03 14:10:06.817328 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.817307    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-54cgl\""
Jul 03 14:10:06.817420 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.817364    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 03 14:10:06.910346 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.910311    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/79e2d165-3d12-4c84-804b-4d7c857419ef-node-exporter-wtmp\") pod \"node-exporter-b59zs\" (UID: \"79e2d165-3d12-4c84-804b-4d7c857419ef\") " pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:06.910515 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.910351    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af196fa9-9734-4b49-a5cd-43c50cf3e54e-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-d77cl\" (UID: \"af196fa9-9734-4b49-a5cd-43c50cf3e54e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-d77cl"
Jul 03 14:10:06.910515 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.910375    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/79e2d165-3d12-4c84-804b-4d7c857419ef-node-exporter-tls\") pod \"node-exporter-b59zs\" (UID: \"79e2d165-3d12-4c84-804b-4d7c857419ef\") " pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:06.910515 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.910400    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/79e2d165-3d12-4c84-804b-4d7c857419ef-node-exporter-accelerators-collector-config\") pod \"node-exporter-b59zs\" (UID: \"79e2d165-3d12-4c84-804b-4d7c857419ef\") " pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:06.910515 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.910441    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/79e2d165-3d12-4c84-804b-4d7c857419ef-root\") pod \"node-exporter-b59zs\" (UID: \"79e2d165-3d12-4c84-804b-4d7c857419ef\") " pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:06.910515 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.910491    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vczrz\" (UniqueName: \"kubernetes.io/projected/79e2d165-3d12-4c84-804b-4d7c857419ef-kube-api-access-vczrz\") pod \"node-exporter-b59zs\" (UID: \"79e2d165-3d12-4c84-804b-4d7c857419ef\") " pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:06.910798 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.910530    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pfpg\" (UniqueName: \"kubernetes.io/projected/af196fa9-9734-4b49-a5cd-43c50cf3e54e-kube-api-access-9pfpg\") pod \"openshift-state-metrics-54cf96dfcf-d77cl\" (UID: \"af196fa9-9734-4b49-a5cd-43c50cf3e54e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-d77cl"
Jul 03 14:10:06.910798 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.910602    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af196fa9-9734-4b49-a5cd-43c50cf3e54e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-d77cl\" (UID: \"af196fa9-9734-4b49-a5cd-43c50cf3e54e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-d77cl"
Jul 03 14:10:06.910798 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.910656    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79e2d165-3d12-4c84-804b-4d7c857419ef-metrics-client-ca\") pod \"node-exporter-b59zs\" (UID: \"79e2d165-3d12-4c84-804b-4d7c857419ef\") " pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:06.910798 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.910694    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/79e2d165-3d12-4c84-804b-4d7c857419ef-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b59zs\" (UID: \"79e2d165-3d12-4c84-804b-4d7c857419ef\") " pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:06.910798 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.910748    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/79e2d165-3d12-4c84-804b-4d7c857419ef-sys\") pod \"node-exporter-b59zs\" (UID: \"79e2d165-3d12-4c84-804b-4d7c857419ef\") " pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:06.910798 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.910789    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/af196fa9-9734-4b49-a5cd-43c50cf3e54e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-d77cl\" (UID: \"af196fa9-9734-4b49-a5cd-43c50cf3e54e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-d77cl"
Jul 03 14:10:06.911141 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.910813    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/79e2d165-3d12-4c84-804b-4d7c857419ef-node-exporter-textfile\") pod \"node-exporter-b59zs\" (UID: \"79e2d165-3d12-4c84-804b-4d7c857419ef\") " pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:06.911257 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.911226    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af196fa9-9734-4b49-a5cd-43c50cf3e54e-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-d77cl\" (UID: \"af196fa9-9734-4b49-a5cd-43c50cf3e54e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-d77cl"
Jul 03 14:10:06.913401 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.913372    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af196fa9-9734-4b49-a5cd-43c50cf3e54e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-d77cl\" (UID: \"af196fa9-9734-4b49-a5cd-43c50cf3e54e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-d77cl"
Jul 03 14:10:06.913656 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.913637    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/af196fa9-9734-4b49-a5cd-43c50cf3e54e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-d77cl\" (UID: \"af196fa9-9734-4b49-a5cd-43c50cf3e54e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-d77cl"
Jul 03 14:10:06.920584 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:06.920559    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pfpg\" (UniqueName: \"kubernetes.io/projected/af196fa9-9734-4b49-a5cd-43c50cf3e54e-kube-api-access-9pfpg\") pod \"openshift-state-metrics-54cf96dfcf-d77cl\" (UID: \"af196fa9-9734-4b49-a5cd-43c50cf3e54e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-d77cl"
Jul 03 14:10:07.012076 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.012034    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/79e2d165-3d12-4c84-804b-4d7c857419ef-root\") pod \"node-exporter-b59zs\" (UID: \"79e2d165-3d12-4c84-804b-4d7c857419ef\") " pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:07.012246 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.012091    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vczrz\" (UniqueName: \"kubernetes.io/projected/79e2d165-3d12-4c84-804b-4d7c857419ef-kube-api-access-vczrz\") pod \"node-exporter-b59zs\" (UID: \"79e2d165-3d12-4c84-804b-4d7c857419ef\") " pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:07.012246 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.012166    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/79e2d165-3d12-4c84-804b-4d7c857419ef-root\") pod \"node-exporter-b59zs\" (UID: \"79e2d165-3d12-4c84-804b-4d7c857419ef\") " pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:07.012246 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.012223    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79e2d165-3d12-4c84-804b-4d7c857419ef-metrics-client-ca\") pod \"node-exporter-b59zs\" (UID: \"79e2d165-3d12-4c84-804b-4d7c857419ef\") " pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:07.012376 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.012261    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/79e2d165-3d12-4c84-804b-4d7c857419ef-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b59zs\" (UID: \"79e2d165-3d12-4c84-804b-4d7c857419ef\") " pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:07.012376 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.012309    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/79e2d165-3d12-4c84-804b-4d7c857419ef-sys\") pod \"node-exporter-b59zs\" (UID: \"79e2d165-3d12-4c84-804b-4d7c857419ef\") " pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:07.012376 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.012347    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/79e2d165-3d12-4c84-804b-4d7c857419ef-node-exporter-textfile\") pod \"node-exporter-b59zs\" (UID: \"79e2d165-3d12-4c84-804b-4d7c857419ef\") " pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:07.012508 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.012388    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/79e2d165-3d12-4c84-804b-4d7c857419ef-node-exporter-wtmp\") pod \"node-exporter-b59zs\" (UID: \"79e2d165-3d12-4c84-804b-4d7c857419ef\") " pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:07.012508 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.012415    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/79e2d165-3d12-4c84-804b-4d7c857419ef-node-exporter-tls\") pod \"node-exporter-b59zs\" (UID: \"79e2d165-3d12-4c84-804b-4d7c857419ef\") " pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:07.012508 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.012441    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/79e2d165-3d12-4c84-804b-4d7c857419ef-node-exporter-accelerators-collector-config\") pod \"node-exporter-b59zs\" (UID: \"79e2d165-3d12-4c84-804b-4d7c857419ef\") " pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:07.012651 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.012625    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/79e2d165-3d12-4c84-804b-4d7c857419ef-node-exporter-wtmp\") pod \"node-exporter-b59zs\" (UID: \"79e2d165-3d12-4c84-804b-4d7c857419ef\") " pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:07.012701 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.012669    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/79e2d165-3d12-4c84-804b-4d7c857419ef-sys\") pod \"node-exporter-b59zs\" (UID: \"79e2d165-3d12-4c84-804b-4d7c857419ef\") " pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:07.013217 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.013187    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/79e2d165-3d12-4c84-804b-4d7c857419ef-node-exporter-textfile\") pod \"node-exporter-b59zs\" (UID: \"79e2d165-3d12-4c84-804b-4d7c857419ef\") " pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:07.013455 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.013417    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79e2d165-3d12-4c84-804b-4d7c857419ef-metrics-client-ca\") pod \"node-exporter-b59zs\" (UID: \"79e2d165-3d12-4c84-804b-4d7c857419ef\") " pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:07.017558 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.017487    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 03 14:10:07.017558 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.017487    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 03 14:10:07.017558 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.017540    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 03 14:10:07.021681 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.021621    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vczrz\" (UniqueName: \"kubernetes.io/projected/79e2d165-3d12-4c84-804b-4d7c857419ef-kube-api-access-vczrz\") pod \"node-exporter-b59zs\" (UID: \"79e2d165-3d12-4c84-804b-4d7c857419ef\") " pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:07.023944 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.023776    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/79e2d165-3d12-4c84-804b-4d7c857419ef-node-exporter-accelerators-collector-config\") pod \"node-exporter-b59zs\" (UID: \"79e2d165-3d12-4c84-804b-4d7c857419ef\") " pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:07.023944 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:10:07.023844    2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 03 14:10:07.023944 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:10:07.023917    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79e2d165-3d12-4c84-804b-4d7c857419ef-node-exporter-tls podName:79e2d165-3d12-4c84-804b-4d7c857419ef nodeName:}" failed. No retries permitted until 2026-07-03 14:10:07.523897226 +0000 UTC m=+120.974030649 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/79e2d165-3d12-4c84-804b-4d7c857419ef-node-exporter-tls") pod "node-exporter-b59zs" (UID: "79e2d165-3d12-4c84-804b-4d7c857419ef") : secret "node-exporter-tls" not found
Jul 03 14:10:07.026905 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.026858    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/79e2d165-3d12-4c84-804b-4d7c857419ef-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b59zs\" (UID: \"79e2d165-3d12-4c84-804b-4d7c857419ef\") " pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:07.085672 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.085637    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-qdn6l\""
Jul 03 14:10:07.093152 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.093114    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-d77cl"
Jul 03 14:10:07.419290 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.419252    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:10:07.618887 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.618851    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/79e2d165-3d12-4c84-804b-4d7c857419ef-node-exporter-tls\") pod \"node-exporter-b59zs\" (UID: \"79e2d165-3d12-4c84-804b-4d7c857419ef\") " pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:07.621342 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.621316    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/79e2d165-3d12-4c84-804b-4d7c857419ef-node-exporter-tls\") pod \"node-exporter-b59zs\" (UID: \"79e2d165-3d12-4c84-804b-4d7c857419ef\") " pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:07.726719 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.726629    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-54cgl\""
Jul 03 14:10:07.734618 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.734584    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b59zs"
Jul 03 14:10:07.787825 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.787787    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 03 14:10:07.791767 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.791741    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.794628 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.794606    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 03 14:10:07.794750 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.794608    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 03 14:10:07.795026 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.795009    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 03 14:10:07.795125 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.795010    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-r7pvf\""
Jul 03 14:10:07.795388 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.795366    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 03 14:10:07.795481 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.795401    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 03 14:10:07.795577 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.795562    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 03 14:10:07.796095 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.795868    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 03 14:10:07.796095 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.795936    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 03 14:10:07.796095 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.795876    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 03 14:10:07.811215 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.811173    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 03 14:10:07.820594 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.820564    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.820739 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.820607    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mnhv\" (UniqueName: \"kubernetes.io/projected/9f129596-3617-4931-9dbe-5c3a4469e16a-kube-api-access-2mnhv\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.820739 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.820659    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f129596-3617-4931-9dbe-5c3a4469e16a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.820739 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.820698    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9f129596-3617-4931-9dbe-5c3a4469e16a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.820739 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.820715    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-config-volume\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.820739 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.820728    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9f129596-3617-4931-9dbe-5c3a4469e16a-config-out\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.821005 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.820743    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.821005 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.820790    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f129596-3617-4931-9dbe-5c3a4469e16a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.821005 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.820828    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-web-config\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.821005 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.820856    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.821005 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.820894    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9f129596-3617-4931-9dbe-5c3a4469e16a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.821005 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.820914    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.821005 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.820932    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.921685 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.921635    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.921879 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.921706    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9f129596-3617-4931-9dbe-5c3a4469e16a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.921879 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.921737    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.921879 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.921766    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.922079 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.921907    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.922079 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.921953    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mnhv\" (UniqueName: \"kubernetes.io/projected/9f129596-3617-4931-9dbe-5c3a4469e16a-kube-api-access-2mnhv\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.922079 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.922035    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f129596-3617-4931-9dbe-5c3a4469e16a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.922239 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.922085    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9f129596-3617-4931-9dbe-5c3a4469e16a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.922239 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.922111    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-config-volume\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.922902 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.922857    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f129596-3617-4931-9dbe-5c3a4469e16a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.923068 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.922957    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9f129596-3617-4931-9dbe-5c3a4469e16a-config-out\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.923068 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.922995    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9f129596-3617-4931-9dbe-5c3a4469e16a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.923068 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.923014    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.923243 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.923071    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f129596-3617-4931-9dbe-5c3a4469e16a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.923243 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.923110    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-web-config\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.923419 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:10:07.923397    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9f129596-3617-4931-9dbe-5c3a4469e16a-alertmanager-trusted-ca-bundle podName:9f129596-3617-4931-9dbe-5c3a4469e16a nodeName:}" failed. No retries permitted until 2026-07-03 14:10:08.42337641 +0000 UTC m=+121.873509833 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/9f129596-3617-4931-9dbe-5c3a4469e16a-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "9f129596-3617-4931-9dbe-5c3a4469e16a") : configmap references non-existent config key: ca-bundle.crt
Jul 03 14:10:07.925462 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.925385    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.925462 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.925415    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9f129596-3617-4931-9dbe-5c3a4469e16a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.925625 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.925561    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.926205 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.926184    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.926493 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.926449    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.926593 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.926492    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.927230 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.927208    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-config-volume\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.927324 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.927261    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9f129596-3617-4931-9dbe-5c3a4469e16a-config-out\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.927660 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.927637    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-web-config\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:07.945478 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:07.945449    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mnhv\" (UniqueName: \"kubernetes.io/projected/9f129596-3617-4931-9dbe-5c3a4469e16a-kube-api-access-2mnhv\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:08.428044 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:08.428002    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f129596-3617-4931-9dbe-5c3a4469e16a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:08.428837 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:08.428804    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f129596-3617-4931-9dbe-5c3a4469e16a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:08.703885 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:08.703796    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:10:09.807671 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:09.807627    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5b8865b898-vrxv9"]
Jul 03 14:10:09.864195 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:09.864159    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5b8865b898-vrxv9"]
Jul 03 14:10:09.864357 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:09.864232    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b94d7cd68-wnlbl"]
Jul 03 14:10:09.864357 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:09.864302    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9"
Jul 03 14:10:09.867213 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:09.867191    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 03 14:10:09.867359 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:09.867191    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 03 14:10:09.867549 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:09.867526    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-6n2v2gekdrehi\""
Jul 03 14:10:09.867633 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:09.867617    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 03 14:10:09.868061 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:09.868022    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-qtqpx\""
Jul 03 14:10:09.868175 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:09.868064    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 03 14:10:09.868733 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:09.868714    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 03 14:10:09.941462 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:09.941431    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/140d5eb5-547f-4490-aa12-f34472b8a472-metrics-client-ca\") pod \"thanos-querier-5b8865b898-vrxv9\" (UID: \"140d5eb5-547f-4490-aa12-f34472b8a472\") " pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9"
Jul 03 14:10:09.941655 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:09.941476    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/140d5eb5-547f-4490-aa12-f34472b8a472-secret-thanos-querier-tls\") pod \"thanos-querier-5b8865b898-vrxv9\" (UID: \"140d5eb5-547f-4490-aa12-f34472b8a472\") " pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9"
Jul 03 14:10:09.941655 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:09.941518    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/140d5eb5-547f-4490-aa12-f34472b8a472-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b8865b898-vrxv9\" (UID: \"140d5eb5-547f-4490-aa12-f34472b8a472\") " pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9"
Jul 03 14:10:09.941655 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:09.941596    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/140d5eb5-547f-4490-aa12-f34472b8a472-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b8865b898-vrxv9\" (UID: \"140d5eb5-547f-4490-aa12-f34472b8a472\") " pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9"
Jul 03 14:10:09.941789 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:09.941652    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/140d5eb5-547f-4490-aa12-f34472b8a472-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b8865b898-vrxv9\" (UID: \"140d5eb5-547f-4490-aa12-f34472b8a472\") " pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9"
Jul 03 14:10:09.941789 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:09.941756    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/140d5eb5-547f-4490-aa12-f34472b8a472-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b8865b898-vrxv9\" (UID: \"140d5eb5-547f-4490-aa12-f34472b8a472\") " pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9"
Jul 03 14:10:09.941789 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:09.941774    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/140d5eb5-547f-4490-aa12-f34472b8a472-secret-grpc-tls\") pod \"thanos-querier-5b8865b898-vrxv9\" (UID: \"140d5eb5-547f-4490-aa12-f34472b8a472\") " pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9"
Jul 03 14:10:09.941906 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:09.941798    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4wwf\" (UniqueName: \"kubernetes.io/projected/140d5eb5-547f-4490-aa12-f34472b8a472-kube-api-access-j4wwf\") pod \"thanos-querier-5b8865b898-vrxv9\" (UID: \"140d5eb5-547f-4490-aa12-f34472b8a472\") " pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9"
Jul 03 14:10:10.042708 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:10.042647    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/140d5eb5-547f-4490-aa12-f34472b8a472-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b8865b898-vrxv9\" (UID: \"140d5eb5-547f-4490-aa12-f34472b8a472\") " pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9"
Jul 03 14:10:10.042894 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:10.042875    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/140d5eb5-547f-4490-aa12-f34472b8a472-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b8865b898-vrxv9\" (UID: \"140d5eb5-547f-4490-aa12-f34472b8a472\") " pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9"
Jul 03 14:10:10.042964 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:10.042911    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/140d5eb5-547f-4490-aa12-f34472b8a472-secret-grpc-tls\") pod \"thanos-querier-5b8865b898-vrxv9\" (UID: \"140d5eb5-547f-4490-aa12-f34472b8a472\") " pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9"
Jul 03 14:10:10.042964 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:10.042947    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4wwf\" (UniqueName: \"kubernetes.io/projected/140d5eb5-547f-4490-aa12-f34472b8a472-kube-api-access-j4wwf\") pod \"thanos-querier-5b8865b898-vrxv9\" (UID: \"140d5eb5-547f-4490-aa12-f34472b8a472\") " pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9"
Jul 03 14:10:10.043100 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:10.043006    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/140d5eb5-547f-4490-aa12-f34472b8a472-metrics-client-ca\") pod \"thanos-querier-5b8865b898-vrxv9\" (UID: \"140d5eb5-547f-4490-aa12-f34472b8a472\") " pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9"
Jul 03 14:10:10.043100 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:10.043041    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/140d5eb5-547f-4490-aa12-f34472b8a472-secret-thanos-querier-tls\") pod \"thanos-querier-5b8865b898-vrxv9\" (UID: \"140d5eb5-547f-4490-aa12-f34472b8a472\") " pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9"
Jul 03 14:10:10.043100 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:10.043094    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/140d5eb5-547f-4490-aa12-f34472b8a472-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b8865b898-vrxv9\" (UID: \"140d5eb5-547f-4490-aa12-f34472b8a472\") " pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9"
Jul 03 14:10:10.043216 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:10.043148    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/140d5eb5-547f-4490-aa12-f34472b8a472-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b8865b898-vrxv9\" (UID: \"140d5eb5-547f-4490-aa12-f34472b8a472\") " pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9"
Jul 03 14:10:10.043942 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:10.043911    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/140d5eb5-547f-4490-aa12-f34472b8a472-metrics-client-ca\") pod \"thanos-querier-5b8865b898-vrxv9\" (UID: \"140d5eb5-547f-4490-aa12-f34472b8a472\") " pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9"
Jul 03 14:10:10.045982 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:10.045918    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/140d5eb5-547f-4490-aa12-f34472b8a472-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b8865b898-vrxv9\" (UID: \"140d5eb5-547f-4490-aa12-f34472b8a472\") " pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9"
Jul 03 14:10:10.046847 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:10.046421    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/140d5eb5-547f-4490-aa12-f34472b8a472-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b8865b898-vrxv9\" (UID: \"140d5eb5-547f-4490-aa12-f34472b8a472\") " pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9"
Jul 03 14:10:10.046847 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:10.046575    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/140d5eb5-547f-4490-aa12-f34472b8a472-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b8865b898-vrxv9\" (UID: \"140d5eb5-547f-4490-aa12-f34472b8a472\") " pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9"
Jul 03 14:10:10.046847 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:10.046799    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/140d5eb5-547f-4490-aa12-f34472b8a472-secret-thanos-querier-tls\") pod \"thanos-querier-5b8865b898-vrxv9\" (UID: \"140d5eb5-547f-4490-aa12-f34472b8a472\") " pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9"
Jul 03 14:10:10.046847 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:10.046809    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/140d5eb5-547f-4490-aa12-f34472b8a472-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b8865b898-vrxv9\" (UID: \"140d5eb5-547f-4490-aa12-f34472b8a472\") " pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9"
Jul 03 14:10:10.047114 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:10.047004    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/140d5eb5-547f-4490-aa12-f34472b8a472-secret-grpc-tls\") pod \"thanos-querier-5b8865b898-vrxv9\" (UID: \"140d5eb5-547f-4490-aa12-f34472b8a472\") " pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9"
Jul 03 14:10:10.056073 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:10.056037    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4wwf\" (UniqueName: \"kubernetes.io/projected/140d5eb5-547f-4490-aa12-f34472b8a472-kube-api-access-j4wwf\") pod \"thanos-querier-5b8865b898-vrxv9\" (UID: \"140d5eb5-547f-4490-aa12-f34472b8a472\") " pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9"
Jul 03 14:10:10.176304 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:10.176268    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9"
Jul 03 14:10:11.271451 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.271415    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-649b8755c8-dfnkp"]
Jul 03 14:10:11.383025 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.382959    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-649b8755c8-dfnkp"]
Jul 03 14:10:11.383213 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.383077    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp"
Jul 03 14:10:11.389701 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.389669    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4nl01rbrgfode\""
Jul 03 14:10:11.390391 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.390368    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 03 14:10:11.390487 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.390400    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 03 14:10:11.390556 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.390511    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 03 14:10:11.390556 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.390526    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 03 14:10:11.390798 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.390532    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-h52nf\""
Jul 03 14:10:11.457165 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.457127    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3a6ec3b-1a67-4151-a95c-eb00d41401f4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-649b8755c8-dfnkp\" (UID: \"d3a6ec3b-1a67-4151-a95c-eb00d41401f4\") " pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp"
Jul 03 14:10:11.457394 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.457234    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d3a6ec3b-1a67-4151-a95c-eb00d41401f4-secret-metrics-server-client-certs\") pod \"metrics-server-649b8755c8-dfnkp\" (UID: \"d3a6ec3b-1a67-4151-a95c-eb00d41401f4\") " pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp"
Jul 03 14:10:11.457394 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.457267    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d3a6ec3b-1a67-4151-a95c-eb00d41401f4-secret-metrics-server-tls\") pod \"metrics-server-649b8755c8-dfnkp\" (UID: \"d3a6ec3b-1a67-4151-a95c-eb00d41401f4\") " pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp"
Jul 03 14:10:11.457394 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.457298    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d3a6ec3b-1a67-4151-a95c-eb00d41401f4-metrics-server-audit-profiles\") pod \"metrics-server-649b8755c8-dfnkp\" (UID: \"d3a6ec3b-1a67-4151-a95c-eb00d41401f4\") " pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp"
Jul 03 14:10:11.457394 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.457330    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhpcb\" (UniqueName: \"kubernetes.io/projected/d3a6ec3b-1a67-4151-a95c-eb00d41401f4-kube-api-access-qhpcb\") pod \"metrics-server-649b8755c8-dfnkp\" (UID: \"d3a6ec3b-1a67-4151-a95c-eb00d41401f4\") " pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp"
Jul 03 14:10:11.457563 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.457505    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d3a6ec3b-1a67-4151-a95c-eb00d41401f4-audit-log\") pod \"metrics-server-649b8755c8-dfnkp\" (UID: \"d3a6ec3b-1a67-4151-a95c-eb00d41401f4\") " pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp"
Jul 03 14:10:11.457563 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.457536    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d3a6ec3b-1a67-4151-a95c-eb00d41401f4-client-ca-bundle\") pod \"metrics-server-649b8755c8-dfnkp\" (UID: \"d3a6ec3b-1a67-4151-a95c-eb00d41401f4\") " pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp"
Jul 03 14:10:11.558521 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.558441    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3a6ec3b-1a67-4151-a95c-eb00d41401f4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-649b8755c8-dfnkp\" (UID: \"d3a6ec3b-1a67-4151-a95c-eb00d41401f4\") " pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp"
Jul 03 14:10:11.558521 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.558498    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d3a6ec3b-1a67-4151-a95c-eb00d41401f4-secret-metrics-server-client-certs\") pod \"metrics-server-649b8755c8-dfnkp\" (UID: \"d3a6ec3b-1a67-4151-a95c-eb00d41401f4\") " pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp"
Jul 03 14:10:11.558521 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.558524    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d3a6ec3b-1a67-4151-a95c-eb00d41401f4-secret-metrics-server-tls\") pod \"metrics-server-649b8755c8-dfnkp\" (UID: \"d3a6ec3b-1a67-4151-a95c-eb00d41401f4\") " pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp"
Jul 03 14:10:11.558886 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.558552    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d3a6ec3b-1a67-4151-a95c-eb00d41401f4-metrics-server-audit-profiles\") pod \"metrics-server-649b8755c8-dfnkp\" (UID: \"d3a6ec3b-1a67-4151-a95c-eb00d41401f4\") " pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp"
Jul 03 14:10:11.558886 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.558582    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhpcb\" (UniqueName: \"kubernetes.io/projected/d3a6ec3b-1a67-4151-a95c-eb00d41401f4-kube-api-access-qhpcb\") pod \"metrics-server-649b8755c8-dfnkp\" (UID: \"d3a6ec3b-1a67-4151-a95c-eb00d41401f4\") " pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp"
Jul 03 14:10:11.558886 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.558656    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d3a6ec3b-1a67-4151-a95c-eb00d41401f4-audit-log\") pod \"metrics-server-649b8755c8-dfnkp\" (UID: \"d3a6ec3b-1a67-4151-a95c-eb00d41401f4\") " pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp"
Jul 03 14:10:11.558886 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.558678    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d3a6ec3b-1a67-4151-a95c-eb00d41401f4-client-ca-bundle\") pod \"metrics-server-649b8755c8-dfnkp\" (UID: \"d3a6ec3b-1a67-4151-a95c-eb00d41401f4\") " pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp"
Jul 03 14:10:11.559397 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.559344    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d3a6ec3b-1a67-4151-a95c-eb00d41401f4-audit-log\") pod \"metrics-server-649b8755c8-dfnkp\" (UID: \"d3a6ec3b-1a67-4151-a95c-eb00d41401f4\") " pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp"
Jul 03 14:10:11.559397 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.559386    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3a6ec3b-1a67-4151-a95c-eb00d41401f4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-649b8755c8-dfnkp\" (UID: \"d3a6ec3b-1a67-4151-a95c-eb00d41401f4\") " pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp"
Jul 03 14:10:11.559676 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.559655    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d3a6ec3b-1a67-4151-a95c-eb00d41401f4-metrics-server-audit-profiles\") pod \"metrics-server-649b8755c8-dfnkp\" (UID: \"d3a6ec3b-1a67-4151-a95c-eb00d41401f4\") " pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp"
Jul 03 14:10:11.561419 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.561396    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d3a6ec3b-1a67-4151-a95c-eb00d41401f4-secret-metrics-server-tls\") pod \"metrics-server-649b8755c8-dfnkp\" (UID: \"d3a6ec3b-1a67-4151-a95c-eb00d41401f4\") " pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp"
Jul 03 14:10:11.561536 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.561396    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d3a6ec3b-1a67-4151-a95c-eb00d41401f4-client-ca-bundle\") pod \"metrics-server-649b8755c8-dfnkp\" (UID: \"d3a6ec3b-1a67-4151-a95c-eb00d41401f4\") " pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp"
Jul 03 14:10:11.561629 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.561607    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d3a6ec3b-1a67-4151-a95c-eb00d41401f4-secret-metrics-server-client-certs\") pod \"metrics-server-649b8755c8-dfnkp\" (UID: \"d3a6ec3b-1a67-4151-a95c-eb00d41401f4\") " pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp"
Jul 03 14:10:11.594567 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.594531    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-rv5xq"]
Jul 03 14:10:11.595805 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.595779    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhpcb\" (UniqueName: \"kubernetes.io/projected/d3a6ec3b-1a67-4151-a95c-eb00d41401f4-kube-api-access-qhpcb\") pod \"metrics-server-649b8755c8-dfnkp\" (UID: \"d3a6ec3b-1a67-4151-a95c-eb00d41401f4\") " pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp"
Jul 03 14:10:11.614166 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.614132    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-rv5xq"
Jul 03 14:10:11.616231 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.616204    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-rv5xq"]
Jul 03 14:10:11.621316 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.621288    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-twbxq\""
Jul 03 14:10:11.621696 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.621676    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 03 14:10:11.659348 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.659310    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3a2a55bf-9ea3-4b36-9a2d-81d9b8877039-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-rv5xq\" (UID: \"3a2a55bf-9ea3-4b36-9a2d-81d9b8877039\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-rv5xq"
Jul 03 14:10:11.659557 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.659473    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcdv7\" (UniqueName: \"kubernetes.io/projected/3a2a55bf-9ea3-4b36-9a2d-81d9b8877039-kube-api-access-gcdv7\") pod \"monitoring-plugin-d5dcff7cf-rv5xq\" (UID: \"3a2a55bf-9ea3-4b36-9a2d-81d9b8877039\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-rv5xq"
Jul 03 14:10:11.694346 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.694309    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp"
Jul 03 14:10:11.760034 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.760006    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcdv7\" (UniqueName: \"kubernetes.io/projected/3a2a55bf-9ea3-4b36-9a2d-81d9b8877039-kube-api-access-gcdv7\") pod \"monitoring-plugin-d5dcff7cf-rv5xq\" (UID: \"3a2a55bf-9ea3-4b36-9a2d-81d9b8877039\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-rv5xq"
Jul 03 14:10:11.760207 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.760065    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3a2a55bf-9ea3-4b36-9a2d-81d9b8877039-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-rv5xq\" (UID: \"3a2a55bf-9ea3-4b36-9a2d-81d9b8877039\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-rv5xq"
Jul 03 14:10:11.762431 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.762409    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3a2a55bf-9ea3-4b36-9a2d-81d9b8877039-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-rv5xq\" (UID: \"3a2a55bf-9ea3-4b36-9a2d-81d9b8877039\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-rv5xq"
Jul 03 14:10:11.771307 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.771280    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcdv7\" (UniqueName: \"kubernetes.io/projected/3a2a55bf-9ea3-4b36-9a2d-81d9b8877039-kube-api-access-gcdv7\") pod \"monitoring-plugin-d5dcff7cf-rv5xq\" (UID: \"3a2a55bf-9ea3-4b36-9a2d-81d9b8877039\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-rv5xq"
Jul 03 14:10:11.925707 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:11.925672    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-rv5xq"
Jul 03 14:10:12.009142 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:10:12.009089    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79e2d165_3d12_4c84_804b_4d7c857419ef.slice/crio-97ced8737546cb4751c1899dd18903aee7fcffe10c63b3aea9b3ebcc90133c23 WatchSource:0}: Error finding container 97ced8737546cb4751c1899dd18903aee7fcffe10c63b3aea9b3ebcc90133c23: Status 404 returned error can't find the container with id 97ced8737546cb4751c1899dd18903aee7fcffe10c63b3aea9b3ebcc90133c23
Jul 03 14:10:12.255904 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:12.255876    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-rv5xq"]
Jul 03 14:10:12.256895 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:10:12.256862    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a2a55bf_9ea3_4b36_9a2d_81d9b8877039.slice/crio-16a8487ce616c439ac0bd3675e65a6883ea3841b449ac598b00d784b663211d1 WatchSource:0}: Error finding container 16a8487ce616c439ac0bd3675e65a6883ea3841b449ac598b00d784b663211d1: Status 404 returned error can't find the container with id 16a8487ce616c439ac0bd3675e65a6883ea3841b449ac598b00d784b663211d1
Jul 03 14:10:12.267659 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:12.267629    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 03 14:10:12.304987 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:12.304938    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-649b8755c8-dfnkp"]
Jul 03 14:10:12.307648 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:10:12.307614    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3a6ec3b_1a67_4151_a95c_eb00d41401f4.slice/crio-ca44f0b46a9cc9c85b50e1c0813130cc31aab806600449d07a048a4bd19abef1 WatchSource:0}: Error finding container ca44f0b46a9cc9c85b50e1c0813130cc31aab806600449d07a048a4bd19abef1: Status 404 returned error can't find the container with id ca44f0b46a9cc9c85b50e1c0813130cc31aab806600449d07a048a4bd19abef1
Jul 03 14:10:12.512275 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:12.512091    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5b8865b898-vrxv9"]
Jul 03 14:10:12.516405 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:10:12.516373    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod140d5eb5_547f_4490_aa12_f34472b8a472.slice/crio-9451801fd0539189516c0c4d79f6f2a7699121e31b3416aa008afa20fd9f86af WatchSource:0}: Error finding container 9451801fd0539189516c0c4d79f6f2a7699121e31b3416aa008afa20fd9f86af: Status 404 returned error can't find the container with id 9451801fd0539189516c0c4d79f6f2a7699121e31b3416aa008afa20fd9f86af
Jul 03 14:10:12.535048 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:10:12.535008    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf196fa9_9734_4b49_a5cd_43c50cf3e54e.slice/crio-93b79a6e1c8b411b3dab2d5d10a1e1cd85fd773f3e088eeab2184b7becc21a17 WatchSource:0}: Error finding container 93b79a6e1c8b411b3dab2d5d10a1e1cd85fd773f3e088eeab2184b7becc21a17: Status 404 returned error can't find the container with id 93b79a6e1c8b411b3dab2d5d10a1e1cd85fd773f3e088eeab2184b7becc21a17
Jul 03 14:10:12.536756 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:12.536707    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-s7j5g" event={"ID":"ee8e766e-8e65-4d2d-abc5-7b7a0dc21b33","Type":"ContainerStarted","Data":"0c54731bb4c244f12cd4464f0029cbc2958db757227146f1cd533de0e596c295"}
Jul 03 14:10:12.537263 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:12.537197    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-s7j5g"
Jul 03 14:10:12.538825 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:12.538784    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-d77cl"]
Jul 03 14:10:12.539359 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:12.539310    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-rv5xq" event={"ID":"3a2a55bf-9ea3-4b36-9a2d-81d9b8877039","Type":"ContainerStarted","Data":"16a8487ce616c439ac0bd3675e65a6883ea3841b449ac598b00d784b663211d1"}
Jul 03 14:10:12.540667 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:12.540620    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9" event={"ID":"140d5eb5-547f-4490-aa12-f34472b8a472","Type":"ContainerStarted","Data":"9451801fd0539189516c0c4d79f6f2a7699121e31b3416aa008afa20fd9f86af"}
Jul 03 14:10:12.542119 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:12.542094    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp" event={"ID":"d3a6ec3b-1a67-4151-a95c-eb00d41401f4","Type":"ContainerStarted","Data":"ca44f0b46a9cc9c85b50e1c0813130cc31aab806600449d07a048a4bd19abef1"}
Jul 03 14:10:12.543753 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:12.543688    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f129596-3617-4931-9dbe-5c3a4469e16a","Type":"ContainerStarted","Data":"a24a8acc12498a5cde233d23a9df73d22e66faf51058728a970facbfeb47d9c2"}
Jul 03 14:10:12.546473 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:12.546443    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v6t5n" event={"ID":"f499c073-70d1-4178-8c52-1e64babb8a37","Type":"ContainerStarted","Data":"7b6a48f2143a78d668cf7386445002ab3af5b327e23960eaa548dd51065f1a4a"}
Jul 03 14:10:12.548647 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:12.548475    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b94d7cd68-wnlbl" event={"ID":"31c13a3d-642e-4dae-b3f4-0d8596014470","Type":"ContainerStarted","Data":"c25f7e0c1f5141e8fc7f0c280b3c7f79b9b59deac635288b29a400039894c011"}
Jul 03 14:10:12.549128 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:12.549110    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-s7j5g"
Jul 03 14:10:12.550611 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:12.550541    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b59zs" event={"ID":"79e2d165-3d12-4c84-804b-4d7c857419ef","Type":"ContainerStarted","Data":"97ced8737546cb4751c1899dd18903aee7fcffe10c63b3aea9b3ebcc90133c23"}
Jul 03 14:10:12.584140 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:12.584079    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-s7j5g" podStartSLOduration=2.081632506 podStartE2EDuration="21.584057539s" totalImagesPullingTime="19.502425033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:09:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:10:12.58295963 +0000 UTC m=+126.033093071" watchObservedRunningTime="2026-07-03 14:10:12.584057539 +0000 UTC m=+126.034190982"
Jul 03 14:10:12.702573 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:12.702507    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-v6t5n" podStartSLOduration=33.98459561 podStartE2EDuration="51.702481742s" totalImagesPullingTime="17.717886132s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:09:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:10:12.639474164 +0000 UTC m=+126.089607605" watchObservedRunningTime="2026-07-03 14:10:12.702481742 +0000 UTC m=+126.152615183"
Jul 03 14:10:12.743105 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:12.743036    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b94d7cd68-wnlbl" podStartSLOduration=5.087618357 podStartE2EDuration="16.743010894s" totalImagesPullingTime="11.655392537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:09:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:10:12.705896426 +0000 UTC m=+126.156029893" watchObservedRunningTime="2026-07-03 14:10:12.743010894 +0000 UTC m=+126.193144330"
Jul 03 14:10:13.564027 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.563985    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-d77cl" event={"ID":"af196fa9-9734-4b49-a5cd-43c50cf3e54e","Type":"ContainerStarted","Data":"0487e4780ece12fdd3b9dbef7ecaff84ce1ad630c8d24740a27cf0d9b7e41e7f"}
Jul 03 14:10:13.565852 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.564065    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-d77cl" event={"ID":"af196fa9-9734-4b49-a5cd-43c50cf3e54e","Type":"ContainerStarted","Data":"69f45f151b648b571ab97ce41cfc523c1c8ea40d9fb96108fe5a89feb25ef52f"}
Jul 03 14:10:13.565852 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.564094    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-d77cl" event={"ID":"af196fa9-9734-4b49-a5cd-43c50cf3e54e","Type":"ContainerStarted","Data":"93b79a6e1c8b411b3dab2d5d10a1e1cd85fd773f3e088eeab2184b7becc21a17"}
Jul 03 14:10:13.569506 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.567488    2563 generic.go:356] "Generic (PLEG): container finished" podID="79e2d165-3d12-4c84-804b-4d7c857419ef" containerID="0b414abad8ec949e9ebcc50bfca013e2e63adb272c1202a5c6e869d3e127e7be" exitCode=0
Jul 03 14:10:13.569506 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.568669    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b59zs" event={"ID":"79e2d165-3d12-4c84-804b-4d7c857419ef","Type":"ContainerDied","Data":"0b414abad8ec949e9ebcc50bfca013e2e63adb272c1202a5c6e869d3e127e7be"}
Jul 03 14:10:13.594993 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.592760    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 03 14:10:13.616052 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.615098    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.678547 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.677865    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 03 14:10:13.678547 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.678012    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 03 14:10:13.678547 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.678149    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 03 14:10:13.678547 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.678224    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 03 14:10:13.678547 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.678363    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 03 14:10:13.678547 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.678387    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 03 14:10:13.678547 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.678539    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 03 14:10:13.679220 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.678865    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-nz67c\""
Jul 03 14:10:13.679220 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.679134    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 03 14:10:13.679803 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.679330    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 03 14:10:13.679803 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.679440    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 03 14:10:13.679803 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.679688    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 03 14:10:13.679803 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.679711    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 03 14:10:13.687657 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.687376    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.687657 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.687424    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.687657 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.687455    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.687657 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.687484    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-config\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.687657 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.687560    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.687657 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.687613    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-config-out\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.691013 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.687639    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.691013 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.688126    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.691013 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.688187    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffjs2\" (UniqueName: \"kubernetes.io/projected/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-kube-api-access-ffjs2\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.691013 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.688226    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.691013 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.688254    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.691013 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.688308    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.691013 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.688347    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.691013 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.688371    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-web-config\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.691013 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.688417    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.691013 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.688452    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.691013 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.688517    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.691013 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.688547    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.691013 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.688578    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.691013 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.688718    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.691013 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.690244    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-53sdi07rac24a\""
Jul 03 14:10:13.695293 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.695134    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 03 14:10:13.721017 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.718773    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 03 14:10:13.750401 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.750098    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 03 14:10:13.791788 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.789529    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-config\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.791788 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.789583    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.791788 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.789624    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-config-out\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.791788 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.789652    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.791788 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.789746    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.791788 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.789775    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffjs2\" (UniqueName: \"kubernetes.io/projected/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-kube-api-access-ffjs2\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.791788 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.789800    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.791788 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.789826    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.791788 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.789864    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.791788 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.789899    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.791788 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.789923    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-web-config\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.791788 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.789951    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.791788 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.789994    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.791788 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.790026    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.791788 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.790052    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.791788 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.790082    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.791788 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.790151    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.792810 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.790201    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.792810 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.790232    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.792810 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.790258    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.792810 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.791043    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.833757 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.812218    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.833757 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.812873    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.833757 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.816927    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.833757 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.818682    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.833757 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.818893    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.833757 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.832602    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.833757 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.833180    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.833757 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.833700    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.834784 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.834740    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-config\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.836996 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.836620    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.838228 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.837772    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.838847 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.838794    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.845678 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.842588    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.845678 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.842773    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-config-out\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.854809 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.854615    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.861289 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.861176    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-web-config\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.862915 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.862622    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.863623 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.863424    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.894302 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.894191    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffjs2\" (UniqueName: \"kubernetes.io/projected/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-kube-api-access-ffjs2\") pod \"prometheus-k8s-0\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:13.933344 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:13.933062    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:14.734405 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:14.733986    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"]
Jul 03 14:10:16.826436 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:16.826391    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28b96845-6419-4df3-bff3-777d753989ce-metrics-certs\") pod \"network-metrics-daemon-9dzdm\" (UID: \"28b96845-6419-4df3-bff3-777d753989ce\") " pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:10:16.829503 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:16.829441    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28b96845-6419-4df3-bff3-777d753989ce-metrics-certs\") pod \"network-metrics-daemon-9dzdm\" (UID: \"28b96845-6419-4df3-bff3-777d753989ce\") " pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:10:16.880545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:16.880404    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-472cc\""
Jul 03 14:10:16.887798 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:16.887768    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dzdm"
Jul 03 14:10:16.937938 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:16.937878    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b94d7cd68-wnlbl"
Jul 03 14:10:18.005933 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:18.004957    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 03 14:10:18.010113 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:10:18.010071    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f49b424_126d_4ca8_8666_bf5ed7d46ae2.slice/crio-d807ae2f778ec52c236413fc6844d8d824f417cc8062961a0032a012188daf49 WatchSource:0}: Error finding container d807ae2f778ec52c236413fc6844d8d824f417cc8062961a0032a012188daf49: Status 404 returned error can't find the container with id d807ae2f778ec52c236413fc6844d8d824f417cc8062961a0032a012188daf49
Jul 03 14:10:18.015860 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:18.015801    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9dzdm"]
Jul 03 14:10:18.020696 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:10:18.020665    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28b96845_6419_4df3_bff3_777d753989ce.slice/crio-658d8188da6755c4aea907d4207a5f0884b08a1e01c73c6c2c482e0db1c7cceb WatchSource:0}: Error finding container 658d8188da6755c4aea907d4207a5f0884b08a1e01c73c6c2c482e0db1c7cceb: Status 404 returned error can't find the container with id 658d8188da6755c4aea907d4207a5f0884b08a1e01c73c6c2c482e0db1c7cceb
Jul 03 14:10:18.594466 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:18.594373    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-d77cl" event={"ID":"af196fa9-9734-4b49-a5cd-43c50cf3e54e","Type":"ContainerStarted","Data":"62c588f8183c73c95bb64b387ab1e912f55962236f52e29a60ad62fb3ddd453b"}
Jul 03 14:10:18.598237 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:18.598183    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b59zs" event={"ID":"79e2d165-3d12-4c84-804b-4d7c857419ef","Type":"ContainerStarted","Data":"59e3df4edeb483e831bd115a4c325b7b72ee21ec6bd65966093a6cf0cccaa4fb"}
Jul 03 14:10:18.598407 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:18.598260    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b59zs" event={"ID":"79e2d165-3d12-4c84-804b-4d7c857419ef","Type":"ContainerStarted","Data":"eaafea4085a317d41e0bde2a09c2fb54a54ac3c6fc09aed9d8176e9b8ecd8b88"}
Jul 03 14:10:18.600470 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:18.600436    2563 generic.go:356] "Generic (PLEG): container finished" podID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerID="bd1840408b657d161019a191865790af9c9e8aca0160b16a4b130751c958080d" exitCode=0
Jul 03 14:10:18.600598 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:18.600527    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f49b424-126d-4ca8-8666-bf5ed7d46ae2","Type":"ContainerDied","Data":"bd1840408b657d161019a191865790af9c9e8aca0160b16a4b130751c958080d"}
Jul 03 14:10:18.600669 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:18.600626    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f49b424-126d-4ca8-8666-bf5ed7d46ae2","Type":"ContainerStarted","Data":"d807ae2f778ec52c236413fc6844d8d824f417cc8062961a0032a012188daf49"}
Jul 03 14:10:18.602757 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:18.602735    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-rv5xq" event={"ID":"3a2a55bf-9ea3-4b36-9a2d-81d9b8877039","Type":"ContainerStarted","Data":"02eea0ca6f35c2266f031372728c55f9567527e0f9d0f685102d9482782e3a58"}
Jul 03 14:10:18.603266 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:18.603243    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-rv5xq"
Jul 03 14:10:18.607273 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:18.607249    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9" event={"ID":"140d5eb5-547f-4490-aa12-f34472b8a472","Type":"ContainerStarted","Data":"2cde4619bdb55e66bdab53830bb49ece8eaf04f61940405eeffa24aa3e6e27be"}
Jul 03 14:10:18.607370 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:18.607313    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9" event={"ID":"140d5eb5-547f-4490-aa12-f34472b8a472","Type":"ContainerStarted","Data":"8a82aabdbc3d9795da36877c86e15e5d1cbcb90d8db6ce7766c6202b909fc4a2"}
Jul 03 14:10:18.607370 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:18.607352    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9" event={"ID":"140d5eb5-547f-4490-aa12-f34472b8a472","Type":"ContainerStarted","Data":"99831566c45703e13dc4da5a877ce9e0445f9a002a8447f9e3675fc3319c89f5"}
Jul 03 14:10:18.609765 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:18.609744    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-rv5xq"
Jul 03 14:10:18.610385 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:18.610360    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp" event={"ID":"d3a6ec3b-1a67-4151-a95c-eb00d41401f4","Type":"ContainerStarted","Data":"f2a246e1f53312a3974ec6d96b7cdf85faaa45776db46ac413ac5e0dee15518c"}
Jul 03 14:10:18.612175 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:18.612149    2563 generic.go:356] "Generic (PLEG): container finished" podID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerID="f520a40fc174d025c46caae0df8cc8bca145c4a51c491ab18b4b57dbf6d19ee8" exitCode=0
Jul 03 14:10:18.612272 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:18.612228    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f129596-3617-4931-9dbe-5c3a4469e16a","Type":"ContainerDied","Data":"f520a40fc174d025c46caae0df8cc8bca145c4a51c491ab18b4b57dbf6d19ee8"}
Jul 03 14:10:18.614377 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:18.614313    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9dzdm" event={"ID":"28b96845-6419-4df3-bff3-777d753989ce","Type":"ContainerStarted","Data":"658d8188da6755c4aea907d4207a5f0884b08a1e01c73c6c2c482e0db1c7cceb"}
Jul 03 14:10:18.662328 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:18.661174    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-d77cl" podStartSLOduration=7.981855564 podStartE2EDuration="12.66115383s" totalImagesPullingTime="4.679298266s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:10:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:10:18.619375456 +0000 UTC m=+132.069508914" watchObservedRunningTime="2026-07-03 14:10:18.66115383 +0000 UTC m=+132.111287273"
Jul 03 14:10:18.683461 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:18.683330    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-rv5xq" podStartSLOduration=2.17742902 podStartE2EDuration="7.683309574s" totalImagesPullingTime="5.505880554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:10:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:10:18.682245964 +0000 UTC m=+132.132379405" watchObservedRunningTime="2026-07-03 14:10:18.683309574 +0000 UTC m=+132.133443014"
Jul 03 14:10:18.707688 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:18.707347    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp" podStartSLOduration=2.256082183 podStartE2EDuration="7.707325428s" totalImagesPullingTime="5.451243245s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:10:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:10:18.70467933 +0000 UTC m=+132.154812772" watchObservedRunningTime="2026-07-03 14:10:18.707325428 +0000 UTC m=+132.157458874"
Jul 03 14:10:18.787287 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:18.787228    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-b59zs" podStartSLOduration=11.590562231 podStartE2EDuration="12.787205508s" totalImagesPullingTime="1.077454472s" totalInitContainerRuntime="119.188805ms" isStatefulPod=true podCreationTimestamp="2026-07-03 14:10:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:10:18.78552017 +0000 UTC m=+132.235653645" watchObservedRunningTime="2026-07-03 14:10:18.787205508 +0000 UTC m=+132.237338947"
Jul 03 14:10:20.632682 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:20.632626    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9" event={"ID":"140d5eb5-547f-4490-aa12-f34472b8a472","Type":"ContainerStarted","Data":"97f0f3b44a43eadf167ae3775537cc288495244761d48d20dd9cf4901466eefb"}
Jul 03 14:10:20.633137 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:20.632719    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9" event={"ID":"140d5eb5-547f-4490-aa12-f34472b8a472","Type":"ContainerStarted","Data":"d1230c15b5c62526df3862256d73d040ac2fcefdd4ca5ecdfdcfb8e144d5ba38"}
Jul 03 14:10:20.633137 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:20.632765    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9" event={"ID":"140d5eb5-547f-4490-aa12-f34472b8a472","Type":"ContainerStarted","Data":"c61a4d039c215845fcb10911d6439d146e1bd7c55b1eef244beb5af95bcaf5e9"}
Jul 03 14:10:20.634915 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:20.634705    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9dzdm" event={"ID":"28b96845-6419-4df3-bff3-777d753989ce","Type":"ContainerStarted","Data":"fbf41e4cfe5f932bf7445330482181d944a96ade1e213583d887da9fccc5220c"}
Jul 03 14:10:20.634915 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:20.634772    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9dzdm" event={"ID":"28b96845-6419-4df3-bff3-777d753989ce","Type":"ContainerStarted","Data":"28fdaad71865a4c813a0c6afffbeee34c5c700265ef268768a36842cc7a4d8f8"}
Jul 03 14:10:20.656361 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:20.656267    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9dzdm" podStartSLOduration=131.842274358 podStartE2EDuration="2m13.656245366s" totalImagesPullingTime="1.813971008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:08:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:10:20.654321534 +0000 UTC m=+134.104454976" watchObservedRunningTime="2026-07-03 14:10:20.656245366 +0000 UTC m=+134.106378807"
Jul 03 14:10:21.647323 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:21.647285    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f129596-3617-4931-9dbe-5c3a4469e16a","Type":"ContainerStarted","Data":"65456891a2f15f6e131fffef4343346a32464223183cabb15527671529641260"}
Jul 03 14:10:21.647801 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:21.647587    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9"
Jul 03 14:10:21.687680 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:21.687570    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9" podStartSLOduration=5.565252989 podStartE2EDuration="12.68755214s" totalImagesPullingTime="7.122299151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:10:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:10:21.6869913 +0000 UTC m=+135.137124738" watchObservedRunningTime="2026-07-03 14:10:21.68755214 +0000 UTC m=+135.137685614"
Jul 03 14:10:22.655407 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:22.655367    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f129596-3617-4931-9dbe-5c3a4469e16a","Type":"ContainerStarted","Data":"efe709216492b2b541f8df3d5f2a2be8b950ec72b23a1b9e40ef3909e38f02fa"}
Jul 03 14:10:23.664100 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:23.664066    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f49b424-126d-4ca8-8666-bf5ed7d46ae2","Type":"ContainerStarted","Data":"d8b6d3c5529436789bcf1ea432259e0399127b780ce97f53ff3f32137aa71530"}
Jul 03 14:10:23.664506 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:23.664145    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f49b424-126d-4ca8-8666-bf5ed7d46ae2","Type":"ContainerStarted","Data":"2ecedefaf5adf1d2e20f57db254a8fcbfd85f4e81b1fa261f34ba62ac8447140"}
Jul 03 14:10:23.664506 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:23.664174    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f49b424-126d-4ca8-8666-bf5ed7d46ae2","Type":"ContainerStarted","Data":"108ca6c14620a636508c2fd2dc795f58c3ca5637c08e6af9bc7e880a207833ca"}
Jul 03 14:10:23.667741 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:23.667714    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f129596-3617-4931-9dbe-5c3a4469e16a","Type":"ContainerStarted","Data":"a3971d489da0a709c192088118f58bbc26528a9c11a12fd9a3ff82282ca78e40"}
Jul 03 14:10:23.667861 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:23.667788    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f129596-3617-4931-9dbe-5c3a4469e16a","Type":"ContainerStarted","Data":"8caf2be42e9daff9d33141f0c29241ea692cf574a156780b213bb4b276cdfbc0"}
Jul 03 14:10:23.667861 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:23.667830    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f129596-3617-4931-9dbe-5c3a4469e16a","Type":"ContainerStarted","Data":"179df6a20b7cae3782550e6903e14aa53a5aac5a4221443c21f7f981d7ce7323"}
Jul 03 14:10:23.667999 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:23.667869    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f129596-3617-4931-9dbe-5c3a4469e16a","Type":"ContainerStarted","Data":"7fb889cb8ba04c5a54adb985f62230a53dc58d21669217112e7172324f3939c3"}
Jul 03 14:10:23.718165 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:23.718105    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=8.376929712 podStartE2EDuration="16.718082364s" totalImagesPullingTime="8.296187391s" totalInitContainerRuntime="44.965261ms" isStatefulPod=false podCreationTimestamp="2026-07-03 14:10:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:10:23.712778274 +0000 UTC m=+137.162911743" watchObservedRunningTime="2026-07-03 14:10:23.718082364 +0000 UTC m=+137.168215805"
Jul 03 14:10:24.675281 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:24.675239    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f49b424-126d-4ca8-8666-bf5ed7d46ae2","Type":"ContainerStarted","Data":"ad47a462a441c5f6ea4fe067960b8b0c340d91fb4c692045b25035b5080525fe"}
Jul 03 14:10:24.675944 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:24.675339    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f49b424-126d-4ca8-8666-bf5ed7d46ae2","Type":"ContainerStarted","Data":"72c55fd67f0614fb1567c4cbbcc2c23759e932804c4c27c1d42c910295bdec7f"}
Jul 03 14:10:24.675944 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:24.675366    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f49b424-126d-4ca8-8666-bf5ed7d46ae2","Type":"ContainerStarted","Data":"30f0f0187a36b3c74dff3bd4e0cd140eb60b3c2dfb42f91cf2b0a1dae1c0acc4"}
Jul 03 14:10:24.770546 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:24.770496    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=7.1201613009999996 podStartE2EDuration="11.770478568s" totalImagesPullingTime="4.62480056s" totalInitContainerRuntime="25.516707ms" isStatefulPod=false podCreationTimestamp="2026-07-03 14:10:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:10:24.768361432 +0000 UTC m=+138.218494897" watchObservedRunningTime="2026-07-03 14:10:24.770478568 +0000 UTC m=+138.220612008"
Jul 03 14:10:27.661511 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:27.661480    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5b8865b898-vrxv9"
Jul 03 14:10:28.934137 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:28.934102    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:10:31.695232 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:31.695194    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp"
Jul 03 14:10:31.695635 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:31.695296    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp"
Jul 03 14:10:37.581620 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:37.581569    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7b94d7cd68-wnlbl" podUID="31c13a3d-642e-4dae-b3f4-0d8596014470" containerName="console" containerID="cri-o://c25f7e0c1f5141e8fc7f0c280b3c7f79b9b59deac635288b29a400039894c011" gracePeriod=15
Jul 03 14:10:37.734955 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:37.734924    2563 generic.go:356] "Generic (PLEG): container finished" podID="31c13a3d-642e-4dae-b3f4-0d8596014470" containerID="c25f7e0c1f5141e8fc7f0c280b3c7f79b9b59deac635288b29a400039894c011" exitCode=0
Jul 03 14:10:37.735148 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:37.734991    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b94d7cd68-wnlbl" event={"ID":"31c13a3d-642e-4dae-b3f4-0d8596014470","Type":"ContainerDied","Data":"c25f7e0c1f5141e8fc7f0c280b3c7f79b9b59deac635288b29a400039894c011"}
Jul 03 14:10:37.858390 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:37.858358    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b94d7cd68-wnlbl"
Jul 03 14:10:37.930904 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:37.930872    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31c13a3d-642e-4dae-b3f4-0d8596014470-service-ca\") pod \"31c13a3d-642e-4dae-b3f4-0d8596014470\" (UID: \"31c13a3d-642e-4dae-b3f4-0d8596014470\") "
Jul 03 14:10:37.931108 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:37.930999    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31c13a3d-642e-4dae-b3f4-0d8596014470-console-config\") pod \"31c13a3d-642e-4dae-b3f4-0d8596014470\" (UID: \"31c13a3d-642e-4dae-b3f4-0d8596014470\") "
Jul 03 14:10:37.931108 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:37.931039    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31c13a3d-642e-4dae-b3f4-0d8596014470-console-oauth-config\") pod \"31c13a3d-642e-4dae-b3f4-0d8596014470\" (UID: \"31c13a3d-642e-4dae-b3f4-0d8596014470\") "
Jul 03 14:10:37.931108 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:37.931086    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31c13a3d-642e-4dae-b3f4-0d8596014470-console-serving-cert\") pod \"31c13a3d-642e-4dae-b3f4-0d8596014470\" (UID: \"31c13a3d-642e-4dae-b3f4-0d8596014470\") "
Jul 03 14:10:37.931238 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:37.931136    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppmfj\" (UniqueName: \"kubernetes.io/projected/31c13a3d-642e-4dae-b3f4-0d8596014470-kube-api-access-ppmfj\") pod \"31c13a3d-642e-4dae-b3f4-0d8596014470\" (UID: \"31c13a3d-642e-4dae-b3f4-0d8596014470\") "
Jul 03 14:10:37.931280 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:37.931227    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31c13a3d-642e-4dae-b3f4-0d8596014470-oauth-serving-cert\") pod \"31c13a3d-642e-4dae-b3f4-0d8596014470\" (UID: \"31c13a3d-642e-4dae-b3f4-0d8596014470\") "
Jul 03 14:10:37.931540 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:37.931411    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31c13a3d-642e-4dae-b3f4-0d8596014470-service-ca" (OuterVolumeSpecName: "service-ca") pod "31c13a3d-642e-4dae-b3f4-0d8596014470" (UID: "31c13a3d-642e-4dae-b3f4-0d8596014470"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:10:37.931635 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:37.931539    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31c13a3d-642e-4dae-b3f4-0d8596014470-console-config" (OuterVolumeSpecName: "console-config") pod "31c13a3d-642e-4dae-b3f4-0d8596014470" (UID: "31c13a3d-642e-4dae-b3f4-0d8596014470"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:10:37.931635 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:37.931560    2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31c13a3d-642e-4dae-b3f4-0d8596014470-service-ca\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:10:37.931635 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:37.931594    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31c13a3d-642e-4dae-b3f4-0d8596014470-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "31c13a3d-642e-4dae-b3f4-0d8596014470" (UID: "31c13a3d-642e-4dae-b3f4-0d8596014470"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:10:37.933381 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:37.933356    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31c13a3d-642e-4dae-b3f4-0d8596014470-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "31c13a3d-642e-4dae-b3f4-0d8596014470" (UID: "31c13a3d-642e-4dae-b3f4-0d8596014470"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:10:37.933470 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:37.933374    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31c13a3d-642e-4dae-b3f4-0d8596014470-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "31c13a3d-642e-4dae-b3f4-0d8596014470" (UID: "31c13a3d-642e-4dae-b3f4-0d8596014470"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:10:37.933470 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:37.933377    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31c13a3d-642e-4dae-b3f4-0d8596014470-kube-api-access-ppmfj" (OuterVolumeSpecName: "kube-api-access-ppmfj") pod "31c13a3d-642e-4dae-b3f4-0d8596014470" (UID: "31c13a3d-642e-4dae-b3f4-0d8596014470"). InnerVolumeSpecName "kube-api-access-ppmfj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:10:38.032246 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:38.032213    2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31c13a3d-642e-4dae-b3f4-0d8596014470-console-config\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:10:38.032246 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:38.032241    2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31c13a3d-642e-4dae-b3f4-0d8596014470-console-oauth-config\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:10:38.032246 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:38.032250    2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31c13a3d-642e-4dae-b3f4-0d8596014470-console-serving-cert\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:10:38.032246 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:38.032259    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppmfj\" (UniqueName: \"kubernetes.io/projected/31c13a3d-642e-4dae-b3f4-0d8596014470-kube-api-access-ppmfj\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:10:38.032537 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:38.032267    2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31c13a3d-642e-4dae-b3f4-0d8596014470-oauth-serving-cert\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:10:38.740527 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:38.740496    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b94d7cd68-wnlbl"
Jul 03 14:10:38.741075 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:38.740531    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b94d7cd68-wnlbl" event={"ID":"31c13a3d-642e-4dae-b3f4-0d8596014470","Type":"ContainerDied","Data":"c217c1f2bbcc6fdcc403ff69a4926c7ae598eef0cf8978b9b83916ae30556e0d"}
Jul 03 14:10:38.741075 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:38.740583    2563 scope.go:117] "RemoveContainer" containerID="c25f7e0c1f5141e8fc7f0c280b3c7f79b9b59deac635288b29a400039894c011"
Jul 03 14:10:38.775323 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:38.775289    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b94d7cd68-wnlbl"]
Jul 03 14:10:38.789918 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:38.789881    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b94d7cd68-wnlbl"]
Jul 03 14:10:39.058080 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:39.057999    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31c13a3d-642e-4dae-b3f4-0d8596014470" path="/var/lib/kubelet/pods/31c13a3d-642e-4dae-b3f4-0d8596014470/volumes"
Jul 03 14:10:39.762095 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:39.762047    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw" podUID="cb3f129d-0917-4c5b-a369-ac9a0f9e63a2" containerName="registry" containerID="cri-o://227304c1a93973836a67ca2043540a5d609ad798269cf8eb9ec64edd292d1f42" gracePeriod=30
Jul 03 14:10:40.039292 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.039216    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:10:40.150504 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.150472    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-bound-sa-token\") pod \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") "
Jul 03 14:10:40.150504 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.150514    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-certificates\") pod \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") "
Jul 03 14:10:40.150762 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.150574    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-tls\") pod \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") "
Jul 03 14:10:40.150762 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.150596    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-ca-trust-extracted\") pod \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") "
Jul 03 14:10:40.150762 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.150621    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-trusted-ca\") pod \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") "
Jul 03 14:10:40.150762 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.150639    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-image-registry-private-configuration\") pod \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") "
Jul 03 14:10:40.150762 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.150663    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-installation-pull-secrets\") pod \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") "
Jul 03 14:10:40.150762 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.150699    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvrgl\" (UniqueName: \"kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-kube-api-access-wvrgl\") pod \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\" (UID: \"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2\") "
Jul 03 14:10:40.151183 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.151152    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "cb3f129d-0917-4c5b-a369-ac9a0f9e63a2" (UID: "cb3f129d-0917-4c5b-a369-ac9a0f9e63a2"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:10:40.151310 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.151287    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "cb3f129d-0917-4c5b-a369-ac9a0f9e63a2" (UID: "cb3f129d-0917-4c5b-a369-ac9a0f9e63a2"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:10:40.153215 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.153187    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "cb3f129d-0917-4c5b-a369-ac9a0f9e63a2" (UID: "cb3f129d-0917-4c5b-a369-ac9a0f9e63a2"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:10:40.153362 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.153332    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "cb3f129d-0917-4c5b-a369-ac9a0f9e63a2" (UID: "cb3f129d-0917-4c5b-a369-ac9a0f9e63a2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:10:40.153497 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.153464    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-kube-api-access-wvrgl" (OuterVolumeSpecName: "kube-api-access-wvrgl") pod "cb3f129d-0917-4c5b-a369-ac9a0f9e63a2" (UID: "cb3f129d-0917-4c5b-a369-ac9a0f9e63a2"). InnerVolumeSpecName "kube-api-access-wvrgl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:10:40.153563 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.153535    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "cb3f129d-0917-4c5b-a369-ac9a0f9e63a2" (UID: "cb3f129d-0917-4c5b-a369-ac9a0f9e63a2"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:10:40.153630 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.153602    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "cb3f129d-0917-4c5b-a369-ac9a0f9e63a2" (UID: "cb3f129d-0917-4c5b-a369-ac9a0f9e63a2"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:10:40.159599 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.159575    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "cb3f129d-0917-4c5b-a369-ac9a0f9e63a2" (UID: "cb3f129d-0917-4c5b-a369-ac9a0f9e63a2"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:10:40.251690 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.251650    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvrgl\" (UniqueName: \"kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-kube-api-access-wvrgl\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:10:40.251690 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.251681    2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-bound-sa-token\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:10:40.251690 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.251691    2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-certificates\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:10:40.251690 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.251703    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-registry-tls\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:10:40.251945 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.251711    2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-ca-trust-extracted\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:10:40.251945 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.251720    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-trusted-ca\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:10:40.251945 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.251729    2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-image-registry-private-configuration\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:10:40.251945 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.251737    2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2-installation-pull-secrets\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:10:40.750654 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.750624    2563 generic.go:356] "Generic (PLEG): container finished" podID="cb3f129d-0917-4c5b-a369-ac9a0f9e63a2" containerID="227304c1a93973836a67ca2043540a5d609ad798269cf8eb9ec64edd292d1f42" exitCode=0
Jul 03 14:10:40.750845 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.750667    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw" event={"ID":"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2","Type":"ContainerDied","Data":"227304c1a93973836a67ca2043540a5d609ad798269cf8eb9ec64edd292d1f42"}
Jul 03 14:10:40.750845 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.750670    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"
Jul 03 14:10:40.750845 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.750707    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cbb57ffb8-2ccdw" event={"ID":"cb3f129d-0917-4c5b-a369-ac9a0f9e63a2","Type":"ContainerDied","Data":"8f6999e21114d524001a09790ff8f8a565465d2c9eaa381f5514bbc94c082359"}
Jul 03 14:10:40.750845 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.750727    2563 scope.go:117] "RemoveContainer" containerID="227304c1a93973836a67ca2043540a5d609ad798269cf8eb9ec64edd292d1f42"
Jul 03 14:10:40.760163 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.760148    2563 scope.go:117] "RemoveContainer" containerID="227304c1a93973836a67ca2043540a5d609ad798269cf8eb9ec64edd292d1f42"
Jul 03 14:10:40.760396 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:10:40.760377    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"227304c1a93973836a67ca2043540a5d609ad798269cf8eb9ec64edd292d1f42\": container with ID starting with 227304c1a93973836a67ca2043540a5d609ad798269cf8eb9ec64edd292d1f42 not found: ID does not exist" containerID="227304c1a93973836a67ca2043540a5d609ad798269cf8eb9ec64edd292d1f42"
Jul 03 14:10:40.760441 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.760403    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"227304c1a93973836a67ca2043540a5d609ad798269cf8eb9ec64edd292d1f42"} err="failed to get container status \"227304c1a93973836a67ca2043540a5d609ad798269cf8eb9ec64edd292d1f42\": rpc error: code = NotFound desc = could not find container \"227304c1a93973836a67ca2043540a5d609ad798269cf8eb9ec64edd292d1f42\": container with ID starting with 227304c1a93973836a67ca2043540a5d609ad798269cf8eb9ec64edd292d1f42 not found: ID does not exist"
Jul 03 14:10:40.774894 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.774865    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"]
Jul 03 14:10:40.784515 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:40.784486    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6cbb57ffb8-2ccdw"]
Jul 03 14:10:41.057943 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:41.057859    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb3f129d-0917-4c5b-a369-ac9a0f9e63a2" path="/var/lib/kubelet/pods/cb3f129d-0917-4c5b-a369-ac9a0f9e63a2/volumes"
Jul 03 14:10:45.771977 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:45.771929    2563 generic.go:356] "Generic (PLEG): container finished" podID="a258d061-bd67-45af-b0a1-4440e92a1258" containerID="5e6c6d893591ebbb6d883f3f2837a98cc5b5d7f7dde8f35d1c079bad9b3d94a5" exitCode=0
Jul 03 14:10:45.772558 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:45.772007    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9kjx7" event={"ID":"a258d061-bd67-45af-b0a1-4440e92a1258","Type":"ContainerDied","Data":"5e6c6d893591ebbb6d883f3f2837a98cc5b5d7f7dde8f35d1c079bad9b3d94a5"}
Jul 03 14:10:45.772558 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:45.772315    2563 scope.go:117] "RemoveContainer" containerID="5e6c6d893591ebbb6d883f3f2837a98cc5b5d7f7dde8f35d1c079bad9b3d94a5"
Jul 03 14:10:46.778958 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:46.778923    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9kjx7" event={"ID":"a258d061-bd67-45af-b0a1-4440e92a1258","Type":"ContainerStarted","Data":"d499082b70aaaaf87b38672e861e4d537995b9853ca64d652fbe2575d855f2e7"}
Jul 03 14:10:51.700017 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:51.699989    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp"
Jul 03 14:10:51.703648 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:51.703622    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-649b8755c8-dfnkp"
Jul 03 14:10:57.827093 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:57.827049    2563 generic.go:356] "Generic (PLEG): container finished" podID="8a71b7ee-121f-474e-a3b6-dbbbaa61daa9" containerID="748df83199c6502863d008fb5c6331cfa50c98c3f91d38d89fb34f35d47fcdd1" exitCode=0
Jul 03 14:10:57.827531 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:57.827113    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-l9qcj" event={"ID":"8a71b7ee-121f-474e-a3b6-dbbbaa61daa9","Type":"ContainerDied","Data":"748df83199c6502863d008fb5c6331cfa50c98c3f91d38d89fb34f35d47fcdd1"}
Jul 03 14:10:57.827531 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:57.827511    2563 scope.go:117] "RemoveContainer" containerID="748df83199c6502863d008fb5c6331cfa50c98c3f91d38d89fb34f35d47fcdd1"
Jul 03 14:10:58.833214 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:10:58.833176    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-l9qcj" event={"ID":"8a71b7ee-121f-474e-a3b6-dbbbaa61daa9","Type":"ContainerStarted","Data":"6a1108be0f2f908779a53e33f7a44e9058b110891a57c07b8475ed56f9c945e2"}
Jul 03 14:11:13.934109 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:13.934076    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:13.952594 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:13.952566    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:14.916872 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:14.916836    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:27.315759 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:27.315722    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 03 14:11:27.316297 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:27.316022    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerName="alertmanager" containerID="cri-o://65456891a2f15f6e131fffef4343346a32464223183cabb15527671529641260" gracePeriod=120
Jul 03 14:11:27.316297 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:27.316091    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerName="kube-rbac-proxy-metric" containerID="cri-o://8caf2be42e9daff9d33141f0c29241ea692cf574a156780b213bb4b276cdfbc0" gracePeriod=120
Jul 03 14:11:27.316297 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:27.316117    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerName="kube-rbac-proxy-web" containerID="cri-o://7fb889cb8ba04c5a54adb985f62230a53dc58d21669217112e7172324f3939c3" gracePeriod=120
Jul 03 14:11:27.316297 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:27.316141    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerName="kube-rbac-proxy" containerID="cri-o://179df6a20b7cae3782550e6903e14aa53a5aac5a4221443c21f7f981d7ce7323" gracePeriod=120
Jul 03 14:11:27.316297 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:27.316163    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerName="config-reloader" containerID="cri-o://efe709216492b2b541f8df3d5f2a2be8b950ec72b23a1b9e40ef3909e38f02fa" gracePeriod=120
Jul 03 14:11:27.316297 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:27.316186    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerName="prom-label-proxy" containerID="cri-o://a3971d489da0a709c192088118f58bbc26528a9c11a12fd9a3ff82282ca78e40" gracePeriod=120
Jul 03 14:11:27.955562 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:27.955528    2563 generic.go:356] "Generic (PLEG): container finished" podID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerID="a3971d489da0a709c192088118f58bbc26528a9c11a12fd9a3ff82282ca78e40" exitCode=0
Jul 03 14:11:27.955562 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:27.955553    2563 generic.go:356] "Generic (PLEG): container finished" podID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerID="179df6a20b7cae3782550e6903e14aa53a5aac5a4221443c21f7f981d7ce7323" exitCode=0
Jul 03 14:11:27.955562 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:27.955560    2563 generic.go:356] "Generic (PLEG): container finished" podID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerID="efe709216492b2b541f8df3d5f2a2be8b950ec72b23a1b9e40ef3909e38f02fa" exitCode=0
Jul 03 14:11:27.955562 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:27.955565    2563 generic.go:356] "Generic (PLEG): container finished" podID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerID="65456891a2f15f6e131fffef4343346a32464223183cabb15527671529641260" exitCode=0
Jul 03 14:11:27.955879 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:27.955604    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f129596-3617-4931-9dbe-5c3a4469e16a","Type":"ContainerDied","Data":"a3971d489da0a709c192088118f58bbc26528a9c11a12fd9a3ff82282ca78e40"}
Jul 03 14:11:27.955879 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:27.955686    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f129596-3617-4931-9dbe-5c3a4469e16a","Type":"ContainerDied","Data":"179df6a20b7cae3782550e6903e14aa53a5aac5a4221443c21f7f981d7ce7323"}
Jul 03 14:11:27.955879 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:27.955728    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f129596-3617-4931-9dbe-5c3a4469e16a","Type":"ContainerDied","Data":"efe709216492b2b541f8df3d5f2a2be8b950ec72b23a1b9e40ef3909e38f02fa"}
Jul 03 14:11:27.955879 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:27.955764    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f129596-3617-4931-9dbe-5c3a4469e16a","Type":"ContainerDied","Data":"65456891a2f15f6e131fffef4343346a32464223183cabb15527671529641260"}
Jul 03 14:11:28.572721 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.572700    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:28.659963 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.659891    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-cluster-tls-config\") pod \"9f129596-3617-4931-9dbe-5c3a4469e16a\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") "
Jul 03 14:11:28.659963 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.659926    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mnhv\" (UniqueName: \"kubernetes.io/projected/9f129596-3617-4931-9dbe-5c3a4469e16a-kube-api-access-2mnhv\") pod \"9f129596-3617-4931-9dbe-5c3a4469e16a\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") "
Jul 03 14:11:28.659963 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.659961    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-secret-alertmanager-main-tls\") pod \"9f129596-3617-4931-9dbe-5c3a4469e16a\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") "
Jul 03 14:11:28.660236 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.659996    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f129596-3617-4931-9dbe-5c3a4469e16a-alertmanager-trusted-ca-bundle\") pod \"9f129596-3617-4931-9dbe-5c3a4469e16a\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") "
Jul 03 14:11:28.660236 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.660030    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"9f129596-3617-4931-9dbe-5c3a4469e16a\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") "
Jul 03 14:11:28.660236 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.660048    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9f129596-3617-4931-9dbe-5c3a4469e16a-config-out\") pod \"9f129596-3617-4931-9dbe-5c3a4469e16a\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") "
Jul 03 14:11:28.660236 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.660064    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-web-config\") pod \"9f129596-3617-4931-9dbe-5c3a4469e16a\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") "
Jul 03 14:11:28.660236 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.660120    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f129596-3617-4931-9dbe-5c3a4469e16a-metrics-client-ca\") pod \"9f129596-3617-4931-9dbe-5c3a4469e16a\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") "
Jul 03 14:11:28.660236 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.660161    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-config-volume\") pod \"9f129596-3617-4931-9dbe-5c3a4469e16a\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") "
Jul 03 14:11:28.660236 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.660188    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-secret-alertmanager-kube-rbac-proxy\") pod \"9f129596-3617-4931-9dbe-5c3a4469e16a\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") "
Jul 03 14:11:28.660236 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.660216    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-secret-alertmanager-kube-rbac-proxy-web\") pod \"9f129596-3617-4931-9dbe-5c3a4469e16a\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") "
Jul 03 14:11:28.660236 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.660240    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9f129596-3617-4931-9dbe-5c3a4469e16a-tls-assets\") pod \"9f129596-3617-4931-9dbe-5c3a4469e16a\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") "
Jul 03 14:11:28.660656 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.660277    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9f129596-3617-4931-9dbe-5c3a4469e16a-alertmanager-main-db\") pod \"9f129596-3617-4931-9dbe-5c3a4469e16a\" (UID: \"9f129596-3617-4931-9dbe-5c3a4469e16a\") "
Jul 03 14:11:28.660656 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.660467    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f129596-3617-4931-9dbe-5c3a4469e16a-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "9f129596-3617-4931-9dbe-5c3a4469e16a" (UID: "9f129596-3617-4931-9dbe-5c3a4469e16a"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:11:28.660764 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.660654    2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f129596-3617-4931-9dbe-5c3a4469e16a-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:28.660820 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.660781    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f129596-3617-4931-9dbe-5c3a4469e16a-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "9f129596-3617-4931-9dbe-5c3a4469e16a" (UID: "9f129596-3617-4931-9dbe-5c3a4469e16a"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:11:28.660879 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.660796    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f129596-3617-4931-9dbe-5c3a4469e16a-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "9f129596-3617-4931-9dbe-5c3a4469e16a" (UID: "9f129596-3617-4931-9dbe-5c3a4469e16a"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:11:28.662887 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.662852    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f129596-3617-4931-9dbe-5c3a4469e16a-kube-api-access-2mnhv" (OuterVolumeSpecName: "kube-api-access-2mnhv") pod "9f129596-3617-4931-9dbe-5c3a4469e16a" (UID: "9f129596-3617-4931-9dbe-5c3a4469e16a"). InnerVolumeSpecName "kube-api-access-2mnhv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:11:28.663446 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.663406    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "9f129596-3617-4931-9dbe-5c3a4469e16a" (UID: "9f129596-3617-4931-9dbe-5c3a4469e16a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:11:28.663636 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.663606    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "9f129596-3617-4931-9dbe-5c3a4469e16a" (UID: "9f129596-3617-4931-9dbe-5c3a4469e16a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:11:28.664212 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.664167    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-config-volume" (OuterVolumeSpecName: "config-volume") pod "9f129596-3617-4931-9dbe-5c3a4469e16a" (UID: "9f129596-3617-4931-9dbe-5c3a4469e16a"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:11:28.664791 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.664771    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "9f129596-3617-4931-9dbe-5c3a4469e16a" (UID: "9f129596-3617-4931-9dbe-5c3a4469e16a"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:11:28.665099 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.665075    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f129596-3617-4931-9dbe-5c3a4469e16a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "9f129596-3617-4931-9dbe-5c3a4469e16a" (UID: "9f129596-3617-4931-9dbe-5c3a4469e16a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:11:28.665223 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.665200    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f129596-3617-4931-9dbe-5c3a4469e16a-config-out" (OuterVolumeSpecName: "config-out") pod "9f129596-3617-4931-9dbe-5c3a4469e16a" (UID: "9f129596-3617-4931-9dbe-5c3a4469e16a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:11:28.665723 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.665700    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "9f129596-3617-4931-9dbe-5c3a4469e16a" (UID: "9f129596-3617-4931-9dbe-5c3a4469e16a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:11:28.668795 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.668770    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "9f129596-3617-4931-9dbe-5c3a4469e16a" (UID: "9f129596-3617-4931-9dbe-5c3a4469e16a"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:11:28.674811 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.674790    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-web-config" (OuterVolumeSpecName: "web-config") pod "9f129596-3617-4931-9dbe-5c3a4469e16a" (UID: "9f129596-3617-4931-9dbe-5c3a4469e16a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:11:28.761147 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.761113    2563 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-config-volume\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:28.761147 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.761143    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:28.761147 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.761153    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:28.761365 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.761163    2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9f129596-3617-4931-9dbe-5c3a4469e16a-tls-assets\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:28.761365 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.761172    2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9f129596-3617-4931-9dbe-5c3a4469e16a-alertmanager-main-db\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:28.761365 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.761180    2563 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-cluster-tls-config\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:28.761365 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.761188    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mnhv\" (UniqueName: \"kubernetes.io/projected/9f129596-3617-4931-9dbe-5c3a4469e16a-kube-api-access-2mnhv\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:28.761365 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.761195    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-secret-alertmanager-main-tls\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:28.761365 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.761204    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:28.761365 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.761213    2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9f129596-3617-4931-9dbe-5c3a4469e16a-config-out\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:28.761365 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.761220    2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9f129596-3617-4931-9dbe-5c3a4469e16a-web-config\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:28.761365 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.761228    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f129596-3617-4931-9dbe-5c3a4469e16a-metrics-client-ca\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:28.962273 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.962189    2563 generic.go:356] "Generic (PLEG): container finished" podID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerID="8caf2be42e9daff9d33141f0c29241ea692cf574a156780b213bb4b276cdfbc0" exitCode=0
Jul 03 14:11:28.962273 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.962216    2563 generic.go:356] "Generic (PLEG): container finished" podID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerID="7fb889cb8ba04c5a54adb985f62230a53dc58d21669217112e7172324f3939c3" exitCode=0
Jul 03 14:11:28.962273 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.962264    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f129596-3617-4931-9dbe-5c3a4469e16a","Type":"ContainerDied","Data":"8caf2be42e9daff9d33141f0c29241ea692cf574a156780b213bb4b276cdfbc0"}
Jul 03 14:11:28.962522 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.962296    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:28.962522 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.962342    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f129596-3617-4931-9dbe-5c3a4469e16a","Type":"ContainerDied","Data":"7fb889cb8ba04c5a54adb985f62230a53dc58d21669217112e7172324f3939c3"}
Jul 03 14:11:28.962522 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.962370    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f129596-3617-4931-9dbe-5c3a4469e16a","Type":"ContainerDied","Data":"a24a8acc12498a5cde233d23a9df73d22e66faf51058728a970facbfeb47d9c2"}
Jul 03 14:11:28.962522 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.962400    2563 scope.go:117] "RemoveContainer" containerID="a3971d489da0a709c192088118f58bbc26528a9c11a12fd9a3ff82282ca78e40"
Jul 03 14:11:28.970207 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.970192    2563 scope.go:117] "RemoveContainer" containerID="8caf2be42e9daff9d33141f0c29241ea692cf574a156780b213bb4b276cdfbc0"
Jul 03 14:11:28.977954 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.977937    2563 scope.go:117] "RemoveContainer" containerID="179df6a20b7cae3782550e6903e14aa53a5aac5a4221443c21f7f981d7ce7323"
Jul 03 14:11:28.984785 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.984764    2563 scope.go:117] "RemoveContainer" containerID="7fb889cb8ba04c5a54adb985f62230a53dc58d21669217112e7172324f3939c3"
Jul 03 14:11:28.991268 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.991239    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 03 14:11:28.992694 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.992672    2563 scope.go:117] "RemoveContainer" containerID="efe709216492b2b541f8df3d5f2a2be8b950ec72b23a1b9e40ef3909e38f02fa"
Jul 03 14:11:28.997339 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:28.997319    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 03 14:11:29.000469 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.000446    2563 scope.go:117] "RemoveContainer" containerID="65456891a2f15f6e131fffef4343346a32464223183cabb15527671529641260"
Jul 03 14:11:29.007374 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.007351    2563 scope.go:117] "RemoveContainer" containerID="f520a40fc174d025c46caae0df8cc8bca145c4a51c491ab18b4b57dbf6d19ee8"
Jul 03 14:11:29.015909 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.015888    2563 scope.go:117] "RemoveContainer" containerID="a3971d489da0a709c192088118f58bbc26528a9c11a12fd9a3ff82282ca78e40"
Jul 03 14:11:29.016549 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:11:29.016521    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3971d489da0a709c192088118f58bbc26528a9c11a12fd9a3ff82282ca78e40\": container with ID starting with a3971d489da0a709c192088118f58bbc26528a9c11a12fd9a3ff82282ca78e40 not found: ID does not exist" containerID="a3971d489da0a709c192088118f58bbc26528a9c11a12fd9a3ff82282ca78e40"
Jul 03 14:11:29.016637 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.016551    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3971d489da0a709c192088118f58bbc26528a9c11a12fd9a3ff82282ca78e40"} err="failed to get container status \"a3971d489da0a709c192088118f58bbc26528a9c11a12fd9a3ff82282ca78e40\": rpc error: code = NotFound desc = could not find container \"a3971d489da0a709c192088118f58bbc26528a9c11a12fd9a3ff82282ca78e40\": container with ID starting with a3971d489da0a709c192088118f58bbc26528a9c11a12fd9a3ff82282ca78e40 not found: ID does not exist"
Jul 03 14:11:29.016637 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.016576    2563 scope.go:117] "RemoveContainer" containerID="8caf2be42e9daff9d33141f0c29241ea692cf574a156780b213bb4b276cdfbc0"
Jul 03 14:11:29.016850 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:11:29.016835    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8caf2be42e9daff9d33141f0c29241ea692cf574a156780b213bb4b276cdfbc0\": container with ID starting with 8caf2be42e9daff9d33141f0c29241ea692cf574a156780b213bb4b276cdfbc0 not found: ID does not exist" containerID="8caf2be42e9daff9d33141f0c29241ea692cf574a156780b213bb4b276cdfbc0"
Jul 03 14:11:29.016914 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.016858    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8caf2be42e9daff9d33141f0c29241ea692cf574a156780b213bb4b276cdfbc0"} err="failed to get container status \"8caf2be42e9daff9d33141f0c29241ea692cf574a156780b213bb4b276cdfbc0\": rpc error: code = NotFound desc = could not find container \"8caf2be42e9daff9d33141f0c29241ea692cf574a156780b213bb4b276cdfbc0\": container with ID starting with 8caf2be42e9daff9d33141f0c29241ea692cf574a156780b213bb4b276cdfbc0 not found: ID does not exist"
Jul 03 14:11:29.016914 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.016877    2563 scope.go:117] "RemoveContainer" containerID="179df6a20b7cae3782550e6903e14aa53a5aac5a4221443c21f7f981d7ce7323"
Jul 03 14:11:29.017146 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:11:29.017129    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"179df6a20b7cae3782550e6903e14aa53a5aac5a4221443c21f7f981d7ce7323\": container with ID starting with 179df6a20b7cae3782550e6903e14aa53a5aac5a4221443c21f7f981d7ce7323 not found: ID does not exist" containerID="179df6a20b7cae3782550e6903e14aa53a5aac5a4221443c21f7f981d7ce7323"
Jul 03 14:11:29.017191 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.017151    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"179df6a20b7cae3782550e6903e14aa53a5aac5a4221443c21f7f981d7ce7323"} err="failed to get container status \"179df6a20b7cae3782550e6903e14aa53a5aac5a4221443c21f7f981d7ce7323\": rpc error: code = NotFound desc = could not find container \"179df6a20b7cae3782550e6903e14aa53a5aac5a4221443c21f7f981d7ce7323\": container with ID starting with 179df6a20b7cae3782550e6903e14aa53a5aac5a4221443c21f7f981d7ce7323 not found: ID does not exist"
Jul 03 14:11:29.017191 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.017167    2563 scope.go:117] "RemoveContainer" containerID="7fb889cb8ba04c5a54adb985f62230a53dc58d21669217112e7172324f3939c3"
Jul 03 14:11:29.017404 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:11:29.017387    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fb889cb8ba04c5a54adb985f62230a53dc58d21669217112e7172324f3939c3\": container with ID starting with 7fb889cb8ba04c5a54adb985f62230a53dc58d21669217112e7172324f3939c3 not found: ID does not exist" containerID="7fb889cb8ba04c5a54adb985f62230a53dc58d21669217112e7172324f3939c3"
Jul 03 14:11:29.017443 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.017410    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fb889cb8ba04c5a54adb985f62230a53dc58d21669217112e7172324f3939c3"} err="failed to get container status \"7fb889cb8ba04c5a54adb985f62230a53dc58d21669217112e7172324f3939c3\": rpc error: code = NotFound desc = could not find container \"7fb889cb8ba04c5a54adb985f62230a53dc58d21669217112e7172324f3939c3\": container with ID starting with 7fb889cb8ba04c5a54adb985f62230a53dc58d21669217112e7172324f3939c3 not found: ID does not exist"
Jul 03 14:11:29.017443 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.017423    2563 scope.go:117] "RemoveContainer" containerID="efe709216492b2b541f8df3d5f2a2be8b950ec72b23a1b9e40ef3909e38f02fa"
Jul 03 14:11:29.017668 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:11:29.017649    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efe709216492b2b541f8df3d5f2a2be8b950ec72b23a1b9e40ef3909e38f02fa\": container with ID starting with efe709216492b2b541f8df3d5f2a2be8b950ec72b23a1b9e40ef3909e38f02fa not found: ID does not exist" containerID="efe709216492b2b541f8df3d5f2a2be8b950ec72b23a1b9e40ef3909e38f02fa"
Jul 03 14:11:29.017719 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.017673    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efe709216492b2b541f8df3d5f2a2be8b950ec72b23a1b9e40ef3909e38f02fa"} err="failed to get container status \"efe709216492b2b541f8df3d5f2a2be8b950ec72b23a1b9e40ef3909e38f02fa\": rpc error: code = NotFound desc = could not find container \"efe709216492b2b541f8df3d5f2a2be8b950ec72b23a1b9e40ef3909e38f02fa\": container with ID starting with efe709216492b2b541f8df3d5f2a2be8b950ec72b23a1b9e40ef3909e38f02fa not found: ID does not exist"
Jul 03 14:11:29.017719 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.017693    2563 scope.go:117] "RemoveContainer" containerID="65456891a2f15f6e131fffef4343346a32464223183cabb15527671529641260"
Jul 03 14:11:29.017913 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:11:29.017897    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65456891a2f15f6e131fffef4343346a32464223183cabb15527671529641260\": container with ID starting with 65456891a2f15f6e131fffef4343346a32464223183cabb15527671529641260 not found: ID does not exist" containerID="65456891a2f15f6e131fffef4343346a32464223183cabb15527671529641260"
Jul 03 14:11:29.017953 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.017919    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65456891a2f15f6e131fffef4343346a32464223183cabb15527671529641260"} err="failed to get container status \"65456891a2f15f6e131fffef4343346a32464223183cabb15527671529641260\": rpc error: code = NotFound desc = could not find container \"65456891a2f15f6e131fffef4343346a32464223183cabb15527671529641260\": container with ID starting with 65456891a2f15f6e131fffef4343346a32464223183cabb15527671529641260 not found: ID does not exist"
Jul 03 14:11:29.017953 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.017933    2563 scope.go:117] "RemoveContainer" containerID="f520a40fc174d025c46caae0df8cc8bca145c4a51c491ab18b4b57dbf6d19ee8"
Jul 03 14:11:29.018169 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:11:29.018153    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f520a40fc174d025c46caae0df8cc8bca145c4a51c491ab18b4b57dbf6d19ee8\": container with ID starting with f520a40fc174d025c46caae0df8cc8bca145c4a51c491ab18b4b57dbf6d19ee8 not found: ID does not exist" containerID="f520a40fc174d025c46caae0df8cc8bca145c4a51c491ab18b4b57dbf6d19ee8"
Jul 03 14:11:29.018215 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.018171    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f520a40fc174d025c46caae0df8cc8bca145c4a51c491ab18b4b57dbf6d19ee8"} err="failed to get container status \"f520a40fc174d025c46caae0df8cc8bca145c4a51c491ab18b4b57dbf6d19ee8\": rpc error: code = NotFound desc = could not find container \"f520a40fc174d025c46caae0df8cc8bca145c4a51c491ab18b4b57dbf6d19ee8\": container with ID starting with f520a40fc174d025c46caae0df8cc8bca145c4a51c491ab18b4b57dbf6d19ee8 not found: ID does not exist"
Jul 03 14:11:29.018215 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.018184    2563 scope.go:117] "RemoveContainer" containerID="a3971d489da0a709c192088118f58bbc26528a9c11a12fd9a3ff82282ca78e40"
Jul 03 14:11:29.018395 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.018376    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3971d489da0a709c192088118f58bbc26528a9c11a12fd9a3ff82282ca78e40"} err="failed to get container status \"a3971d489da0a709c192088118f58bbc26528a9c11a12fd9a3ff82282ca78e40\": rpc error: code = NotFound desc = could not find container \"a3971d489da0a709c192088118f58bbc26528a9c11a12fd9a3ff82282ca78e40\": container with ID starting with a3971d489da0a709c192088118f58bbc26528a9c11a12fd9a3ff82282ca78e40 not found: ID does not exist"
Jul 03 14:11:29.018434 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.018395    2563 scope.go:117] "RemoveContainer" containerID="8caf2be42e9daff9d33141f0c29241ea692cf574a156780b213bb4b276cdfbc0"
Jul 03 14:11:29.018615 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.018595    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8caf2be42e9daff9d33141f0c29241ea692cf574a156780b213bb4b276cdfbc0"} err="failed to get container status \"8caf2be42e9daff9d33141f0c29241ea692cf574a156780b213bb4b276cdfbc0\": rpc error: code = NotFound desc = could not find container \"8caf2be42e9daff9d33141f0c29241ea692cf574a156780b213bb4b276cdfbc0\": container with ID starting with 8caf2be42e9daff9d33141f0c29241ea692cf574a156780b213bb4b276cdfbc0 not found: ID does not exist"
Jul 03 14:11:29.018665 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.018616    2563 scope.go:117] "RemoveContainer" containerID="179df6a20b7cae3782550e6903e14aa53a5aac5a4221443c21f7f981d7ce7323"
Jul 03 14:11:29.018814 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.018798    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"179df6a20b7cae3782550e6903e14aa53a5aac5a4221443c21f7f981d7ce7323"} err="failed to get container status \"179df6a20b7cae3782550e6903e14aa53a5aac5a4221443c21f7f981d7ce7323\": rpc error: code = NotFound desc = could not find container \"179df6a20b7cae3782550e6903e14aa53a5aac5a4221443c21f7f981d7ce7323\": container with ID starting with 179df6a20b7cae3782550e6903e14aa53a5aac5a4221443c21f7f981d7ce7323 not found: ID does not exist"
Jul 03 14:11:29.018855 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.018813    2563 scope.go:117] "RemoveContainer" containerID="7fb889cb8ba04c5a54adb985f62230a53dc58d21669217112e7172324f3939c3"
Jul 03 14:11:29.019036 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.019016    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fb889cb8ba04c5a54adb985f62230a53dc58d21669217112e7172324f3939c3"} err="failed to get container status \"7fb889cb8ba04c5a54adb985f62230a53dc58d21669217112e7172324f3939c3\": rpc error: code = NotFound desc = could not find container \"7fb889cb8ba04c5a54adb985f62230a53dc58d21669217112e7172324f3939c3\": container with ID starting with 7fb889cb8ba04c5a54adb985f62230a53dc58d21669217112e7172324f3939c3 not found: ID does not exist"
Jul 03 14:11:29.019090 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.019036    2563 scope.go:117] "RemoveContainer" containerID="efe709216492b2b541f8df3d5f2a2be8b950ec72b23a1b9e40ef3909e38f02fa"
Jul 03 14:11:29.019255 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.019237    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efe709216492b2b541f8df3d5f2a2be8b950ec72b23a1b9e40ef3909e38f02fa"} err="failed to get container status \"efe709216492b2b541f8df3d5f2a2be8b950ec72b23a1b9e40ef3909e38f02fa\": rpc error: code = NotFound desc = could not find container \"efe709216492b2b541f8df3d5f2a2be8b950ec72b23a1b9e40ef3909e38f02fa\": container with ID starting with efe709216492b2b541f8df3d5f2a2be8b950ec72b23a1b9e40ef3909e38f02fa not found: ID does not exist"
Jul 03 14:11:29.019303 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.019255    2563 scope.go:117] "RemoveContainer" containerID="65456891a2f15f6e131fffef4343346a32464223183cabb15527671529641260"
Jul 03 14:11:29.019448 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.019433    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65456891a2f15f6e131fffef4343346a32464223183cabb15527671529641260"} err="failed to get container status \"65456891a2f15f6e131fffef4343346a32464223183cabb15527671529641260\": rpc error: code = NotFound desc = could not find container \"65456891a2f15f6e131fffef4343346a32464223183cabb15527671529641260\": container with ID starting with 65456891a2f15f6e131fffef4343346a32464223183cabb15527671529641260 not found: ID does not exist"
Jul 03 14:11:29.019485 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.019447    2563 scope.go:117] "RemoveContainer" containerID="f520a40fc174d025c46caae0df8cc8bca145c4a51c491ab18b4b57dbf6d19ee8"
Jul 03 14:11:29.019634 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.019616    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f520a40fc174d025c46caae0df8cc8bca145c4a51c491ab18b4b57dbf6d19ee8"} err="failed to get container status \"f520a40fc174d025c46caae0df8cc8bca145c4a51c491ab18b4b57dbf6d19ee8\": rpc error: code = NotFound desc = could not find container \"f520a40fc174d025c46caae0df8cc8bca145c4a51c491ab18b4b57dbf6d19ee8\": container with ID starting with f520a40fc174d025c46caae0df8cc8bca145c4a51c491ab18b4b57dbf6d19ee8 not found: ID does not exist"
Jul 03 14:11:29.036359 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.036335    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 03 14:11:29.036745 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.036729    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerName="prom-label-proxy"
Jul 03 14:11:29.036796 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.036747    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerName="prom-label-proxy"
Jul 03 14:11:29.036796 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.036762    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerName="config-reloader"
Jul 03 14:11:29.036796 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.036767    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerName="config-reloader"
Jul 03 14:11:29.036796 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.036776    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerName="kube-rbac-proxy-web"
Jul 03 14:11:29.036796 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.036781    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerName="kube-rbac-proxy-web"
Jul 03 14:11:29.036796 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.036791    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerName="kube-rbac-proxy-metric"
Jul 03 14:11:29.036796 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.036796    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerName="kube-rbac-proxy-metric"
Jul 03 14:11:29.037050 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.036806    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerName="init-config-reloader"
Jul 03 14:11:29.037050 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.036812    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerName="init-config-reloader"
Jul 03 14:11:29.037050 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.036818    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerName="alertmanager"
Jul 03 14:11:29.037050 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.036823    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerName="alertmanager"
Jul 03 14:11:29.037050 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.036833    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb3f129d-0917-4c5b-a369-ac9a0f9e63a2" containerName="registry"
Jul 03 14:11:29.037050 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.036838    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb3f129d-0917-4c5b-a369-ac9a0f9e63a2" containerName="registry"
Jul 03 14:11:29.037050 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.036843    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31c13a3d-642e-4dae-b3f4-0d8596014470" containerName="console"
Jul 03 14:11:29.037050 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.036848    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="31c13a3d-642e-4dae-b3f4-0d8596014470" containerName="console"
Jul 03 14:11:29.037050 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.036854    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerName="kube-rbac-proxy"
Jul 03 14:11:29.037050 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.036860    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerName="kube-rbac-proxy"
Jul 03 14:11:29.037050 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.036905    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerName="kube-rbac-proxy"
Jul 03 14:11:29.037050 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.036912    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerName="kube-rbac-proxy-metric"
Jul 03 14:11:29.037050 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.036918    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerName="alertmanager"
Jul 03 14:11:29.037050 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.036925    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb3f129d-0917-4c5b-a369-ac9a0f9e63a2" containerName="registry"
Jul 03 14:11:29.037050 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.036929    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerName="prom-label-proxy"
Jul 03 14:11:29.037050 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.036935    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerName="kube-rbac-proxy-web"
Jul 03 14:11:29.037050 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.036940    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f129596-3617-4931-9dbe-5c3a4469e16a" containerName="config-reloader"
Jul 03 14:11:29.037050 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.036945    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="31c13a3d-642e-4dae-b3f4-0d8596014470" containerName="console"
Jul 03 14:11:29.040302 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.040283    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.050856 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.050837    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 03 14:11:29.051030 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.050924    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 03 14:11:29.051030 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.051007    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 03 14:11:29.051158 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.051059    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 03 14:11:29.051311 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.051297    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 03 14:11:29.051358 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.051300    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 03 14:11:29.051358 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.051301    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 03 14:11:29.051358 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.051344    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-r7pvf\""
Jul 03 14:11:29.051567 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.051554    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 03 14:11:29.057506 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.057481    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f129596-3617-4931-9dbe-5c3a4469e16a" path="/var/lib/kubelet/pods/9f129596-3617-4931-9dbe-5c3a4469e16a/volumes"
Jul 03 14:11:29.062832 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.062812    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38e7636c-38ae-49dd-8ecd-57d50095e31e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.062980 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.062846    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38e7636c-38ae-49dd-8ecd-57d50095e31e-config-out\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.062980 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.062870    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/38e7636c-38ae-49dd-8ecd-57d50095e31e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.063150 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.063088    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38e7636c-38ae-49dd-8ecd-57d50095e31e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.063150 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.063126    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txvtl\" (UniqueName: \"kubernetes.io/projected/38e7636c-38ae-49dd-8ecd-57d50095e31e-kube-api-access-txvtl\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.063285 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.063155    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38e7636c-38ae-49dd-8ecd-57d50095e31e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.063285 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.063192    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38e7636c-38ae-49dd-8ecd-57d50095e31e-web-config\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.063285 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.063217    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38e7636c-38ae-49dd-8ecd-57d50095e31e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.063285 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.063255    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/38e7636c-38ae-49dd-8ecd-57d50095e31e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.063514 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.063291    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38e7636c-38ae-49dd-8ecd-57d50095e31e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.063514 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.063310    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/38e7636c-38ae-49dd-8ecd-57d50095e31e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.063514 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.063348    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/38e7636c-38ae-49dd-8ecd-57d50095e31e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.063514 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.063378    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/38e7636c-38ae-49dd-8ecd-57d50095e31e-config-volume\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.063724 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.063708    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 03 14:11:29.077960 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.077938    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 03 14:11:29.164490 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.164454    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38e7636c-38ae-49dd-8ecd-57d50095e31e-web-config\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.164490 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.164495    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38e7636c-38ae-49dd-8ecd-57d50095e31e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.164714 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.164516    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/38e7636c-38ae-49dd-8ecd-57d50095e31e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.164714 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.164534    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38e7636c-38ae-49dd-8ecd-57d50095e31e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.164820 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.164794    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/38e7636c-38ae-49dd-8ecd-57d50095e31e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.164875 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.164848    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/38e7636c-38ae-49dd-8ecd-57d50095e31e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.164921 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.164881    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/38e7636c-38ae-49dd-8ecd-57d50095e31e-config-volume\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.164988 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.164955    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38e7636c-38ae-49dd-8ecd-57d50095e31e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.165061 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.165015    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38e7636c-38ae-49dd-8ecd-57d50095e31e-config-out\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.165061 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.165043    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/38e7636c-38ae-49dd-8ecd-57d50095e31e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.165205 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.165091    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38e7636c-38ae-49dd-8ecd-57d50095e31e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.165205 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.165135    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txvtl\" (UniqueName: \"kubernetes.io/projected/38e7636c-38ae-49dd-8ecd-57d50095e31e-kube-api-access-txvtl\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.165205 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.165168    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38e7636c-38ae-49dd-8ecd-57d50095e31e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.165540 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.165510    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38e7636c-38ae-49dd-8ecd-57d50095e31e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.167121 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.167096    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38e7636c-38ae-49dd-8ecd-57d50095e31e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.167468 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.167443    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/38e7636c-38ae-49dd-8ecd-57d50095e31e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.167560 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.167523    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38e7636c-38ae-49dd-8ecd-57d50095e31e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.167747 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.167725    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38e7636c-38ae-49dd-8ecd-57d50095e31e-web-config\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.168234 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.168214    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/38e7636c-38ae-49dd-8ecd-57d50095e31e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.168327 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.168240    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38e7636c-38ae-49dd-8ecd-57d50095e31e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.168327 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.168274    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/38e7636c-38ae-49dd-8ecd-57d50095e31e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.168415 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.168395    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/38e7636c-38ae-49dd-8ecd-57d50095e31e-config-volume\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.168458 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.168403    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/38e7636c-38ae-49dd-8ecd-57d50095e31e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.169034 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.169019    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38e7636c-38ae-49dd-8ecd-57d50095e31e-config-out\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.169276 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.169259    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38e7636c-38ae-49dd-8ecd-57d50095e31e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.176163 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.176142    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txvtl\" (UniqueName: \"kubernetes.io/projected/38e7636c-38ae-49dd-8ecd-57d50095e31e-kube-api-access-txvtl\") pod \"alertmanager-main-0\" (UID: \"38e7636c-38ae-49dd-8ecd-57d50095e31e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.349220 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.349138    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 03 14:11:29.501641 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.501300    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 03 14:11:29.504211 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:11:29.504181    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38e7636c_38ae_49dd_8ecd_57d50095e31e.slice/crio-6e7a0915397f3e9eeacf558f6c8ba4323e215207319ed481f27923b4db49d829 WatchSource:0}: Error finding container 6e7a0915397f3e9eeacf558f6c8ba4323e215207319ed481f27923b4db49d829: Status 404 returned error can't find the container with id 6e7a0915397f3e9eeacf558f6c8ba4323e215207319ed481f27923b4db49d829
Jul 03 14:11:29.968232 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.968201    2563 generic.go:356] "Generic (PLEG): container finished" podID="38e7636c-38ae-49dd-8ecd-57d50095e31e" containerID="ba3f2ee67e95e86a81d4d43390d8e5c1c1a0747a419cea9bd445c3d453b11459" exitCode=0
Jul 03 14:11:29.968622 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.968275    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38e7636c-38ae-49dd-8ecd-57d50095e31e","Type":"ContainerDied","Data":"ba3f2ee67e95e86a81d4d43390d8e5c1c1a0747a419cea9bd445c3d453b11459"}
Jul 03 14:11:29.968622 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:29.968341    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38e7636c-38ae-49dd-8ecd-57d50095e31e","Type":"ContainerStarted","Data":"6e7a0915397f3e9eeacf558f6c8ba4323e215207319ed481f27923b4db49d829"}
Jul 03 14:11:30.975432 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:30.975395    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38e7636c-38ae-49dd-8ecd-57d50095e31e","Type":"ContainerStarted","Data":"c24b075d0ee3cc674e35bdffdbd86625b113687968d94485cdb656d4b3445add"}
Jul 03 14:11:30.975818 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:30.975507    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38e7636c-38ae-49dd-8ecd-57d50095e31e","Type":"ContainerStarted","Data":"80b55f2d67312d892cf5eb3e12f772784bddb14b0929132a430825269320fd3a"}
Jul 03 14:11:30.975818 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:30.975550    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38e7636c-38ae-49dd-8ecd-57d50095e31e","Type":"ContainerStarted","Data":"100439c34b3ccd4c55d0d9781d41e6dc9f7179374348c3b37b57b6851a938c0d"}
Jul 03 14:11:30.975818 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:30.975590    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38e7636c-38ae-49dd-8ecd-57d50095e31e","Type":"ContainerStarted","Data":"97aaf0b6d5d18178a6d232cd99d0c80361b9bbe1870f8beaf686deb58ad231ad"}
Jul 03 14:11:30.975818 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:30.975632    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38e7636c-38ae-49dd-8ecd-57d50095e31e","Type":"ContainerStarted","Data":"5576c7bca0a788e7b940e16ea283a2b64eba068f37c8a50d64893215d4379fcf"}
Jul 03 14:11:30.975818 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:30.975671    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38e7636c-38ae-49dd-8ecd-57d50095e31e","Type":"ContainerStarted","Data":"9c27140a4ce94439631657e969bbe69b685aa4ab691e97d9d1c0993344d66f24"}
Jul 03 14:11:31.023670 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.023613    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.009820553 podStartE2EDuration="2.023597626s" totalImagesPullingTime="0s" totalInitContainerRuntime="13.777073ms" isStatefulPod=false podCreationTimestamp="2026-07-03 14:11:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:11:31.020703615 +0000 UTC m=+204.470837076" watchObservedRunningTime="2026-07-03 14:11:31.023597626 +0000 UTC m=+204.473731065"
Jul 03 14:11:31.558282 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.558242    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7cd5b956b9-r25ff"]
Jul 03 14:11:31.560658 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.560635    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff"
Jul 03 14:11:31.577281 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.577252    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 03 14:11:31.577281 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.577255    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 03 14:11:31.577485 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.577450    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 03 14:11:31.577725 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.577702    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 03 14:11:31.577839 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.577799    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-h67b8\""
Jul 03 14:11:31.584959 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.584939    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 03 14:11:31.586163 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.586146    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 03 14:11:31.587336 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.587317    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d6eca64-8303-4ad4-ae80-197d0983d553-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7cd5b956b9-r25ff\" (UID: \"9d6eca64-8303-4ad4-ae80-197d0983d553\") " pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff"
Jul 03 14:11:31.587402 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.587355    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d6eca64-8303-4ad4-ae80-197d0983d553-serving-certs-ca-bundle\") pod \"telemeter-client-7cd5b956b9-r25ff\" (UID: \"9d6eca64-8303-4ad4-ae80-197d0983d553\") " pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff"
Jul 03 14:11:31.587402 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.587380    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9d6eca64-8303-4ad4-ae80-197d0983d553-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7cd5b956b9-r25ff\" (UID: \"9d6eca64-8303-4ad4-ae80-197d0983d553\") " pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff"
Jul 03 14:11:31.587471 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.587417    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbshm\" (UniqueName: \"kubernetes.io/projected/9d6eca64-8303-4ad4-ae80-197d0983d553-kube-api-access-bbshm\") pod \"telemeter-client-7cd5b956b9-r25ff\" (UID: \"9d6eca64-8303-4ad4-ae80-197d0983d553\") " pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff"
Jul 03 14:11:31.587522 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.587505    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9d6eca64-8303-4ad4-ae80-197d0983d553-telemeter-client-tls\") pod \"telemeter-client-7cd5b956b9-r25ff\" (UID: \"9d6eca64-8303-4ad4-ae80-197d0983d553\") " pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff"
Jul 03 14:11:31.587557 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.587539    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d6eca64-8303-4ad4-ae80-197d0983d553-metrics-client-ca\") pod \"telemeter-client-7cd5b956b9-r25ff\" (UID: \"9d6eca64-8303-4ad4-ae80-197d0983d553\") " pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff"
Jul 03 14:11:31.587591 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.587568    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9d6eca64-8303-4ad4-ae80-197d0983d553-secret-telemeter-client\") pod \"telemeter-client-7cd5b956b9-r25ff\" (UID: \"9d6eca64-8303-4ad4-ae80-197d0983d553\") " pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff"
Jul 03 14:11:31.587634 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.587622    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9d6eca64-8303-4ad4-ae80-197d0983d553-federate-client-tls\") pod \"telemeter-client-7cd5b956b9-r25ff\" (UID: \"9d6eca64-8303-4ad4-ae80-197d0983d553\") " pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff"
Jul 03 14:11:31.610021 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.609992    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7cd5b956b9-r25ff"]
Jul 03 14:11:31.688458 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.688427    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9d6eca64-8303-4ad4-ae80-197d0983d553-secret-telemeter-client\") pod \"telemeter-client-7cd5b956b9-r25ff\" (UID: \"9d6eca64-8303-4ad4-ae80-197d0983d553\") " pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff"
Jul 03 14:11:31.688643 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.688469    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9d6eca64-8303-4ad4-ae80-197d0983d553-federate-client-tls\") pod \"telemeter-client-7cd5b956b9-r25ff\" (UID: \"9d6eca64-8303-4ad4-ae80-197d0983d553\") " pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff"
Jul 03 14:11:31.688643 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.688495    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d6eca64-8303-4ad4-ae80-197d0983d553-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7cd5b956b9-r25ff\" (UID: \"9d6eca64-8303-4ad4-ae80-197d0983d553\") " pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff"
Jul 03 14:11:31.688643 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.688528    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d6eca64-8303-4ad4-ae80-197d0983d553-serving-certs-ca-bundle\") pod \"telemeter-client-7cd5b956b9-r25ff\" (UID: \"9d6eca64-8303-4ad4-ae80-197d0983d553\") " pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff"
Jul 03 14:11:31.688643 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.688545    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9d6eca64-8303-4ad4-ae80-197d0983d553-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7cd5b956b9-r25ff\" (UID: \"9d6eca64-8303-4ad4-ae80-197d0983d553\") " pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff"
Jul 03 14:11:31.688643 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.688583    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbshm\" (UniqueName: \"kubernetes.io/projected/9d6eca64-8303-4ad4-ae80-197d0983d553-kube-api-access-bbshm\") pod \"telemeter-client-7cd5b956b9-r25ff\" (UID: \"9d6eca64-8303-4ad4-ae80-197d0983d553\") " pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff"
Jul 03 14:11:31.688901 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.688643    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9d6eca64-8303-4ad4-ae80-197d0983d553-telemeter-client-tls\") pod \"telemeter-client-7cd5b956b9-r25ff\" (UID: \"9d6eca64-8303-4ad4-ae80-197d0983d553\") " pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff"
Jul 03 14:11:31.688901 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.688668    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d6eca64-8303-4ad4-ae80-197d0983d553-metrics-client-ca\") pod \"telemeter-client-7cd5b956b9-r25ff\" (UID: \"9d6eca64-8303-4ad4-ae80-197d0983d553\") " pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff"
Jul 03 14:11:31.689372 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.689347    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d6eca64-8303-4ad4-ae80-197d0983d553-serving-certs-ca-bundle\") pod \"telemeter-client-7cd5b956b9-r25ff\" (UID: \"9d6eca64-8303-4ad4-ae80-197d0983d553\") " pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff"
Jul 03 14:11:31.689683 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.689656    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d6eca64-8303-4ad4-ae80-197d0983d553-metrics-client-ca\") pod \"telemeter-client-7cd5b956b9-r25ff\" (UID: \"9d6eca64-8303-4ad4-ae80-197d0983d553\") " pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff"
Jul 03 14:11:31.690004 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.689962    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d6eca64-8303-4ad4-ae80-197d0983d553-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7cd5b956b9-r25ff\" (UID: \"9d6eca64-8303-4ad4-ae80-197d0983d553\") " pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff"
Jul 03 14:11:31.691116 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.691090    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9d6eca64-8303-4ad4-ae80-197d0983d553-federate-client-tls\") pod \"telemeter-client-7cd5b956b9-r25ff\" (UID: \"9d6eca64-8303-4ad4-ae80-197d0983d553\") " pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff"
Jul 03 14:11:31.691218 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.691124    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9d6eca64-8303-4ad4-ae80-197d0983d553-telemeter-client-tls\") pod \"telemeter-client-7cd5b956b9-r25ff\" (UID: \"9d6eca64-8303-4ad4-ae80-197d0983d553\") " pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff"
Jul 03 14:11:31.691218 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.691122    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9d6eca64-8303-4ad4-ae80-197d0983d553-secret-telemeter-client\") pod \"telemeter-client-7cd5b956b9-r25ff\" (UID: \"9d6eca64-8303-4ad4-ae80-197d0983d553\") " pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff"
Jul 03 14:11:31.691218 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.691192    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9d6eca64-8303-4ad4-ae80-197d0983d553-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7cd5b956b9-r25ff\" (UID: \"9d6eca64-8303-4ad4-ae80-197d0983d553\") " pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff"
Jul 03 14:11:31.699236 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.699213    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbshm\" (UniqueName: \"kubernetes.io/projected/9d6eca64-8303-4ad4-ae80-197d0983d553-kube-api-access-bbshm\") pod \"telemeter-client-7cd5b956b9-r25ff\" (UID: \"9d6eca64-8303-4ad4-ae80-197d0983d553\") " pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff"
Jul 03 14:11:31.866291 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.866251    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 03 14:11:31.866795 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.866594    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerName="prometheus" containerID="cri-o://108ca6c14620a636508c2fd2dc795f58c3ca5637c08e6af9bc7e880a207833ca" gracePeriod=600
Jul 03 14:11:31.866795 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.866632    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerName="thanos-sidecar" containerID="cri-o://d8b6d3c5529436789bcf1ea432259e0399127b780ce97f53ff3f32137aa71530" gracePeriod=600
Jul 03 14:11:31.866795 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.866652    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerName="config-reloader" containerID="cri-o://2ecedefaf5adf1d2e20f57db254a8fcbfd85f4e81b1fa261f34ba62ac8447140" gracePeriod=600
Jul 03 14:11:31.866795 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.866679    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerName="kube-rbac-proxy-thanos" containerID="cri-o://ad47a462a441c5f6ea4fe067960b8b0c340d91fb4c692045b25035b5080525fe" gracePeriod=600
Jul 03 14:11:31.866795 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.866681    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerName="kube-rbac-proxy" containerID="cri-o://72c55fd67f0614fb1567c4cbbcc2c23759e932804c4c27c1d42c910295bdec7f" gracePeriod=600
Jul 03 14:11:31.866795 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.866632    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerName="kube-rbac-proxy-web" containerID="cri-o://30f0f0187a36b3c74dff3bd4e0cd140eb60b3c2dfb42f91cf2b0a1dae1c0acc4" gracePeriod=600
Jul 03 14:11:31.869332 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.869306    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff"
Jul 03 14:11:31.987310 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.987278    2563 generic.go:356] "Generic (PLEG): container finished" podID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerID="72c55fd67f0614fb1567c4cbbcc2c23759e932804c4c27c1d42c910295bdec7f" exitCode=0
Jul 03 14:11:31.987310 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.987307    2563 generic.go:356] "Generic (PLEG): container finished" podID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerID="30f0f0187a36b3c74dff3bd4e0cd140eb60b3c2dfb42f91cf2b0a1dae1c0acc4" exitCode=0
Jul 03 14:11:31.987735 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.987318    2563 generic.go:356] "Generic (PLEG): container finished" podID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerID="d8b6d3c5529436789bcf1ea432259e0399127b780ce97f53ff3f32137aa71530" exitCode=0
Jul 03 14:11:31.987735 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.987326    2563 generic.go:356] "Generic (PLEG): container finished" podID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerID="108ca6c14620a636508c2fd2dc795f58c3ca5637c08e6af9bc7e880a207833ca" exitCode=0
Jul 03 14:11:31.987735 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.987349    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f49b424-126d-4ca8-8666-bf5ed7d46ae2","Type":"ContainerDied","Data":"72c55fd67f0614fb1567c4cbbcc2c23759e932804c4c27c1d42c910295bdec7f"}
Jul 03 14:11:31.987735 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.987427    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f49b424-126d-4ca8-8666-bf5ed7d46ae2","Type":"ContainerDied","Data":"30f0f0187a36b3c74dff3bd4e0cd140eb60b3c2dfb42f91cf2b0a1dae1c0acc4"}
Jul 03 14:11:31.987735 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.987484    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f49b424-126d-4ca8-8666-bf5ed7d46ae2","Type":"ContainerDied","Data":"d8b6d3c5529436789bcf1ea432259e0399127b780ce97f53ff3f32137aa71530"}
Jul 03 14:11:31.987735 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:31.987527    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f49b424-126d-4ca8-8666-bf5ed7d46ae2","Type":"ContainerDied","Data":"108ca6c14620a636508c2fd2dc795f58c3ca5637c08e6af9bc7e880a207833ca"}
Jul 03 14:11:32.027218 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.027184    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7cd5b956b9-r25ff"]
Jul 03 14:11:32.031408 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:11:32.031380    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d6eca64_8303_4ad4_ae80_197d0983d553.slice/crio-e0f9b7cd5b9a0757d60a5c28be57c5cefc25181da32c9423d7947afaeb3bc697 WatchSource:0}: Error finding container e0f9b7cd5b9a0757d60a5c28be57c5cefc25181da32c9423d7947afaeb3bc697: Status 404 returned error can't find the container with id e0f9b7cd5b9a0757d60a5c28be57c5cefc25181da32c9423d7947afaeb3bc697
Jul 03 14:11:32.107582 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.107558    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:32.193338 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.193252    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-metrics-client-certs\") pod \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") "
Jul 03 14:11:32.193338 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.193297    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-prometheus-k8s-tls\") pod \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") "
Jul 03 14:11:32.193536 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.193347    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-config\") pod \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") "
Jul 03 14:11:32.193536 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.193374    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffjs2\" (UniqueName: \"kubernetes.io/projected/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-kube-api-access-ffjs2\") pod \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") "
Jul 03 14:11:32.193536 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.193402    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-k8s-rulefiles-1\") pod \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") "
Jul 03 14:11:32.193536 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.193432    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-web-config\") pod \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") "
Jul 03 14:11:32.193536 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.193473    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-k8s-rulefiles-0\") pod \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") "
Jul 03 14:11:32.193536 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.193501    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-configmap-kubelet-serving-ca-bundle\") pod \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") "
Jul 03 14:11:32.193536 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.193527    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-kube-rbac-proxy\") pod \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") "
Jul 03 14:11:32.193871 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.193558    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-k8s-db\") pod \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") "
Jul 03 14:11:32.193871 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.193600    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-thanos-prometheus-http-client-file\") pod \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") "
Jul 03 14:11:32.193871 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.193644    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-trusted-ca-bundle\") pod \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") "
Jul 03 14:11:32.193871 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.193680    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-config-out\") pod \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") "
Jul 03 14:11:32.193871 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.193718    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") "
Jul 03 14:11:32.193871 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.193760    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-configmap-serving-certs-ca-bundle\") pod \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") "
Jul 03 14:11:32.193871 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.193792    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") "
Jul 03 14:11:32.193871 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.193822    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-tls-assets\") pod \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") "
Jul 03 14:11:32.193871 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.193824    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "5f49b424-126d-4ca8-8666-bf5ed7d46ae2" (UID: "5f49b424-126d-4ca8-8666-bf5ed7d46ae2"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:11:32.193871 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.193859    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-configmap-metrics-client-ca\") pod \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") "
Jul 03 14:11:32.194366 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.193885    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-grpc-tls\") pod \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") "
Jul 03 14:11:32.194366 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.194081    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-k8s-rulefiles-2\") pod \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\" (UID: \"5f49b424-126d-4ca8-8666-bf5ed7d46ae2\") "
Jul 03 14:11:32.194464 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.194440    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:32.194719 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.194692    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "5f49b424-126d-4ca8-8666-bf5ed7d46ae2" (UID: "5f49b424-126d-4ca8-8666-bf5ed7d46ae2"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:11:32.194799 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.194757    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "5f49b424-126d-4ca8-8666-bf5ed7d46ae2" (UID: "5f49b424-126d-4ca8-8666-bf5ed7d46ae2"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:11:32.194799 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.194766    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "5f49b424-126d-4ca8-8666-bf5ed7d46ae2" (UID: "5f49b424-126d-4ca8-8666-bf5ed7d46ae2"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:11:32.195294 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.195246    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "5f49b424-126d-4ca8-8666-bf5ed7d46ae2" (UID: "5f49b424-126d-4ca8-8666-bf5ed7d46ae2"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:11:32.196182 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.196153    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "5f49b424-126d-4ca8-8666-bf5ed7d46ae2" (UID: "5f49b424-126d-4ca8-8666-bf5ed7d46ae2"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:11:32.196344 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.196213    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "5f49b424-126d-4ca8-8666-bf5ed7d46ae2" (UID: "5f49b424-126d-4ca8-8666-bf5ed7d46ae2"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:11:32.197366 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.197306    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "5f49b424-126d-4ca8-8666-bf5ed7d46ae2" (UID: "5f49b424-126d-4ca8-8666-bf5ed7d46ae2"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 03 14:11:32.197738 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.197692    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "5f49b424-126d-4ca8-8666-bf5ed7d46ae2" (UID: "5f49b424-126d-4ca8-8666-bf5ed7d46ae2"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:11:32.197827 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.197745    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "5f49b424-126d-4ca8-8666-bf5ed7d46ae2" (UID: "5f49b424-126d-4ca8-8666-bf5ed7d46ae2"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:11:32.198213 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.198179    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-config" (OuterVolumeSpecName: "config") pod "5f49b424-126d-4ca8-8666-bf5ed7d46ae2" (UID: "5f49b424-126d-4ca8-8666-bf5ed7d46ae2"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:11:32.198432 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.198395    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "5f49b424-126d-4ca8-8666-bf5ed7d46ae2" (UID: "5f49b424-126d-4ca8-8666-bf5ed7d46ae2"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:11:32.198671 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.198645    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-kube-api-access-ffjs2" (OuterVolumeSpecName: "kube-api-access-ffjs2") pod "5f49b424-126d-4ca8-8666-bf5ed7d46ae2" (UID: "5f49b424-126d-4ca8-8666-bf5ed7d46ae2"). InnerVolumeSpecName "kube-api-access-ffjs2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:11:32.199248 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.199218    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "5f49b424-126d-4ca8-8666-bf5ed7d46ae2" (UID: "5f49b424-126d-4ca8-8666-bf5ed7d46ae2"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:11:32.199397 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.199377    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "5f49b424-126d-4ca8-8666-bf5ed7d46ae2" (UID: "5f49b424-126d-4ca8-8666-bf5ed7d46ae2"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:11:32.199463 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.199406    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "5f49b424-126d-4ca8-8666-bf5ed7d46ae2" (UID: "5f49b424-126d-4ca8-8666-bf5ed7d46ae2"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:11:32.199463 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.199438    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "5f49b424-126d-4ca8-8666-bf5ed7d46ae2" (UID: "5f49b424-126d-4ca8-8666-bf5ed7d46ae2"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:11:32.199534 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.199461    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "5f49b424-126d-4ca8-8666-bf5ed7d46ae2" (UID: "5f49b424-126d-4ca8-8666-bf5ed7d46ae2"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:11:32.199718 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.199689    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-config-out" (OuterVolumeSpecName: "config-out") pod "5f49b424-126d-4ca8-8666-bf5ed7d46ae2" (UID: "5f49b424-126d-4ca8-8666-bf5ed7d46ae2"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:11:32.209340 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.209318    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-web-config" (OuterVolumeSpecName: "web-config") pod "5f49b424-126d-4ca8-8666-bf5ed7d46ae2" (UID: "5f49b424-126d-4ca8-8666-bf5ed7d46ae2"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:11:32.295046 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.295011    2563 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-metrics-client-certs\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:32.295046 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.295039    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-prometheus-k8s-tls\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:32.295046 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.295049    2563 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-config\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:32.295262 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.295059    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffjs2\" (UniqueName: \"kubernetes.io/projected/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-kube-api-access-ffjs2\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:32.295262 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.295069    2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-web-config\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:32.295262 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.295079    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:32.295262 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.295088    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:32.295262 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.295097    2563 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-kube-rbac-proxy\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:32.295262 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.295106    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-k8s-db\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:32.295262 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.295114    2563 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-thanos-prometheus-http-client-file\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:32.295262 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.295123    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-trusted-ca-bundle\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:32.295262 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.295131    2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-config-out\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:32.295262 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.295139    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:32.295262 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.295148    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:32.295262 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.295156    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:32.295262 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.295165    2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-tls-assets\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:32.295262 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.295174    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-configmap-metrics-client-ca\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:32.295262 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.295182    2563 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-secret-grpc-tls\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:32.295262 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.295192    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5f49b424-126d-4ca8-8666-bf5ed7d46ae2-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:11:32.995541 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.995441    2563 generic.go:356] "Generic (PLEG): container finished" podID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerID="ad47a462a441c5f6ea4fe067960b8b0c340d91fb4c692045b25035b5080525fe" exitCode=0
Jul 03 14:11:32.995541 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.995473    2563 generic.go:356] "Generic (PLEG): container finished" podID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerID="2ecedefaf5adf1d2e20f57db254a8fcbfd85f4e81b1fa261f34ba62ac8447140" exitCode=0
Jul 03 14:11:32.995541 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.995523    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:32.995541 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.995532    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f49b424-126d-4ca8-8666-bf5ed7d46ae2","Type":"ContainerDied","Data":"ad47a462a441c5f6ea4fe067960b8b0c340d91fb4c692045b25035b5080525fe"}
Jul 03 14:11:32.996123 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.995628    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f49b424-126d-4ca8-8666-bf5ed7d46ae2","Type":"ContainerDied","Data":"2ecedefaf5adf1d2e20f57db254a8fcbfd85f4e81b1fa261f34ba62ac8447140"}
Jul 03 14:11:32.996123 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.995672    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5f49b424-126d-4ca8-8666-bf5ed7d46ae2","Type":"ContainerDied","Data":"d807ae2f778ec52c236413fc6844d8d824f417cc8062961a0032a012188daf49"}
Jul 03 14:11:32.996123 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.995703    2563 scope.go:117] "RemoveContainer" containerID="ad47a462a441c5f6ea4fe067960b8b0c340d91fb4c692045b25035b5080525fe"
Jul 03 14:11:32.997122 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:32.996857    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff" event={"ID":"9d6eca64-8303-4ad4-ae80-197d0983d553","Type":"ContainerStarted","Data":"e0f9b7cd5b9a0757d60a5c28be57c5cefc25181da32c9423d7947afaeb3bc697"}
Jul 03 14:11:33.004168 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.004070    2563 scope.go:117] "RemoveContainer" containerID="72c55fd67f0614fb1567c4cbbcc2c23759e932804c4c27c1d42c910295bdec7f"
Jul 03 14:11:33.013469 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.013450    2563 scope.go:117] "RemoveContainer" containerID="30f0f0187a36b3c74dff3bd4e0cd140eb60b3c2dfb42f91cf2b0a1dae1c0acc4"
Jul 03 14:11:33.020103 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.020086    2563 scope.go:117] "RemoveContainer" containerID="d8b6d3c5529436789bcf1ea432259e0399127b780ce97f53ff3f32137aa71530"
Jul 03 14:11:33.026184 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.026159    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 03 14:11:33.027734 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.027719    2563 scope.go:117] "RemoveContainer" containerID="2ecedefaf5adf1d2e20f57db254a8fcbfd85f4e81b1fa261f34ba62ac8447140"
Jul 03 14:11:33.036933 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.036887    2563 scope.go:117] "RemoveContainer" containerID="108ca6c14620a636508c2fd2dc795f58c3ca5637c08e6af9bc7e880a207833ca"
Jul 03 14:11:33.037613 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.037591    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 03 14:11:33.045529 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.045507    2563 scope.go:117] "RemoveContainer" containerID="bd1840408b657d161019a191865790af9c9e8aca0160b16a4b130751c958080d"
Jul 03 14:11:33.053673 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.053652    2563 scope.go:117] "RemoveContainer" containerID="ad47a462a441c5f6ea4fe067960b8b0c340d91fb4c692045b25035b5080525fe"
Jul 03 14:11:33.054020 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:11:33.053996    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad47a462a441c5f6ea4fe067960b8b0c340d91fb4c692045b25035b5080525fe\": container with ID starting with ad47a462a441c5f6ea4fe067960b8b0c340d91fb4c692045b25035b5080525fe not found: ID does not exist" containerID="ad47a462a441c5f6ea4fe067960b8b0c340d91fb4c692045b25035b5080525fe"
Jul 03 14:11:33.054123 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.054028    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad47a462a441c5f6ea4fe067960b8b0c340d91fb4c692045b25035b5080525fe"} err="failed to get container status \"ad47a462a441c5f6ea4fe067960b8b0c340d91fb4c692045b25035b5080525fe\": rpc error: code = NotFound desc = could not find container \"ad47a462a441c5f6ea4fe067960b8b0c340d91fb4c692045b25035b5080525fe\": container with ID starting with ad47a462a441c5f6ea4fe067960b8b0c340d91fb4c692045b25035b5080525fe not found: ID does not exist"
Jul 03 14:11:33.054123 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.054047    2563 scope.go:117] "RemoveContainer" containerID="72c55fd67f0614fb1567c4cbbcc2c23759e932804c4c27c1d42c910295bdec7f"
Jul 03 14:11:33.054386 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:11:33.054337    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72c55fd67f0614fb1567c4cbbcc2c23759e932804c4c27c1d42c910295bdec7f\": container with ID starting with 72c55fd67f0614fb1567c4cbbcc2c23759e932804c4c27c1d42c910295bdec7f not found: ID does not exist" containerID="72c55fd67f0614fb1567c4cbbcc2c23759e932804c4c27c1d42c910295bdec7f"
Jul 03 14:11:33.054475 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.054396    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72c55fd67f0614fb1567c4cbbcc2c23759e932804c4c27c1d42c910295bdec7f"} err="failed to get container status \"72c55fd67f0614fb1567c4cbbcc2c23759e932804c4c27c1d42c910295bdec7f\": rpc error: code = NotFound desc = could not find container \"72c55fd67f0614fb1567c4cbbcc2c23759e932804c4c27c1d42c910295bdec7f\": container with ID starting with 72c55fd67f0614fb1567c4cbbcc2c23759e932804c4c27c1d42c910295bdec7f not found: ID does not exist"
Jul 03 14:11:33.054475 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.054420    2563 scope.go:117] "RemoveContainer" containerID="30f0f0187a36b3c74dff3bd4e0cd140eb60b3c2dfb42f91cf2b0a1dae1c0acc4"
Jul 03 14:11:33.054728 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:11:33.054702    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30f0f0187a36b3c74dff3bd4e0cd140eb60b3c2dfb42f91cf2b0a1dae1c0acc4\": container with ID starting with 30f0f0187a36b3c74dff3bd4e0cd140eb60b3c2dfb42f91cf2b0a1dae1c0acc4 not found: ID does not exist" containerID="30f0f0187a36b3c74dff3bd4e0cd140eb60b3c2dfb42f91cf2b0a1dae1c0acc4"
Jul 03 14:11:33.054837 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.054735    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30f0f0187a36b3c74dff3bd4e0cd140eb60b3c2dfb42f91cf2b0a1dae1c0acc4"} err="failed to get container status \"30f0f0187a36b3c74dff3bd4e0cd140eb60b3c2dfb42f91cf2b0a1dae1c0acc4\": rpc error: code = NotFound desc = could not find container \"30f0f0187a36b3c74dff3bd4e0cd140eb60b3c2dfb42f91cf2b0a1dae1c0acc4\": container with ID starting with 30f0f0187a36b3c74dff3bd4e0cd140eb60b3c2dfb42f91cf2b0a1dae1c0acc4 not found: ID does not exist"
Jul 03 14:11:33.054837 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.054758    2563 scope.go:117] "RemoveContainer" containerID="d8b6d3c5529436789bcf1ea432259e0399127b780ce97f53ff3f32137aa71530"
Jul 03 14:11:33.055165 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:11:33.055139    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8b6d3c5529436789bcf1ea432259e0399127b780ce97f53ff3f32137aa71530\": container with ID starting with d8b6d3c5529436789bcf1ea432259e0399127b780ce97f53ff3f32137aa71530 not found: ID does not exist" containerID="d8b6d3c5529436789bcf1ea432259e0399127b780ce97f53ff3f32137aa71530"
Jul 03 14:11:33.055254 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.055170    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8b6d3c5529436789bcf1ea432259e0399127b780ce97f53ff3f32137aa71530"} err="failed to get container status \"d8b6d3c5529436789bcf1ea432259e0399127b780ce97f53ff3f32137aa71530\": rpc error: code = NotFound desc = could not find container \"d8b6d3c5529436789bcf1ea432259e0399127b780ce97f53ff3f32137aa71530\": container with ID starting with d8b6d3c5529436789bcf1ea432259e0399127b780ce97f53ff3f32137aa71530 not found: ID does not exist"
Jul 03 14:11:33.055254 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.055189    2563 scope.go:117] "RemoveContainer" containerID="2ecedefaf5adf1d2e20f57db254a8fcbfd85f4e81b1fa261f34ba62ac8447140"
Jul 03 14:11:33.055454 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:11:33.055433    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ecedefaf5adf1d2e20f57db254a8fcbfd85f4e81b1fa261f34ba62ac8447140\": container with ID starting with 2ecedefaf5adf1d2e20f57db254a8fcbfd85f4e81b1fa261f34ba62ac8447140 not found: ID does not exist" containerID="2ecedefaf5adf1d2e20f57db254a8fcbfd85f4e81b1fa261f34ba62ac8447140"
Jul 03 14:11:33.055535 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.055458    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ecedefaf5adf1d2e20f57db254a8fcbfd85f4e81b1fa261f34ba62ac8447140"} err="failed to get container status \"2ecedefaf5adf1d2e20f57db254a8fcbfd85f4e81b1fa261f34ba62ac8447140\": rpc error: code = NotFound desc = could not find container \"2ecedefaf5adf1d2e20f57db254a8fcbfd85f4e81b1fa261f34ba62ac8447140\": container with ID starting with 2ecedefaf5adf1d2e20f57db254a8fcbfd85f4e81b1fa261f34ba62ac8447140 not found: ID does not exist"
Jul 03 14:11:33.055535 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.055476    2563 scope.go:117] "RemoveContainer" containerID="108ca6c14620a636508c2fd2dc795f58c3ca5637c08e6af9bc7e880a207833ca"
Jul 03 14:11:33.055763 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:11:33.055736    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"108ca6c14620a636508c2fd2dc795f58c3ca5637c08e6af9bc7e880a207833ca\": container with ID starting with 108ca6c14620a636508c2fd2dc795f58c3ca5637c08e6af9bc7e880a207833ca not found: ID does not exist" containerID="108ca6c14620a636508c2fd2dc795f58c3ca5637c08e6af9bc7e880a207833ca"
Jul 03 14:11:33.055853 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.055767    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"108ca6c14620a636508c2fd2dc795f58c3ca5637c08e6af9bc7e880a207833ca"} err="failed to get container status \"108ca6c14620a636508c2fd2dc795f58c3ca5637c08e6af9bc7e880a207833ca\": rpc error: code = NotFound desc = could not find container \"108ca6c14620a636508c2fd2dc795f58c3ca5637c08e6af9bc7e880a207833ca\": container with ID starting with 108ca6c14620a636508c2fd2dc795f58c3ca5637c08e6af9bc7e880a207833ca not found: ID does not exist"
Jul 03 14:11:33.055853 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.055787    2563 scope.go:117] "RemoveContainer" containerID="bd1840408b657d161019a191865790af9c9e8aca0160b16a4b130751c958080d"
Jul 03 14:11:33.056120 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:11:33.056101    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd1840408b657d161019a191865790af9c9e8aca0160b16a4b130751c958080d\": container with ID starting with bd1840408b657d161019a191865790af9c9e8aca0160b16a4b130751c958080d not found: ID does not exist" containerID="bd1840408b657d161019a191865790af9c9e8aca0160b16a4b130751c958080d"
Jul 03 14:11:33.056217 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.056129    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd1840408b657d161019a191865790af9c9e8aca0160b16a4b130751c958080d"} err="failed to get container status \"bd1840408b657d161019a191865790af9c9e8aca0160b16a4b130751c958080d\": rpc error: code = NotFound desc = could not find container \"bd1840408b657d161019a191865790af9c9e8aca0160b16a4b130751c958080d\": container with ID starting with bd1840408b657d161019a191865790af9c9e8aca0160b16a4b130751c958080d not found: ID does not exist"
Jul 03 14:11:33.056217 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.056148    2563 scope.go:117] "RemoveContainer" containerID="ad47a462a441c5f6ea4fe067960b8b0c340d91fb4c692045b25035b5080525fe"
Jul 03 14:11:33.056518 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.056454    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad47a462a441c5f6ea4fe067960b8b0c340d91fb4c692045b25035b5080525fe"} err="failed to get container status \"ad47a462a441c5f6ea4fe067960b8b0c340d91fb4c692045b25035b5080525fe\": rpc error: code = NotFound desc = could not find container \"ad47a462a441c5f6ea4fe067960b8b0c340d91fb4c692045b25035b5080525fe\": container with ID starting with ad47a462a441c5f6ea4fe067960b8b0c340d91fb4c692045b25035b5080525fe not found: ID does not exist"
Jul 03 14:11:33.056518 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.056485    2563 scope.go:117] "RemoveContainer" containerID="72c55fd67f0614fb1567c4cbbcc2c23759e932804c4c27c1d42c910295bdec7f"
Jul 03 14:11:33.056781 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.056758    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72c55fd67f0614fb1567c4cbbcc2c23759e932804c4c27c1d42c910295bdec7f"} err="failed to get container status \"72c55fd67f0614fb1567c4cbbcc2c23759e932804c4c27c1d42c910295bdec7f\": rpc error: code = NotFound desc = could not find container \"72c55fd67f0614fb1567c4cbbcc2c23759e932804c4c27c1d42c910295bdec7f\": container with ID starting with 72c55fd67f0614fb1567c4cbbcc2c23759e932804c4c27c1d42c910295bdec7f not found: ID does not exist"
Jul 03 14:11:33.056862 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.056783    2563 scope.go:117] "RemoveContainer" containerID="30f0f0187a36b3c74dff3bd4e0cd140eb60b3c2dfb42f91cf2b0a1dae1c0acc4"
Jul 03 14:11:33.057139 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.057109    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30f0f0187a36b3c74dff3bd4e0cd140eb60b3c2dfb42f91cf2b0a1dae1c0acc4"} err="failed to get container status \"30f0f0187a36b3c74dff3bd4e0cd140eb60b3c2dfb42f91cf2b0a1dae1c0acc4\": rpc error: code = NotFound desc = could not find container \"30f0f0187a36b3c74dff3bd4e0cd140eb60b3c2dfb42f91cf2b0a1dae1c0acc4\": container with ID starting with 30f0f0187a36b3c74dff3bd4e0cd140eb60b3c2dfb42f91cf2b0a1dae1c0acc4 not found: ID does not exist"
Jul 03 14:11:33.057139 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.057139    2563 scope.go:117] "RemoveContainer" containerID="d8b6d3c5529436789bcf1ea432259e0399127b780ce97f53ff3f32137aa71530"
Jul 03 14:11:33.057406 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.057378    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8b6d3c5529436789bcf1ea432259e0399127b780ce97f53ff3f32137aa71530"} err="failed to get container status \"d8b6d3c5529436789bcf1ea432259e0399127b780ce97f53ff3f32137aa71530\": rpc error: code = NotFound desc = could not find container \"d8b6d3c5529436789bcf1ea432259e0399127b780ce97f53ff3f32137aa71530\": container with ID starting with d8b6d3c5529436789bcf1ea432259e0399127b780ce97f53ff3f32137aa71530 not found: ID does not exist"
Jul 03 14:11:33.057406 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.057406    2563 scope.go:117] "RemoveContainer" containerID="2ecedefaf5adf1d2e20f57db254a8fcbfd85f4e81b1fa261f34ba62ac8447140"
Jul 03 14:11:33.057697 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.057662    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ecedefaf5adf1d2e20f57db254a8fcbfd85f4e81b1fa261f34ba62ac8447140"} err="failed to get container status \"2ecedefaf5adf1d2e20f57db254a8fcbfd85f4e81b1fa261f34ba62ac8447140\": rpc error: code = NotFound desc = could not find container \"2ecedefaf5adf1d2e20f57db254a8fcbfd85f4e81b1fa261f34ba62ac8447140\": container with ID starting with 2ecedefaf5adf1d2e20f57db254a8fcbfd85f4e81b1fa261f34ba62ac8447140 not found: ID does not exist"
Jul 03 14:11:33.057697 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.057683    2563 scope.go:117] "RemoveContainer" containerID="108ca6c14620a636508c2fd2dc795f58c3ca5637c08e6af9bc7e880a207833ca"
Jul 03 14:11:33.058030 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.058009    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"108ca6c14620a636508c2fd2dc795f58c3ca5637c08e6af9bc7e880a207833ca"} err="failed to get container status \"108ca6c14620a636508c2fd2dc795f58c3ca5637c08e6af9bc7e880a207833ca\": rpc error: code = NotFound desc = could not find container \"108ca6c14620a636508c2fd2dc795f58c3ca5637c08e6af9bc7e880a207833ca\": container with ID starting with 108ca6c14620a636508c2fd2dc795f58c3ca5637c08e6af9bc7e880a207833ca not found: ID does not exist"
Jul 03 14:11:33.058030 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.058030    2563 scope.go:117] "RemoveContainer" containerID="bd1840408b657d161019a191865790af9c9e8aca0160b16a4b130751c958080d"
Jul 03 14:11:33.058274 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.058248    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd1840408b657d161019a191865790af9c9e8aca0160b16a4b130751c958080d"} err="failed to get container status \"bd1840408b657d161019a191865790af9c9e8aca0160b16a4b130751c958080d\": rpc error: code = NotFound desc = could not find container \"bd1840408b657d161019a191865790af9c9e8aca0160b16a4b130751c958080d\": container with ID starting with bd1840408b657d161019a191865790af9c9e8aca0160b16a4b130751c958080d not found: ID does not exist"
Jul 03 14:11:33.061212 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.061188    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" path="/var/lib/kubelet/pods/5f49b424-126d-4ca8-8666-bf5ed7d46ae2/volumes"
Jul 03 14:11:33.115187 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.115154    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 03 14:11:33.115641 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.115624    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerName="kube-rbac-proxy"
Jul 03 14:11:33.115704 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.115644    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerName="kube-rbac-proxy"
Jul 03 14:11:33.115704 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.115658    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerName="prometheus"
Jul 03 14:11:33.115704 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.115667    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerName="prometheus"
Jul 03 14:11:33.115704 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.115679    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerName="thanos-sidecar"
Jul 03 14:11:33.115704 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.115687    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerName="thanos-sidecar"
Jul 03 14:11:33.115884 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.115712    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerName="init-config-reloader"
Jul 03 14:11:33.115884 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.115721    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerName="init-config-reloader"
Jul 03 14:11:33.115884 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.115734    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerName="kube-rbac-proxy-web"
Jul 03 14:11:33.115884 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.115742    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerName="kube-rbac-proxy-web"
Jul 03 14:11:33.115884 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.115753    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerName="kube-rbac-proxy-thanos"
Jul 03 14:11:33.115884 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.115762    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerName="kube-rbac-proxy-thanos"
Jul 03 14:11:33.115884 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.115779    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerName="config-reloader"
Jul 03 14:11:33.115884 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.115787    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerName="config-reloader"
Jul 03 14:11:33.115884 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.115856    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerName="config-reloader"
Jul 03 14:11:33.115884 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.115867    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerName="kube-rbac-proxy-web"
Jul 03 14:11:33.115884 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.115881    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerName="thanos-sidecar"
Jul 03 14:11:33.116387 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.115892    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerName="kube-rbac-proxy-thanos"
Jul 03 14:11:33.116387 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.115900    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerName="prometheus"
Jul 03 14:11:33.116387 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.115911    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f49b424-126d-4ca8-8666-bf5ed7d46ae2" containerName="kube-rbac-proxy"
Jul 03 14:11:33.120046 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.120021    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.125126 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.125104    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 03 14:11:33.125243 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.125107    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 03 14:11:33.125412 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.125391    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-53sdi07rac24a\""
Jul 03 14:11:33.125543 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.125524    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 03 14:11:33.125640 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.125623    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-nz67c\""
Jul 03 14:11:33.126253 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.126227    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 03 14:11:33.126353 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.126318    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 03 14:11:33.126410 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.126385    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 03 14:11:33.126492 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.126463    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 03 14:11:33.126582 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.126475    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 03 14:11:33.126654 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.126639    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 03 14:11:33.126716 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.126652    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 03 14:11:33.127743 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.127724    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 03 14:11:33.132653 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.132632    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 03 14:11:33.133354 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.133338    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 03 14:11:33.134186 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.134168    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 03 14:11:33.167551 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.167525    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 03 14:11:33.202627 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.202587    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8b71262a-e44f-4bba-accf-896149be23ae-config\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.202627 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.202624    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b71262a-e44f-4bba-accf-896149be23ae-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.202851 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.202648    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b71262a-e44f-4bba-accf-896149be23ae-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.202851 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.202722    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8b71262a-e44f-4bba-accf-896149be23ae-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.202851 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.202748    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8b71262a-e44f-4bba-accf-896149be23ae-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.202851 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.202772    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8b71262a-e44f-4bba-accf-896149be23ae-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.202851 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.202817    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b71262a-e44f-4bba-accf-896149be23ae-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.203255 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.202867    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b71262a-e44f-4bba-accf-896149be23ae-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.203255 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.202909    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8b71262a-e44f-4bba-accf-896149be23ae-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.203255 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.202934    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8b71262a-e44f-4bba-accf-896149be23ae-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.203255 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.202989    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8b71262a-e44f-4bba-accf-896149be23ae-web-config\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.203255 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.203020    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b71262a-e44f-4bba-accf-896149be23ae-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.203255 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.203047    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r62t\" (UniqueName: \"kubernetes.io/projected/8b71262a-e44f-4bba-accf-896149be23ae-kube-api-access-6r62t\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.203255 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.203073    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8b71262a-e44f-4bba-accf-896149be23ae-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.203255 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.203123    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b71262a-e44f-4bba-accf-896149be23ae-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.203255 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.203168    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8b71262a-e44f-4bba-accf-896149be23ae-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.203255 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.203191    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8b71262a-e44f-4bba-accf-896149be23ae-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.203255 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.203236    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8b71262a-e44f-4bba-accf-896149be23ae-config-out\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.203695 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.203291    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8b71262a-e44f-4bba-accf-896149be23ae-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.203695 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.203320    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8b71262a-e44f-4bba-accf-896149be23ae-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.304062 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.303957    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8b71262a-e44f-4bba-accf-896149be23ae-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.304062 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.304013    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8b71262a-e44f-4bba-accf-896149be23ae-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.304258 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.304131    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8b71262a-e44f-4bba-accf-896149be23ae-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.304258 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.304174    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b71262a-e44f-4bba-accf-896149be23ae-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.304258 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.304210    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b71262a-e44f-4bba-accf-896149be23ae-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.304258 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.304255    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8b71262a-e44f-4bba-accf-896149be23ae-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.304451 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.304287    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8b71262a-e44f-4bba-accf-896149be23ae-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.304451 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.304323    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8b71262a-e44f-4bba-accf-896149be23ae-web-config\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.304451 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.304356    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b71262a-e44f-4bba-accf-896149be23ae-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.304451 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.304391    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r62t\" (UniqueName: \"kubernetes.io/projected/8b71262a-e44f-4bba-accf-896149be23ae-kube-api-access-6r62t\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.304451 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.304415    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8b71262a-e44f-4bba-accf-896149be23ae-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.304678 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.304450    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b71262a-e44f-4bba-accf-896149be23ae-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.304678 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.304486    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8b71262a-e44f-4bba-accf-896149be23ae-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.304678 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.304509    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8b71262a-e44f-4bba-accf-896149be23ae-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.304678 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.304559    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8b71262a-e44f-4bba-accf-896149be23ae-config-out\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.304678 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.304606    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8b71262a-e44f-4bba-accf-896149be23ae-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.304678 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.304639    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8b71262a-e44f-4bba-accf-896149be23ae-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.304678 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.304670    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8b71262a-e44f-4bba-accf-896149be23ae-config\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.305157 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.304701    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b71262a-e44f-4bba-accf-896149be23ae-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.305157 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.304726    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b71262a-e44f-4bba-accf-896149be23ae-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.305157 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.304869    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8b71262a-e44f-4bba-accf-896149be23ae-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.305157 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.305002    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b71262a-e44f-4bba-accf-896149be23ae-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.305715 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.305672    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b71262a-e44f-4bba-accf-896149be23ae-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.306410 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.306334    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b71262a-e44f-4bba-accf-896149be23ae-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.307269 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.307085    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b71262a-e44f-4bba-accf-896149be23ae-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.307269 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.307170    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8b71262a-e44f-4bba-accf-896149be23ae-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.307269 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.307230    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8b71262a-e44f-4bba-accf-896149be23ae-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.307486 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.307346    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8b71262a-e44f-4bba-accf-896149be23ae-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.307577 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.307556    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8b71262a-e44f-4bba-accf-896149be23ae-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.307880 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.307752    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8b71262a-e44f-4bba-accf-896149be23ae-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.307880 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.307771    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8b71262a-e44f-4bba-accf-896149be23ae-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.308395 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.308370    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b71262a-e44f-4bba-accf-896149be23ae-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.308783 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.308747    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8b71262a-e44f-4bba-accf-896149be23ae-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.309303 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.309279    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8b71262a-e44f-4bba-accf-896149be23ae-config\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.309827 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.309803    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b71262a-e44f-4bba-accf-896149be23ae-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.310040 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.310009    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8b71262a-e44f-4bba-accf-896149be23ae-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.310124 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.310040    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8b71262a-e44f-4bba-accf-896149be23ae-config-out\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.310307 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.310289    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8b71262a-e44f-4bba-accf-896149be23ae-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.310440 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.310418    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8b71262a-e44f-4bba-accf-896149be23ae-web-config\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.322176 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.322154    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r62t\" (UniqueName: \"kubernetes.io/projected/8b71262a-e44f-4bba-accf-896149be23ae-kube-api-access-6r62t\") pod \"prometheus-k8s-0\" (UID: \"8b71262a-e44f-4bba-accf-896149be23ae\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.432235 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.432202    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:11:33.594140 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:33.594114    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 03 14:11:34.035074 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:11:34.035037    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b71262a_e44f_4bba_accf_896149be23ae.slice/crio-79242e7b6a2996bba2882cb0879c75b926c79ebae1916c1888a2d9b874695717 WatchSource:0}: Error finding container 79242e7b6a2996bba2882cb0879c75b926c79ebae1916c1888a2d9b874695717: Status 404 returned error can't find the container with id 79242e7b6a2996bba2882cb0879c75b926c79ebae1916c1888a2d9b874695717
Jul 03 14:11:35.009140 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:35.009053    2563 generic.go:356] "Generic (PLEG): container finished" podID="8b71262a-e44f-4bba-accf-896149be23ae" containerID="3ecd3a12b32f1e5f0ae989128c01b1284851c89de4616a69ac9466c803882f22" exitCode=0
Jul 03 14:11:35.009274 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:35.009142    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b71262a-e44f-4bba-accf-896149be23ae","Type":"ContainerDied","Data":"3ecd3a12b32f1e5f0ae989128c01b1284851c89de4616a69ac9466c803882f22"}
Jul 03 14:11:35.009274 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:35.009241    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b71262a-e44f-4bba-accf-896149be23ae","Type":"ContainerStarted","Data":"79242e7b6a2996bba2882cb0879c75b926c79ebae1916c1888a2d9b874695717"}
Jul 03 14:11:35.011233 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:35.011210    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff" event={"ID":"9d6eca64-8303-4ad4-ae80-197d0983d553","Type":"ContainerStarted","Data":"e5f682ca8f7e2dca8346718de634f458344a5f732f2c1088f32c65a900dc683a"}
Jul 03 14:11:35.011308 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:35.011260    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff" event={"ID":"9d6eca64-8303-4ad4-ae80-197d0983d553","Type":"ContainerStarted","Data":"5016ac6381f194c743c7a32fc8f8da9e9ed069a7e05777efae0c1a89a13f9f3c"}
Jul 03 14:11:35.011308 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:35.011284    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff" event={"ID":"9d6eca64-8303-4ad4-ae80-197d0983d553","Type":"ContainerStarted","Data":"67397d881ecb2aa54fd00458e7f84eaf6a2f89009817ceb911aaeca47fef30eb"}
Jul 03 14:11:35.073657 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:35.073612    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7cd5b956b9-r25ff" podStartSLOduration=2.042098908 podStartE2EDuration="4.073595339s" totalImagesPullingTime="2.031496431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:11:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:11:35.071656086 +0000 UTC m=+208.521789562" watchObservedRunningTime="2026-07-03 14:11:35.073595339 +0000 UTC m=+208.523728779"
Jul 03 14:11:36.018994 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:36.018939    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b71262a-e44f-4bba-accf-896149be23ae","Type":"ContainerStarted","Data":"79ec26059f44d9cddc9952b02138ec11aa4b142350b5f323c0b52267c8e0587f"}
Jul 03 14:11:36.019175 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:36.019017    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b71262a-e44f-4bba-accf-896149be23ae","Type":"ContainerStarted","Data":"9dc537e056aa93a03af3b5a3f3d7adb3e0e26545303da019bc0ca102020de6f1"}
Jul 03 14:11:36.019175 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:36.019043    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b71262a-e44f-4bba-accf-896149be23ae","Type":"ContainerStarted","Data":"50fb1ac03ef8004b8f2bce4eb010d874609671145d519fe50a721f06e31697e6"}
Jul 03 14:11:36.019175 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:36.019067    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b71262a-e44f-4bba-accf-896149be23ae","Type":"ContainerStarted","Data":"3eb86c04652518882361b02d2c5bd78c06238c61f8bb927741edcb5f7e4e6e15"}
Jul 03 14:11:36.019175 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:36.019090    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b71262a-e44f-4bba-accf-896149be23ae","Type":"ContainerStarted","Data":"cf8e3d613427d8b7c50df6efd97f005aebb7067dd7ffe334e317e136dcab6f47"}
Jul 03 14:11:36.019175 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:36.019120    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b71262a-e44f-4bba-accf-896149be23ae","Type":"ContainerStarted","Data":"b7ce26d38fc27c449e2d5470a04ad1923b24a92a959abc895ca2ae90c1db1a12"}
Jul 03 14:11:36.063928 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:36.063875    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.045809524 podStartE2EDuration="3.063858722s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.049198ms" isStatefulPod=false podCreationTimestamp="2026-07-03 14:11:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:11:36.061823175 +0000 UTC m=+209.511956616" watchObservedRunningTime="2026-07-03 14:11:36.063858722 +0000 UTC m=+209.513992161"
Jul 03 14:11:38.433046 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:11:38.433007    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:12:33.433002 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:12:33.432896    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:12:33.448730 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:12:33.448705    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:12:34.283455 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:12:34.283428    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 03 14:13:06.977852 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:13:06.977824    2563 kubelet.go:1648] "Image garbage collection succeeded"
Jul 03 14:15:18.632419 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:18.632385    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-78cbbf6689-pgfl6"]
Jul 03 14:15:18.635562 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:18.635545    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78cbbf6689-pgfl6"
Jul 03 14:15:18.639385 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:18.639363    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 03 14:15:18.640216 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:18.640187    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 03 14:15:18.640296 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:18.640188    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 03 14:15:18.640346 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:18.640204    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 03 14:15:18.640584 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:18.640567    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-w2zrb\""
Jul 03 14:15:18.640651 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:18.640589    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 03 14:15:18.653060 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:18.647961    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78cbbf6689-pgfl6"]
Jul 03 14:15:18.731641 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:18.731597    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6fc5b6b3-ed32-44fd-8d81-1bc1c035e90d-cert\") pod \"lws-controller-manager-78cbbf6689-pgfl6\" (UID: \"6fc5b6b3-ed32-44fd-8d81-1bc1c035e90d\") " pod="openshift-lws-operator/lws-controller-manager-78cbbf6689-pgfl6"
Jul 03 14:15:18.731818 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:18.731667    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6fc5b6b3-ed32-44fd-8d81-1bc1c035e90d-manager-config\") pod \"lws-controller-manager-78cbbf6689-pgfl6\" (UID: \"6fc5b6b3-ed32-44fd-8d81-1bc1c035e90d\") " pod="openshift-lws-operator/lws-controller-manager-78cbbf6689-pgfl6"
Jul 03 14:15:18.731818 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:18.731697    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw2lb\" (UniqueName: \"kubernetes.io/projected/6fc5b6b3-ed32-44fd-8d81-1bc1c035e90d-kube-api-access-xw2lb\") pod \"lws-controller-manager-78cbbf6689-pgfl6\" (UID: \"6fc5b6b3-ed32-44fd-8d81-1bc1c035e90d\") " pod="openshift-lws-operator/lws-controller-manager-78cbbf6689-pgfl6"
Jul 03 14:15:18.731818 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:18.731740    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6fc5b6b3-ed32-44fd-8d81-1bc1c035e90d-metrics-cert\") pod \"lws-controller-manager-78cbbf6689-pgfl6\" (UID: \"6fc5b6b3-ed32-44fd-8d81-1bc1c035e90d\") " pod="openshift-lws-operator/lws-controller-manager-78cbbf6689-pgfl6"
Jul 03 14:15:18.832893 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:18.832847    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6fc5b6b3-ed32-44fd-8d81-1bc1c035e90d-manager-config\") pod \"lws-controller-manager-78cbbf6689-pgfl6\" (UID: \"6fc5b6b3-ed32-44fd-8d81-1bc1c035e90d\") " pod="openshift-lws-operator/lws-controller-manager-78cbbf6689-pgfl6"
Jul 03 14:15:18.832893 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:18.832895    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw2lb\" (UniqueName: \"kubernetes.io/projected/6fc5b6b3-ed32-44fd-8d81-1bc1c035e90d-kube-api-access-xw2lb\") pod \"lws-controller-manager-78cbbf6689-pgfl6\" (UID: \"6fc5b6b3-ed32-44fd-8d81-1bc1c035e90d\") " pod="openshift-lws-operator/lws-controller-manager-78cbbf6689-pgfl6"
Jul 03 14:15:18.833188 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:18.832931    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6fc5b6b3-ed32-44fd-8d81-1bc1c035e90d-metrics-cert\") pod \"lws-controller-manager-78cbbf6689-pgfl6\" (UID: \"6fc5b6b3-ed32-44fd-8d81-1bc1c035e90d\") " pod="openshift-lws-operator/lws-controller-manager-78cbbf6689-pgfl6"
Jul 03 14:15:18.833188 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:18.833019    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6fc5b6b3-ed32-44fd-8d81-1bc1c035e90d-cert\") pod \"lws-controller-manager-78cbbf6689-pgfl6\" (UID: \"6fc5b6b3-ed32-44fd-8d81-1bc1c035e90d\") " pod="openshift-lws-operator/lws-controller-manager-78cbbf6689-pgfl6"
Jul 03 14:15:18.833571 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:18.833530    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6fc5b6b3-ed32-44fd-8d81-1bc1c035e90d-manager-config\") pod \"lws-controller-manager-78cbbf6689-pgfl6\" (UID: \"6fc5b6b3-ed32-44fd-8d81-1bc1c035e90d\") " pod="openshift-lws-operator/lws-controller-manager-78cbbf6689-pgfl6"
Jul 03 14:15:18.835498 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:18.835469    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6fc5b6b3-ed32-44fd-8d81-1bc1c035e90d-cert\") pod \"lws-controller-manager-78cbbf6689-pgfl6\" (UID: \"6fc5b6b3-ed32-44fd-8d81-1bc1c035e90d\") " pod="openshift-lws-operator/lws-controller-manager-78cbbf6689-pgfl6"
Jul 03 14:15:18.835609 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:18.835502    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6fc5b6b3-ed32-44fd-8d81-1bc1c035e90d-metrics-cert\") pod \"lws-controller-manager-78cbbf6689-pgfl6\" (UID: \"6fc5b6b3-ed32-44fd-8d81-1bc1c035e90d\") " pod="openshift-lws-operator/lws-controller-manager-78cbbf6689-pgfl6"
Jul 03 14:15:18.846137 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:18.846111    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw2lb\" (UniqueName: \"kubernetes.io/projected/6fc5b6b3-ed32-44fd-8d81-1bc1c035e90d-kube-api-access-xw2lb\") pod \"lws-controller-manager-78cbbf6689-pgfl6\" (UID: \"6fc5b6b3-ed32-44fd-8d81-1bc1c035e90d\") " pod="openshift-lws-operator/lws-controller-manager-78cbbf6689-pgfl6"
Jul 03 14:15:18.945376 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:18.945295    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78cbbf6689-pgfl6"
Jul 03 14:15:19.079051 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:19.079022    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78cbbf6689-pgfl6"]
Jul 03 14:15:19.081622 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:15:19.081596    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fc5b6b3_ed32_44fd_8d81_1bc1c035e90d.slice/crio-121f1ba7f1dc4a41b004da47697985c803fdf7b589f59932e434cc24d7b29b5b WatchSource:0}: Error finding container 121f1ba7f1dc4a41b004da47697985c803fdf7b589f59932e434cc24d7b29b5b: Status 404 returned error can't find the container with id 121f1ba7f1dc4a41b004da47697985c803fdf7b589f59932e434cc24d7b29b5b
Jul 03 14:15:19.083506 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:19.083489    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 03 14:15:19.945460 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:19.945420    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78cbbf6689-pgfl6" event={"ID":"6fc5b6b3-ed32-44fd-8d81-1bc1c035e90d","Type":"ContainerStarted","Data":"121f1ba7f1dc4a41b004da47697985c803fdf7b589f59932e434cc24d7b29b5b"}
Jul 03 14:15:21.956246 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:21.956211    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78cbbf6689-pgfl6" event={"ID":"6fc5b6b3-ed32-44fd-8d81-1bc1c035e90d","Type":"ContainerStarted","Data":"2ffbfbc8492c71926aeea0659ce304d0253d0ccf70d134f23f21fd85a11dadd6"}
Jul 03 14:15:21.956630 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:21.956271    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-78cbbf6689-pgfl6"
Jul 03 14:15:21.979081 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:21.979031    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-78cbbf6689-pgfl6" podStartSLOduration=1.924070157 podStartE2EDuration="3.979013544s" totalImagesPullingTime="2.054943387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:15:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:15:21.977741911 +0000 UTC m=+435.427875351" watchObservedRunningTime="2026-07-03 14:15:21.979013544 +0000 UTC m=+435.429146984"
Jul 03 14:15:28.317074 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:28.317047    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-977f7bbf6-j5w8d"]
Jul 03 14:15:28.320652 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:28.320636    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-j5w8d"
Jul 03 14:15:28.324642 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:28.324617    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 03 14:15:28.324768 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:28.324651    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 03 14:15:28.324768 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:28.324617    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 03 14:15:28.324905 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:28.324888    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 03 14:15:28.325148 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:28.325135    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-xwgf7\""
Jul 03 14:15:28.346196 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:28.346170    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-977f7bbf6-j5w8d"]
Jul 03 14:15:28.419205 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:28.419171    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/06a8df1e-5f9f-4593-9204-61af185ad4ff-apiservice-cert\") pod \"opendatahub-operator-controller-manager-977f7bbf6-j5w8d\" (UID: \"06a8df1e-5f9f-4593-9204-61af185ad4ff\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-j5w8d"
Jul 03 14:15:28.419205 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:28.419211    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06a8df1e-5f9f-4593-9204-61af185ad4ff-tmp\") pod \"opendatahub-operator-controller-manager-977f7bbf6-j5w8d\" (UID: \"06a8df1e-5f9f-4593-9204-61af185ad4ff\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-j5w8d"
Jul 03 14:15:28.419411 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:28.419281    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/06a8df1e-5f9f-4593-9204-61af185ad4ff-webhook-cert\") pod \"opendatahub-operator-controller-manager-977f7bbf6-j5w8d\" (UID: \"06a8df1e-5f9f-4593-9204-61af185ad4ff\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-j5w8d"
Jul 03 14:15:28.419411 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:28.419346    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm6d4\" (UniqueName: \"kubernetes.io/projected/06a8df1e-5f9f-4593-9204-61af185ad4ff-kube-api-access-mm6d4\") pod \"opendatahub-operator-controller-manager-977f7bbf6-j5w8d\" (UID: \"06a8df1e-5f9f-4593-9204-61af185ad4ff\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-j5w8d"
Jul 03 14:15:28.419411 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:28.419374    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/06a8df1e-5f9f-4593-9204-61af185ad4ff-manifests\") pod \"opendatahub-operator-controller-manager-977f7bbf6-j5w8d\" (UID: \"06a8df1e-5f9f-4593-9204-61af185ad4ff\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-j5w8d"
Jul 03 14:15:28.520217 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:28.520181    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/06a8df1e-5f9f-4593-9204-61af185ad4ff-webhook-cert\") pod \"opendatahub-operator-controller-manager-977f7bbf6-j5w8d\" (UID: \"06a8df1e-5f9f-4593-9204-61af185ad4ff\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-j5w8d"
Jul 03 14:15:28.520398 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:28.520233    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm6d4\" (UniqueName: \"kubernetes.io/projected/06a8df1e-5f9f-4593-9204-61af185ad4ff-kube-api-access-mm6d4\") pod \"opendatahub-operator-controller-manager-977f7bbf6-j5w8d\" (UID: \"06a8df1e-5f9f-4593-9204-61af185ad4ff\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-j5w8d"
Jul 03 14:15:28.520398 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:28.520258    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/06a8df1e-5f9f-4593-9204-61af185ad4ff-manifests\") pod \"opendatahub-operator-controller-manager-977f7bbf6-j5w8d\" (UID: \"06a8df1e-5f9f-4593-9204-61af185ad4ff\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-j5w8d"
Jul 03 14:15:28.520398 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:28.520287    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/06a8df1e-5f9f-4593-9204-61af185ad4ff-apiservice-cert\") pod \"opendatahub-operator-controller-manager-977f7bbf6-j5w8d\" (UID: \"06a8df1e-5f9f-4593-9204-61af185ad4ff\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-j5w8d"
Jul 03 14:15:28.520398 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:28.520308    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06a8df1e-5f9f-4593-9204-61af185ad4ff-tmp\") pod \"opendatahub-operator-controller-manager-977f7bbf6-j5w8d\" (UID: \"06a8df1e-5f9f-4593-9204-61af185ad4ff\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-j5w8d"
Jul 03 14:15:28.520699 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:28.520672    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/06a8df1e-5f9f-4593-9204-61af185ad4ff-manifests\") pod \"opendatahub-operator-controller-manager-977f7bbf6-j5w8d\" (UID: \"06a8df1e-5f9f-4593-9204-61af185ad4ff\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-j5w8d"
Jul 03 14:15:28.522537 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:28.522507    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06a8df1e-5f9f-4593-9204-61af185ad4ff-tmp\") pod \"opendatahub-operator-controller-manager-977f7bbf6-j5w8d\" (UID: \"06a8df1e-5f9f-4593-9204-61af185ad4ff\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-j5w8d"
Jul 03 14:15:28.522732 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:28.522708    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/06a8df1e-5f9f-4593-9204-61af185ad4ff-apiservice-cert\") pod \"opendatahub-operator-controller-manager-977f7bbf6-j5w8d\" (UID: \"06a8df1e-5f9f-4593-9204-61af185ad4ff\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-j5w8d"
Jul 03 14:15:28.522829 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:28.522711    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/06a8df1e-5f9f-4593-9204-61af185ad4ff-webhook-cert\") pod \"opendatahub-operator-controller-manager-977f7bbf6-j5w8d\" (UID: \"06a8df1e-5f9f-4593-9204-61af185ad4ff\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-j5w8d"
Jul 03 14:15:28.545252 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:28.545226    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm6d4\" (UniqueName: \"kubernetes.io/projected/06a8df1e-5f9f-4593-9204-61af185ad4ff-kube-api-access-mm6d4\") pod \"opendatahub-operator-controller-manager-977f7bbf6-j5w8d\" (UID: \"06a8df1e-5f9f-4593-9204-61af185ad4ff\") " pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-j5w8d"
Jul 03 14:15:28.630508 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:28.630483    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-j5w8d"
Jul 03 14:15:28.779407 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:15:28.779377    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06a8df1e_5f9f_4593_9204_61af185ad4ff.slice/crio-c3f661bfdaa3c7057d5fb9a4345b7cd65b595fcfe94008237b0e3e2fccbfb52a WatchSource:0}: Error finding container c3f661bfdaa3c7057d5fb9a4345b7cd65b595fcfe94008237b0e3e2fccbfb52a: Status 404 returned error can't find the container with id c3f661bfdaa3c7057d5fb9a4345b7cd65b595fcfe94008237b0e3e2fccbfb52a
Jul 03 14:15:28.782742 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:28.782717    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-977f7bbf6-j5w8d"]
Jul 03 14:15:28.986853 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:28.986716    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-j5w8d" event={"ID":"06a8df1e-5f9f-4593-9204-61af185ad4ff","Type":"ContainerStarted","Data":"c3f661bfdaa3c7057d5fb9a4345b7cd65b595fcfe94008237b0e3e2fccbfb52a"}
Jul 03 14:15:32.002675 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:32.002590    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-j5w8d" event={"ID":"06a8df1e-5f9f-4593-9204-61af185ad4ff","Type":"ContainerStarted","Data":"e5c7b3416d9cff332c1e52041bbc8db95eb0c7072249e868e8b409dfdbdc7de9"}
Jul 03 14:15:32.962498 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:32.962471    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-78cbbf6689-pgfl6"
Jul 03 14:15:33.008803 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:33.008770    2563 generic.go:356] "Generic (PLEG): container finished" podID="06a8df1e-5f9f-4593-9204-61af185ad4ff" containerID="e5c7b3416d9cff332c1e52041bbc8db95eb0c7072249e868e8b409dfdbdc7de9" exitCode=0
Jul 03 14:15:33.009180 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:33.008858    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-j5w8d" event={"ID":"06a8df1e-5f9f-4593-9204-61af185ad4ff","Type":"ContainerDied","Data":"e5c7b3416d9cff332c1e52041bbc8db95eb0c7072249e868e8b409dfdbdc7de9"}
Jul 03 14:15:34.015533 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:34.015500    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-j5w8d" event={"ID":"06a8df1e-5f9f-4593-9204-61af185ad4ff","Type":"ContainerStarted","Data":"2826d952e9128257f927615d4e85307d79f233ea121ee06620db789ac4529485"}
Jul 03 14:15:34.015945 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:34.015630    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-j5w8d"
Jul 03 14:15:34.067575 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:34.067525    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-j5w8d" podStartSLOduration=1.6449502790000001 podStartE2EDuration="6.067510937s" totalImagesPullingTime="3.331504096s" totalInitContainerRuntime="1.091056562s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:15:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:15:34.060792753 +0000 UTC m=+447.510926216" watchObservedRunningTime="2026-07-03 14:15:34.067510937 +0000 UTC m=+447.517644449"
Jul 03 14:15:45.021793 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:45.021753    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-977f7bbf6-j5w8d"
Jul 03 14:15:48.450900 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:48.450866    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-557657c976-htz8w"]
Jul 03 14:15:48.460964 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:48.460945    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-557657c976-htz8w"
Jul 03 14:15:48.464308 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:48.464287    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 03 14:15:48.464433 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:48.464321    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 03 14:15:48.464433 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:48.464356    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-94fj6\""
Jul 03 14:15:48.464433 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:48.464359    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 03 14:15:48.465343 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:48.465331    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 03 14:15:48.474360 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:48.474339    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-557657c976-htz8w"]
Jul 03 14:15:48.500455 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:48.500428    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d31500b1-8a7c-4d74-b3b0-5b9090730c00-tls-certs\") pod \"kube-auth-proxy-557657c976-htz8w\" (UID: \"d31500b1-8a7c-4d74-b3b0-5b9090730c00\") " pod="openshift-ingress/kube-auth-proxy-557657c976-htz8w"
Jul 03 14:15:48.500570 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:48.500474    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d31500b1-8a7c-4d74-b3b0-5b9090730c00-tmp\") pod \"kube-auth-proxy-557657c976-htz8w\" (UID: \"d31500b1-8a7c-4d74-b3b0-5b9090730c00\") " pod="openshift-ingress/kube-auth-proxy-557657c976-htz8w"
Jul 03 14:15:48.500570 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:48.500505    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7spfn\" (UniqueName: \"kubernetes.io/projected/d31500b1-8a7c-4d74-b3b0-5b9090730c00-kube-api-access-7spfn\") pod \"kube-auth-proxy-557657c976-htz8w\" (UID: \"d31500b1-8a7c-4d74-b3b0-5b9090730c00\") " pod="openshift-ingress/kube-auth-proxy-557657c976-htz8w"
Jul 03 14:15:48.600887 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:48.600853    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7spfn\" (UniqueName: \"kubernetes.io/projected/d31500b1-8a7c-4d74-b3b0-5b9090730c00-kube-api-access-7spfn\") pod \"kube-auth-proxy-557657c976-htz8w\" (UID: \"d31500b1-8a7c-4d74-b3b0-5b9090730c00\") " pod="openshift-ingress/kube-auth-proxy-557657c976-htz8w"
Jul 03 14:15:48.601108 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:48.600913    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d31500b1-8a7c-4d74-b3b0-5b9090730c00-tls-certs\") pod \"kube-auth-proxy-557657c976-htz8w\" (UID: \"d31500b1-8a7c-4d74-b3b0-5b9090730c00\") " pod="openshift-ingress/kube-auth-proxy-557657c976-htz8w"
Jul 03 14:15:48.601108 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:48.600997    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d31500b1-8a7c-4d74-b3b0-5b9090730c00-tmp\") pod \"kube-auth-proxy-557657c976-htz8w\" (UID: \"d31500b1-8a7c-4d74-b3b0-5b9090730c00\") " pod="openshift-ingress/kube-auth-proxy-557657c976-htz8w"
Jul 03 14:15:48.603136 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:48.603115    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d31500b1-8a7c-4d74-b3b0-5b9090730c00-tmp\") pod \"kube-auth-proxy-557657c976-htz8w\" (UID: \"d31500b1-8a7c-4d74-b3b0-5b9090730c00\") " pod="openshift-ingress/kube-auth-proxy-557657c976-htz8w"
Jul 03 14:15:48.603260 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:48.603234    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d31500b1-8a7c-4d74-b3b0-5b9090730c00-tls-certs\") pod \"kube-auth-proxy-557657c976-htz8w\" (UID: \"d31500b1-8a7c-4d74-b3b0-5b9090730c00\") " pod="openshift-ingress/kube-auth-proxy-557657c976-htz8w"
Jul 03 14:15:48.614791 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:48.614754    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7spfn\" (UniqueName: \"kubernetes.io/projected/d31500b1-8a7c-4d74-b3b0-5b9090730c00-kube-api-access-7spfn\") pod \"kube-auth-proxy-557657c976-htz8w\" (UID: \"d31500b1-8a7c-4d74-b3b0-5b9090730c00\") " pod="openshift-ingress/kube-auth-proxy-557657c976-htz8w"
Jul 03 14:15:48.769923 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:48.769836    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-557657c976-htz8w"
Jul 03 14:15:48.901877 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:48.901852    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-557657c976-htz8w"]
Jul 03 14:15:48.903889 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:15:48.903862    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd31500b1_8a7c_4d74_b3b0_5b9090730c00.slice/crio-bd8826acec313609cdbdb1e71043eff416927751a99efc9cdde244a2788bd5f1 WatchSource:0}: Error finding container bd8826acec313609cdbdb1e71043eff416927751a99efc9cdde244a2788bd5f1: Status 404 returned error can't find the container with id bd8826acec313609cdbdb1e71043eff416927751a99efc9cdde244a2788bd5f1
Jul 03 14:15:49.083965 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:49.083885    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-557657c976-htz8w" event={"ID":"d31500b1-8a7c-4d74-b3b0-5b9090730c00","Type":"ContainerStarted","Data":"bd8826acec313609cdbdb1e71043eff416927751a99efc9cdde244a2788bd5f1"}
Jul 03 14:15:52.102126 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:52.102086    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-557657c976-htz8w" event={"ID":"d31500b1-8a7c-4d74-b3b0-5b9090730c00","Type":"ContainerStarted","Data":"b20b0d9cfa278dd5e2eca624a7023d7b3b3e67393b405c8455fb320ae2461749"}
Jul 03 14:15:52.124694 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:15:52.124651    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-557657c976-htz8w" podStartSLOduration=1.153459859 podStartE2EDuration="4.124635482s" totalImagesPullingTime="2.971175623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:15:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:15:52.123854847 +0000 UTC m=+465.573988304" watchObservedRunningTime="2026-07-03 14:15:52.124635482 +0000 UTC m=+465.574768921"
Jul 03 14:17:27.999057 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:17:27.998959    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8qm8w"]
Jul 03 14:17:28.002432 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:17:28.002414    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8qm8w"
Jul 03 14:17:28.005334 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:17:28.005312    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 03 14:17:28.006133 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:17:28.006117    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 03 14:17:28.006219 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:17:28.006124    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4p5v9\""
Jul 03 14:17:28.006219 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:17:28.006124    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 03 14:17:28.006219 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:17:28.006135    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 03 14:17:28.017795 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:17:28.017774    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8qm8w"]
Jul 03 14:17:28.116401 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:17:28.116361    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/673f8290-39c8-41f1-afbf-9da9db2f895d-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8qm8w\" (UID: \"673f8290-39c8-41f1-afbf-9da9db2f895d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8qm8w"
Jul 03 14:17:28.116575 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:17:28.116415    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/673f8290-39c8-41f1-afbf-9da9db2f895d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8qm8w\" (UID: \"673f8290-39c8-41f1-afbf-9da9db2f895d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8qm8w"
Jul 03 14:17:28.116622 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:17:28.116580    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44ldx\" (UniqueName: \"kubernetes.io/projected/673f8290-39c8-41f1-afbf-9da9db2f895d-kube-api-access-44ldx\") pod \"kuadrant-console-plugin-66b5774985-8qm8w\" (UID: \"673f8290-39c8-41f1-afbf-9da9db2f895d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8qm8w"
Jul 03 14:17:28.217302 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:17:28.217263    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44ldx\" (UniqueName: \"kubernetes.io/projected/673f8290-39c8-41f1-afbf-9da9db2f895d-kube-api-access-44ldx\") pod \"kuadrant-console-plugin-66b5774985-8qm8w\" (UID: \"673f8290-39c8-41f1-afbf-9da9db2f895d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8qm8w"
Jul 03 14:17:28.217480 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:17:28.217337    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/673f8290-39c8-41f1-afbf-9da9db2f895d-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8qm8w\" (UID: \"673f8290-39c8-41f1-afbf-9da9db2f895d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8qm8w"
Jul 03 14:17:28.217480 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:17:28.217370    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/673f8290-39c8-41f1-afbf-9da9db2f895d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8qm8w\" (UID: \"673f8290-39c8-41f1-afbf-9da9db2f895d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8qm8w"
Jul 03 14:17:28.218032 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:17:28.218006    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/673f8290-39c8-41f1-afbf-9da9db2f895d-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8qm8w\" (UID: \"673f8290-39c8-41f1-afbf-9da9db2f895d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8qm8w"
Jul 03 14:17:28.219743 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:17:28.219724    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/673f8290-39c8-41f1-afbf-9da9db2f895d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8qm8w\" (UID: \"673f8290-39c8-41f1-afbf-9da9db2f895d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8qm8w"
Jul 03 14:17:28.226409 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:17:28.226382    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44ldx\" (UniqueName: \"kubernetes.io/projected/673f8290-39c8-41f1-afbf-9da9db2f895d-kube-api-access-44ldx\") pod \"kuadrant-console-plugin-66b5774985-8qm8w\" (UID: \"673f8290-39c8-41f1-afbf-9da9db2f895d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8qm8w"
Jul 03 14:17:28.324735 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:17:28.324657    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8qm8w"
Jul 03 14:17:28.479993 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:17:28.479956    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8qm8w"]
Jul 03 14:17:28.482364 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:17:28.482340    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod673f8290_39c8_41f1_afbf_9da9db2f895d.slice/crio-1d7b391db49d150eaeacdccb1db31e9e04a63786fd31a3b1abbd0c7ef1818b89 WatchSource:0}: Error finding container 1d7b391db49d150eaeacdccb1db31e9e04a63786fd31a3b1abbd0c7ef1818b89: Status 404 returned error can't find the container with id 1d7b391db49d150eaeacdccb1db31e9e04a63786fd31a3b1abbd0c7ef1818b89
Jul 03 14:17:28.529909 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:17:28.529873    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8qm8w" event={"ID":"673f8290-39c8-41f1-afbf-9da9db2f895d","Type":"ContainerStarted","Data":"1d7b391db49d150eaeacdccb1db31e9e04a63786fd31a3b1abbd0c7ef1818b89"}
Jul 03 14:17:53.663138 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:17:53.663100    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8qm8w" event={"ID":"673f8290-39c8-41f1-afbf-9da9db2f895d","Type":"ContainerStarted","Data":"3e6b6b726b5d6ae6d6c69de8e58af66cd8984ab7c55ed7c53d8c3b0c53ef1a75"}
Jul 03 14:17:53.708895 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:17:53.708847    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8qm8w" podStartSLOduration=2.257853163 podStartE2EDuration="26.708830388s" totalImagesPullingTime="24.450977225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:17:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:17:53.708070516 +0000 UTC m=+587.158203957" watchObservedRunningTime="2026-07-03 14:17:53.708830388 +0000 UTC m=+587.158963828"
Jul 03 14:18:10.179199 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.179164    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"]
Jul 03 14:18:10.182846 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.182824    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:10.188397 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.188370    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-5qpl8\""
Jul 03 14:18:10.188507 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.188432    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 03 14:18:10.261516 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.261483    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"]
Jul 03 14:18:10.310913 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.310869    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14e864ac-ec94-4ca1-8496-4d1fc83fb11d-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-8ksxz\" (UID: \"14e864ac-ec94-4ca1-8496-4d1fc83fb11d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:10.311113 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.310919    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14e864ac-ec94-4ca1-8496-4d1fc83fb11d-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-8ksxz\" (UID: \"14e864ac-ec94-4ca1-8496-4d1fc83fb11d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:10.311113 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.310951    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14e864ac-ec94-4ca1-8496-4d1fc83fb11d-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-8ksxz\" (UID: \"14e864ac-ec94-4ca1-8496-4d1fc83fb11d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:10.311113 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.311000    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14e864ac-ec94-4ca1-8496-4d1fc83fb11d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-8ksxz\" (UID: \"14e864ac-ec94-4ca1-8496-4d1fc83fb11d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:10.311113 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.311045    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14e864ac-ec94-4ca1-8496-4d1fc83fb11d-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-8ksxz\" (UID: \"14e864ac-ec94-4ca1-8496-4d1fc83fb11d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:10.311113 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.311095    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sjt4\" (UniqueName: \"kubernetes.io/projected/14e864ac-ec94-4ca1-8496-4d1fc83fb11d-kube-api-access-5sjt4\") pod \"maas-default-gateway-openshift-default-56654dcf7-8ksxz\" (UID: \"14e864ac-ec94-4ca1-8496-4d1fc83fb11d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:10.311295 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.311126    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14e864ac-ec94-4ca1-8496-4d1fc83fb11d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-8ksxz\" (UID: \"14e864ac-ec94-4ca1-8496-4d1fc83fb11d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:10.311295 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.311153    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14e864ac-ec94-4ca1-8496-4d1fc83fb11d-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-8ksxz\" (UID: \"14e864ac-ec94-4ca1-8496-4d1fc83fb11d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:10.311295 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.311184    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14e864ac-ec94-4ca1-8496-4d1fc83fb11d-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-8ksxz\" (UID: \"14e864ac-ec94-4ca1-8496-4d1fc83fb11d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:10.411620 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.411588    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14e864ac-ec94-4ca1-8496-4d1fc83fb11d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-8ksxz\" (UID: \"14e864ac-ec94-4ca1-8496-4d1fc83fb11d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:10.411793 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.411628    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14e864ac-ec94-4ca1-8496-4d1fc83fb11d-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-8ksxz\" (UID: \"14e864ac-ec94-4ca1-8496-4d1fc83fb11d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:10.411793 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.411750    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14e864ac-ec94-4ca1-8496-4d1fc83fb11d-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-8ksxz\" (UID: \"14e864ac-ec94-4ca1-8496-4d1fc83fb11d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:10.411874 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.411798    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14e864ac-ec94-4ca1-8496-4d1fc83fb11d-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-8ksxz\" (UID: \"14e864ac-ec94-4ca1-8496-4d1fc83fb11d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:10.411874 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.411826    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14e864ac-ec94-4ca1-8496-4d1fc83fb11d-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-8ksxz\" (UID: \"14e864ac-ec94-4ca1-8496-4d1fc83fb11d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:10.411874 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.411858    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14e864ac-ec94-4ca1-8496-4d1fc83fb11d-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-8ksxz\" (UID: \"14e864ac-ec94-4ca1-8496-4d1fc83fb11d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:10.412050 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.411885    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14e864ac-ec94-4ca1-8496-4d1fc83fb11d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-8ksxz\" (UID: \"14e864ac-ec94-4ca1-8496-4d1fc83fb11d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:10.412050 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.411960    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14e864ac-ec94-4ca1-8496-4d1fc83fb11d-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-8ksxz\" (UID: \"14e864ac-ec94-4ca1-8496-4d1fc83fb11d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:10.412160 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.412066    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sjt4\" (UniqueName: \"kubernetes.io/projected/14e864ac-ec94-4ca1-8496-4d1fc83fb11d-kube-api-access-5sjt4\") pod \"maas-default-gateway-openshift-default-56654dcf7-8ksxz\" (UID: \"14e864ac-ec94-4ca1-8496-4d1fc83fb11d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:10.412216 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.412187    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14e864ac-ec94-4ca1-8496-4d1fc83fb11d-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-8ksxz\" (UID: \"14e864ac-ec94-4ca1-8496-4d1fc83fb11d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:10.412271 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.412238    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14e864ac-ec94-4ca1-8496-4d1fc83fb11d-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-8ksxz\" (UID: \"14e864ac-ec94-4ca1-8496-4d1fc83fb11d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:10.412324 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.412296    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14e864ac-ec94-4ca1-8496-4d1fc83fb11d-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-8ksxz\" (UID: \"14e864ac-ec94-4ca1-8496-4d1fc83fb11d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:10.412559 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.412541    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14e864ac-ec94-4ca1-8496-4d1fc83fb11d-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-8ksxz\" (UID: \"14e864ac-ec94-4ca1-8496-4d1fc83fb11d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:10.412559 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.412552    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14e864ac-ec94-4ca1-8496-4d1fc83fb11d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-8ksxz\" (UID: \"14e864ac-ec94-4ca1-8496-4d1fc83fb11d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:10.414214 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.414192    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14e864ac-ec94-4ca1-8496-4d1fc83fb11d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-8ksxz\" (UID: \"14e864ac-ec94-4ca1-8496-4d1fc83fb11d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:10.414323 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.414292    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14e864ac-ec94-4ca1-8496-4d1fc83fb11d-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-8ksxz\" (UID: \"14e864ac-ec94-4ca1-8496-4d1fc83fb11d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:10.451547 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.451474    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14e864ac-ec94-4ca1-8496-4d1fc83fb11d-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-8ksxz\" (UID: \"14e864ac-ec94-4ca1-8496-4d1fc83fb11d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:10.453720 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.453697    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sjt4\" (UniqueName: \"kubernetes.io/projected/14e864ac-ec94-4ca1-8496-4d1fc83fb11d-kube-api-access-5sjt4\") pod \"maas-default-gateway-openshift-default-56654dcf7-8ksxz\" (UID: \"14e864ac-ec94-4ca1-8496-4d1fc83fb11d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:10.492908 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.492881    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:10.644545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.644460    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"]
Jul 03 14:18:10.647335 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:18:10.647312    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14e864ac_ec94_4ca1_8496_4d1fc83fb11d.slice/crio-3e141351d2800e17d29398915afada1795525085c67f121b5d15b0180b016928 WatchSource:0}: Error finding container 3e141351d2800e17d29398915afada1795525085c67f121b5d15b0180b016928: Status 404 returned error can't find the container with id 3e141351d2800e17d29398915afada1795525085c67f121b5d15b0180b016928
Jul 03 14:18:10.741167 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:10.741085    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz" event={"ID":"14e864ac-ec94-4ca1-8496-4d1fc83fb11d","Type":"ContainerStarted","Data":"3e141351d2800e17d29398915afada1795525085c67f121b5d15b0180b016928"}
Jul 03 14:18:13.342622 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:13.342581    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 03 14:18:13.342940 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:13.342654    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 03 14:18:13.342940 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:13.342682    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 03 14:18:13.759808 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:13.759771    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz" event={"ID":"14e864ac-ec94-4ca1-8496-4d1fc83fb11d","Type":"ContainerStarted","Data":"894c2f6078d9a11670a5e6f367985c56f66d4d79badd502dcff4b41c91677865"}
Jul 03 14:18:13.807663 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:13.807603    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz" podStartSLOduration=1.114555354 podStartE2EDuration="3.807587207s" totalImagesPullingTime="2.693031853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:18:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:18:13.805798864 +0000 UTC m=+607.255932304" watchObservedRunningTime="2026-07-03 14:18:13.807587207 +0000 UTC m=+607.257720647"
Jul 03 14:18:14.493932 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:14.493892    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:14.498392 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:14.498367    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:14.764589 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:14.764506    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:14.765539 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:14.765520    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:18:15.520650 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:15.520611    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:18:15.524085 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:15.524064    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"
Jul 03 14:18:15.527160 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:15.527139    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 03 14:18:15.543479 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:15.543446    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:18:15.580985 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:15.580946    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:18:15.665791 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:15.665753    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6swsx\" (UniqueName: \"kubernetes.io/projected/b29859bb-2b3d-4218-92a3-1329e905803a-kube-api-access-6swsx\") pod \"limitador-limitador-54d6d988d5-xrsrt\" (UID: \"b29859bb-2b3d-4218-92a3-1329e905803a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"
Jul 03 14:18:15.665948 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:15.665844    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b29859bb-2b3d-4218-92a3-1329e905803a-config-file\") pod \"limitador-limitador-54d6d988d5-xrsrt\" (UID: \"b29859bb-2b3d-4218-92a3-1329e905803a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"
Jul 03 14:18:15.766373 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:15.766339    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b29859bb-2b3d-4218-92a3-1329e905803a-config-file\") pod \"limitador-limitador-54d6d988d5-xrsrt\" (UID: \"b29859bb-2b3d-4218-92a3-1329e905803a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"
Jul 03 14:18:15.766533 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:15.766396    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6swsx\" (UniqueName: \"kubernetes.io/projected/b29859bb-2b3d-4218-92a3-1329e905803a-kube-api-access-6swsx\") pod \"limitador-limitador-54d6d988d5-xrsrt\" (UID: \"b29859bb-2b3d-4218-92a3-1329e905803a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"
Jul 03 14:18:15.767038 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:15.767016    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b29859bb-2b3d-4218-92a3-1329e905803a-config-file\") pod \"limitador-limitador-54d6d988d5-xrsrt\" (UID: \"b29859bb-2b3d-4218-92a3-1329e905803a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"
Jul 03 14:18:15.782683 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:15.782620    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6swsx\" (UniqueName: \"kubernetes.io/projected/b29859bb-2b3d-4218-92a3-1329e905803a-kube-api-access-6swsx\") pod \"limitador-limitador-54d6d988d5-xrsrt\" (UID: \"b29859bb-2b3d-4218-92a3-1329e905803a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"
Jul 03 14:18:15.834212 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:15.834172    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"
Jul 03 14:18:15.976323 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:15.976298    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:18:15.978725 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:18:15.978700    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb29859bb_2b3d_4218_92a3_1329e905803a.slice/crio-05bfa298f44ace9ed9f0a9f8f352cf8026af0d94ec9860d0911841571b1ba096 WatchSource:0}: Error finding container 05bfa298f44ace9ed9f0a9f8f352cf8026af0d94ec9860d0911841571b1ba096: Status 404 returned error can't find the container with id 05bfa298f44ace9ed9f0a9f8f352cf8026af0d94ec9860d0911841571b1ba096
Jul 03 14:18:16.278180 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:16.278147    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-crsl8"]
Jul 03 14:18:16.282701 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:16.282680    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-crsl8"
Jul 03 14:18:16.308424 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:18:16.308390    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"authorino-75d556bd77-crsl8\" is forbidden: User \"system:node:ip-10-0-142-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-233.ec2.internal' and this object" podUID="d56d9279-1c8a-4e1b-8991-e4d2a5133725" pod="kuadrant-system/authorino-75d556bd77-crsl8"
Jul 03 14:18:16.308424 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:18:16.308400    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"authorino-authorino-dockercfg-28xd8\" is forbidden: User \"system:node:ip-10-0-142-233.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-233.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-28xd8\"" type="*v1.Secret"
Jul 03 14:18:16.332012 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:16.331978    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-crsl8"]
Jul 03 14:18:16.373555 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:16.373524    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xghvr\" (UniqueName: \"kubernetes.io/projected/d56d9279-1c8a-4e1b-8991-e4d2a5133725-kube-api-access-xghvr\") pod \"authorino-75d556bd77-crsl8\" (UID: \"d56d9279-1c8a-4e1b-8991-e4d2a5133725\") " pod="kuadrant-system/authorino-75d556bd77-crsl8"
Jul 03 14:18:16.475043 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:16.475005    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xghvr\" (UniqueName: \"kubernetes.io/projected/d56d9279-1c8a-4e1b-8991-e4d2a5133725-kube-api-access-xghvr\") pod \"authorino-75d556bd77-crsl8\" (UID: \"d56d9279-1c8a-4e1b-8991-e4d2a5133725\") " pod="kuadrant-system/authorino-75d556bd77-crsl8"
Jul 03 14:18:16.490513 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:16.490448    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xghvr\" (UniqueName: \"kubernetes.io/projected/d56d9279-1c8a-4e1b-8991-e4d2a5133725-kube-api-access-xghvr\") pod \"authorino-75d556bd77-crsl8\" (UID: \"d56d9279-1c8a-4e1b-8991-e4d2a5133725\") " pod="kuadrant-system/authorino-75d556bd77-crsl8"
Jul 03 14:18:16.776898 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:16.776851    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-xrsrt" event={"ID":"b29859bb-2b3d-4218-92a3-1329e905803a","Type":"ContainerStarted","Data":"05bfa298f44ace9ed9f0a9f8f352cf8026af0d94ec9860d0911841571b1ba096"}
Jul 03 14:18:17.283939 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:17.283902    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-28xd8\""
Jul 03 14:18:17.292597 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:17.292571    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-crsl8"
Jul 03 14:18:18.538806 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:18.538763    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-crsl8"]
Jul 03 14:18:18.542251 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:18:18.542217    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd56d9279_1c8a_4e1b_8991_e4d2a5133725.slice/crio-c1391c4a2fae55a5a4443b210151974b85859a0b719aeefe4bddd75b0f3e513e WatchSource:0}: Error finding container c1391c4a2fae55a5a4443b210151974b85859a0b719aeefe4bddd75b0f3e513e: Status 404 returned error can't find the container with id c1391c4a2fae55a5a4443b210151974b85859a0b719aeefe4bddd75b0f3e513e
Jul 03 14:18:18.788645 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:18.788551    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-crsl8" event={"ID":"d56d9279-1c8a-4e1b-8991-e4d2a5133725","Type":"ContainerStarted","Data":"c1391c4a2fae55a5a4443b210151974b85859a0b719aeefe4bddd75b0f3e513e"}
Jul 03 14:18:18.789914 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:18.789888    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-xrsrt" event={"ID":"b29859bb-2b3d-4218-92a3-1329e905803a","Type":"ContainerStarted","Data":"7b4d5926d45a9b6add42dba74a8efacc4eda871453f94c35af9fafea0420df8f"}
Jul 03 14:18:18.790035 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:18.790024    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"
Jul 03 14:18:18.824580 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:18.824530    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-xrsrt" podStartSLOduration=1.354092039 podStartE2EDuration="3.824513654s" totalImagesPullingTime="2.470421615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:18:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:18:18.820619814 +0000 UTC m=+612.270753278" watchObservedRunningTime="2026-07-03 14:18:18.824513654 +0000 UTC m=+612.274647093"
Jul 03 14:18:22.812305 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:22.812263    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-crsl8" event={"ID":"d56d9279-1c8a-4e1b-8991-e4d2a5133725","Type":"ContainerStarted","Data":"c42681178c4e1068245e7cac288bb77d406dae2dbd7eb6db73c0604e0a57c582"}
Jul 03 14:18:22.836578 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:22.836525    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-crsl8" podStartSLOduration=3.34709655 podStartE2EDuration="6.836509662s" totalImagesPullingTime="3.489413112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:18:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:18:22.834483671 +0000 UTC m=+616.284617124" watchObservedRunningTime="2026-07-03 14:18:22.836509662 +0000 UTC m=+616.286643130"
Jul 03 14:18:29.798085 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:29.798011    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"
Jul 03 14:18:46.814641 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:46.814609    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-crsl8"]
Jul 03 14:18:46.817167 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:46.814770    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-crsl8" podUID="d56d9279-1c8a-4e1b-8991-e4d2a5133725" containerName="authorino" containerID="cri-o://c42681178c4e1068245e7cac288bb77d406dae2dbd7eb6db73c0604e0a57c582" gracePeriod=30
Jul 03 14:18:46.933746 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:46.933716    2563 generic.go:356] "Generic (PLEG): container finished" podID="d56d9279-1c8a-4e1b-8991-e4d2a5133725" containerID="c42681178c4e1068245e7cac288bb77d406dae2dbd7eb6db73c0604e0a57c582" exitCode=0
Jul 03 14:18:46.933884 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:46.933774    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-crsl8" event={"ID":"d56d9279-1c8a-4e1b-8991-e4d2a5133725","Type":"ContainerDied","Data":"c42681178c4e1068245e7cac288bb77d406dae2dbd7eb6db73c0604e0a57c582"}
Jul 03 14:18:47.058363 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:47.058335    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-crsl8"
Jul 03 14:18:47.158292 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:47.158259    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xghvr\" (UniqueName: \"kubernetes.io/projected/d56d9279-1c8a-4e1b-8991-e4d2a5133725-kube-api-access-xghvr\") pod \"d56d9279-1c8a-4e1b-8991-e4d2a5133725\" (UID: \"d56d9279-1c8a-4e1b-8991-e4d2a5133725\") "
Jul 03 14:18:47.160469 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:47.160440    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d56d9279-1c8a-4e1b-8991-e4d2a5133725-kube-api-access-xghvr" (OuterVolumeSpecName: "kube-api-access-xghvr") pod "d56d9279-1c8a-4e1b-8991-e4d2a5133725" (UID: "d56d9279-1c8a-4e1b-8991-e4d2a5133725"). InnerVolumeSpecName "kube-api-access-xghvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:18:47.259674 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:47.259643    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xghvr\" (UniqueName: \"kubernetes.io/projected/d56d9279-1c8a-4e1b-8991-e4d2a5133725-kube-api-access-xghvr\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:18:47.939666 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:47.939626    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-crsl8" event={"ID":"d56d9279-1c8a-4e1b-8991-e4d2a5133725","Type":"ContainerDied","Data":"c1391c4a2fae55a5a4443b210151974b85859a0b719aeefe4bddd75b0f3e513e"}
Jul 03 14:18:47.939666 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:47.939671    2563 scope.go:117] "RemoveContainer" containerID="c42681178c4e1068245e7cac288bb77d406dae2dbd7eb6db73c0604e0a57c582"
Jul 03 14:18:47.940188 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:47.939641    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-crsl8"
Jul 03 14:18:47.968214 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:47.968192    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-crsl8"]
Jul 03 14:18:47.971259 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:47.971237    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-crsl8"]
Jul 03 14:18:49.058064 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:18:49.058032    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d56d9279-1c8a-4e1b-8991-e4d2a5133725" path="/var/lib/kubelet/pods/d56d9279-1c8a-4e1b-8991-e4d2a5133725/volumes"
Jul 03 14:19:27.048479 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:19:27.048441    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:19:35.164068 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:19:35.164032    2563 generic.go:356] "Generic (PLEG): container finished" podID="14e864ac-ec94-4ca1-8496-4d1fc83fb11d" containerID="894c2f6078d9a11670a5e6f367985c56f66d4d79badd502dcff4b41c91677865" exitCode=0
Jul 03 14:19:35.164539 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:19:35.164103    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz" event={"ID":"14e864ac-ec94-4ca1-8496-4d1fc83fb11d","Type":"ContainerDied","Data":"894c2f6078d9a11670a5e6f367985c56f66d4d79badd502dcff4b41c91677865"}
Jul 03 14:19:35.164539 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:19:35.164503    2563 scope.go:117] "RemoveContainer" containerID="894c2f6078d9a11670a5e6f367985c56f66d4d79badd502dcff4b41c91677865"
Jul 03 14:19:35.165448 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:19:35.165422    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 03 14:19:35.165545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:19:35.165485    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 03 14:19:35.165545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:19:35.165516    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 03 14:19:36.170285 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:19:36.170252    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz" event={"ID":"14e864ac-ec94-4ca1-8496-4d1fc83fb11d","Type":"ContainerStarted","Data":"7fffc7325e04aba6443134dafdaca0b2e6b1eba74cb762b33bef87c8ed70b4a2"}
Jul 03 14:19:36.493768 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:19:36.493674    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:19:36.493919 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:19:36.493890    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-8ksxz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Jul 03 14:19:36.493961 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:19:36.493947    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz" podUID="14e864ac-ec94-4ca1-8496-4d1fc83fb11d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Jul 03 14:19:37.493608 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:19:37.493570    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-8ksxz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Jul 03 14:19:37.494042 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:19:37.493625    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz" podUID="14e864ac-ec94-4ca1-8496-4d1fc83fb11d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Jul 03 14:19:38.494135 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:19:38.494098    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-8ksxz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Jul 03 14:19:38.494520 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:19:38.494153    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz" podUID="14e864ac-ec94-4ca1-8496-4d1fc83fb11d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Jul 03 14:19:39.494089 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:19:39.494049    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-8ksxz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Jul 03 14:19:39.494260 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:19:39.494135    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz" podUID="14e864ac-ec94-4ca1-8496-4d1fc83fb11d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Jul 03 14:19:40.497138 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:19:40.497105    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:19:40.497543 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:19:40.497450    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:19:40.498230 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:19:40.498213    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8ksxz"
Jul 03 14:19:42.053584 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:19:42.053552    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:19:49.957577 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:19:49.957544    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:20:04.060689 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.060611    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f"]
Jul 03 14:20:04.061193 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.061007    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d56d9279-1c8a-4e1b-8991-e4d2a5133725" containerName="authorino"
Jul 03 14:20:04.061193 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.061020    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d56d9279-1c8a-4e1b-8991-e4d2a5133725" containerName="authorino"
Jul 03 14:20:04.061193 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.061092    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d56d9279-1c8a-4e1b-8991-e4d2a5133725" containerName="authorino"
Jul 03 14:20:04.063113 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.063096    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f"
Jul 03 14:20:04.065785 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.065755    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xhx89\""
Jul 03 14:20:04.066606 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.066589    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 03 14:20:04.066708 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.066688    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 03 14:20:04.066788 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.066771    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 03 14:20:04.077792 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.077768    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f"]
Jul 03 14:20:04.160299 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.160258    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9vfl\" (UniqueName: \"kubernetes.io/projected/4210ce8e-6505-4f3f-b726-ec0e2292e845-kube-api-access-q9vfl\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xzg2f\" (UID: \"4210ce8e-6505-4f3f-b726-ec0e2292e845\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f"
Jul 03 14:20:04.160513 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.160311    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4210ce8e-6505-4f3f-b726-ec0e2292e845-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xzg2f\" (UID: \"4210ce8e-6505-4f3f-b726-ec0e2292e845\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f"
Jul 03 14:20:04.160513 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.160445    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4210ce8e-6505-4f3f-b726-ec0e2292e845-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xzg2f\" (UID: \"4210ce8e-6505-4f3f-b726-ec0e2292e845\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f"
Jul 03 14:20:04.160513 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.160489    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4210ce8e-6505-4f3f-b726-ec0e2292e845-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xzg2f\" (UID: \"4210ce8e-6505-4f3f-b726-ec0e2292e845\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f"
Jul 03 14:20:04.160513 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.160508    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4210ce8e-6505-4f3f-b726-ec0e2292e845-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xzg2f\" (UID: \"4210ce8e-6505-4f3f-b726-ec0e2292e845\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f"
Jul 03 14:20:04.160704 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.160575    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4210ce8e-6505-4f3f-b726-ec0e2292e845-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xzg2f\" (UID: \"4210ce8e-6505-4f3f-b726-ec0e2292e845\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f"
Jul 03 14:20:04.262150 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.262114    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9vfl\" (UniqueName: \"kubernetes.io/projected/4210ce8e-6505-4f3f-b726-ec0e2292e845-kube-api-access-q9vfl\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xzg2f\" (UID: \"4210ce8e-6505-4f3f-b726-ec0e2292e845\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f"
Jul 03 14:20:04.262329 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.262164    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4210ce8e-6505-4f3f-b726-ec0e2292e845-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xzg2f\" (UID: \"4210ce8e-6505-4f3f-b726-ec0e2292e845\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f"
Jul 03 14:20:04.262329 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.262231    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4210ce8e-6505-4f3f-b726-ec0e2292e845-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xzg2f\" (UID: \"4210ce8e-6505-4f3f-b726-ec0e2292e845\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f"
Jul 03 14:20:04.262329 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.262278    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4210ce8e-6505-4f3f-b726-ec0e2292e845-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xzg2f\" (UID: \"4210ce8e-6505-4f3f-b726-ec0e2292e845\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f"
Jul 03 14:20:04.262329 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.262303    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4210ce8e-6505-4f3f-b726-ec0e2292e845-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xzg2f\" (UID: \"4210ce8e-6505-4f3f-b726-ec0e2292e845\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f"
Jul 03 14:20:04.262522 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.262348    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4210ce8e-6505-4f3f-b726-ec0e2292e845-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xzg2f\" (UID: \"4210ce8e-6505-4f3f-b726-ec0e2292e845\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f"
Jul 03 14:20:04.262682 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.262643    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4210ce8e-6505-4f3f-b726-ec0e2292e845-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xzg2f\" (UID: \"4210ce8e-6505-4f3f-b726-ec0e2292e845\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f"
Jul 03 14:20:04.262783 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.262649    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4210ce8e-6505-4f3f-b726-ec0e2292e845-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xzg2f\" (UID: \"4210ce8e-6505-4f3f-b726-ec0e2292e845\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f"
Jul 03 14:20:04.262783 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.262734    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4210ce8e-6505-4f3f-b726-ec0e2292e845-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xzg2f\" (UID: \"4210ce8e-6505-4f3f-b726-ec0e2292e845\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f"
Jul 03 14:20:04.264634 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.264611    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4210ce8e-6505-4f3f-b726-ec0e2292e845-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xzg2f\" (UID: \"4210ce8e-6505-4f3f-b726-ec0e2292e845\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f"
Jul 03 14:20:04.264794 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.264777    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4210ce8e-6505-4f3f-b726-ec0e2292e845-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xzg2f\" (UID: \"4210ce8e-6505-4f3f-b726-ec0e2292e845\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f"
Jul 03 14:20:04.272228 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.272206    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9vfl\" (UniqueName: \"kubernetes.io/projected/4210ce8e-6505-4f3f-b726-ec0e2292e845-kube-api-access-q9vfl\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xzg2f\" (UID: \"4210ce8e-6505-4f3f-b726-ec0e2292e845\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f"
Jul 03 14:20:04.373940 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.373900    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f"
Jul 03 14:20:04.523436 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:04.521034    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f"]
Jul 03 14:20:05.270401 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:05.270370    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:20:05.307845 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:05.307812    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f" event={"ID":"4210ce8e-6505-4f3f-b726-ec0e2292e845","Type":"ContainerStarted","Data":"e722918a693562e22bcd468bf231f8bc334f748cb19b610c3b911c890126a3d4"}
Jul 03 14:20:06.314541 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:06.314504    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f" event={"ID":"4210ce8e-6505-4f3f-b726-ec0e2292e845","Type":"ContainerStarted","Data":"bddd72144bb378c7cc940b644248160ebc5b02c530a58d78f54edaec25e4c491"}
Jul 03 14:20:06.337516 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:06.337456    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f" podStartSLOduration=1.136332342 podStartE2EDuration="2.337439427s" totalImagesPullingTime="1.201107085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:20:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:20:06.334640571 +0000 UTC m=+719.784774024" watchObservedRunningTime="2026-07-03 14:20:06.337439427 +0000 UTC m=+719.787572845"
Jul 03 14:20:10.863055 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:10.863018    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:20:14.375079 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:14.375045    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f"
Jul 03 14:20:14.375453 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:14.375105    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f"
Jul 03 14:20:14.387419 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:14.387399    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f"
Jul 03 14:20:15.366847 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:15.366813    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xzg2f"
Jul 03 14:20:16.655149 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:16.655111    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:20:30.637294 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:30.637263    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5"]
Jul 03 14:20:30.640737 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:30.640721    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5"
Jul 03 14:20:30.648834 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:30.648806    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 03 14:20:30.656687 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:30.656665    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5"]
Jul 03 14:20:30.710934 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:30.710893    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhxm2\" (UniqueName: \"kubernetes.io/projected/f6281067-c0ac-44ee-af48-34cbea8597c2-kube-api-access-nhxm2\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5\" (UID: \"f6281067-c0ac-44ee-af48-34cbea8597c2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5"
Jul 03 14:20:30.710934 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:30.710939    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6281067-c0ac-44ee-af48-34cbea8597c2-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5\" (UID: \"f6281067-c0ac-44ee-af48-34cbea8597c2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5"
Jul 03 14:20:30.711188 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:30.711052    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6281067-c0ac-44ee-af48-34cbea8597c2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5\" (UID: \"f6281067-c0ac-44ee-af48-34cbea8597c2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5"
Jul 03 14:20:30.711188 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:30.711099    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6281067-c0ac-44ee-af48-34cbea8597c2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5\" (UID: \"f6281067-c0ac-44ee-af48-34cbea8597c2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5"
Jul 03 14:20:30.711188 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:30.711149    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6281067-c0ac-44ee-af48-34cbea8597c2-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5\" (UID: \"f6281067-c0ac-44ee-af48-34cbea8597c2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5"
Jul 03 14:20:30.711188 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:30.711169    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6281067-c0ac-44ee-af48-34cbea8597c2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5\" (UID: \"f6281067-c0ac-44ee-af48-34cbea8597c2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5"
Jul 03 14:20:30.811817 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:30.811783    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhxm2\" (UniqueName: \"kubernetes.io/projected/f6281067-c0ac-44ee-af48-34cbea8597c2-kube-api-access-nhxm2\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5\" (UID: \"f6281067-c0ac-44ee-af48-34cbea8597c2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5"
Jul 03 14:20:30.812024 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:30.811822    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6281067-c0ac-44ee-af48-34cbea8597c2-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5\" (UID: \"f6281067-c0ac-44ee-af48-34cbea8597c2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5"
Jul 03 14:20:30.812024 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:30.811850    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6281067-c0ac-44ee-af48-34cbea8597c2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5\" (UID: \"f6281067-c0ac-44ee-af48-34cbea8597c2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5"
Jul 03 14:20:30.812024 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:30.811900    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6281067-c0ac-44ee-af48-34cbea8597c2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5\" (UID: \"f6281067-c0ac-44ee-af48-34cbea8597c2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5"
Jul 03 14:20:30.812024 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:30.811940    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6281067-c0ac-44ee-af48-34cbea8597c2-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5\" (UID: \"f6281067-c0ac-44ee-af48-34cbea8597c2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5"
Jul 03 14:20:30.812238 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:30.812087    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6281067-c0ac-44ee-af48-34cbea8597c2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5\" (UID: \"f6281067-c0ac-44ee-af48-34cbea8597c2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5"
Jul 03 14:20:30.812335 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:30.812312    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6281067-c0ac-44ee-af48-34cbea8597c2-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5\" (UID: \"f6281067-c0ac-44ee-af48-34cbea8597c2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5"
Jul 03 14:20:30.812398 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:30.812340    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6281067-c0ac-44ee-af48-34cbea8597c2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5\" (UID: \"f6281067-c0ac-44ee-af48-34cbea8597c2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5"
Jul 03 14:20:30.812449 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:30.812402    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6281067-c0ac-44ee-af48-34cbea8597c2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5\" (UID: \"f6281067-c0ac-44ee-af48-34cbea8597c2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5"
Jul 03 14:20:30.814161 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:30.814132    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6281067-c0ac-44ee-af48-34cbea8597c2-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5\" (UID: \"f6281067-c0ac-44ee-af48-34cbea8597c2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5"
Jul 03 14:20:30.814506 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:30.814485    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6281067-c0ac-44ee-af48-34cbea8597c2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5\" (UID: \"f6281067-c0ac-44ee-af48-34cbea8597c2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5"
Jul 03 14:20:30.825550 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:30.825522    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhxm2\" (UniqueName: \"kubernetes.io/projected/f6281067-c0ac-44ee-af48-34cbea8597c2-kube-api-access-nhxm2\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5\" (UID: \"f6281067-c0ac-44ee-af48-34cbea8597c2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5"
Jul 03 14:20:30.949656 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:30.949568    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5"
Jul 03 14:20:31.083684 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:31.083656    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5"]
Jul 03 14:20:31.085738 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:20:31.085707    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6281067_c0ac_44ee_af48_34cbea8597c2.slice/crio-0f61711cf1002799a5fd7e7d79f54402b7e104774bece35a96d59ac461407eb7 WatchSource:0}: Error finding container 0f61711cf1002799a5fd7e7d79f54402b7e104774bece35a96d59ac461407eb7: Status 404 returned error can't find the container with id 0f61711cf1002799a5fd7e7d79f54402b7e104774bece35a96d59ac461407eb7
Jul 03 14:20:31.087076 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:31.087056    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 03 14:20:31.431985 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:31.431944    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5" event={"ID":"f6281067-c0ac-44ee-af48-34cbea8597c2","Type":"ContainerStarted","Data":"b492a2e574bbbbf499fc4358516417f55628be6fad074a716927d3a1332693e0"}
Jul 03 14:20:31.432162 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:31.432012    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5" event={"ID":"f6281067-c0ac-44ee-af48-34cbea8597c2","Type":"ContainerStarted","Data":"0f61711cf1002799a5fd7e7d79f54402b7e104774bece35a96d59ac461407eb7"}
Jul 03 14:20:31.455627 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:31.455574    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5" podStartSLOduration=1.26429889 podStartE2EDuration="1.455557983s" totalImagesPullingTime="191.259093ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:20:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:20:31.452136006 +0000 UTC m=+744.902269446" watchObservedRunningTime="2026-07-03 14:20:31.455557983 +0000 UTC m=+744.905691434"
Jul 03 14:20:32.260985 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:32.260942    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:20:40.950021 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:40.949962    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5"
Jul 03 14:20:40.950435 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:40.950035    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5"
Jul 03 14:20:40.962550 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:40.962524    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5"
Jul 03 14:20:41.497175 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:20:41.497141    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5"
Jul 03 14:22:15.951375 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:22:15.951343    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:22:28.244994 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:22:28.244940    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:22:36.547615 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:22:36.547576    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:22:46.245880 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:22:46.245846    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:22:57.255447 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:22:57.255414    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:23:05.311084 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:23:05.310992    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:23:27.219674 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:23:27.219637    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:23:40.680237 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:23:40.680200    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:24:17.451908 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:24:17.451873    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:24:26.370140 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:24:26.370106    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:24:35.662213 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:24:35.662177    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:24:44.064772 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:24:44.064736    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:24:55.265811 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:24:55.265770    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:25:05.561644 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:25:05.561605    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:25:16.254023 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:25:16.253986    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:25:24.487633 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:25:24.487597    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:25:29.859665 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:25:29.859632    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:25:40.545334 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:25:40.545298    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:25:51.655148 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:25:51.655114    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:26:03.840528 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:26:03.840447    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:26:27.751427 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:26:27.751394    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:26:38.448638 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:26:38.448607    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:26:52.836911 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:26:52.836873    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:27:03.844104 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:27:03.844070    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:27:14.344894 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:27:14.344861    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:27:24.137290 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:27:24.137246    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:27:34.045477 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:27:34.045396    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:29:20.447022 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:29:20.446906    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:29:32.537414 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:29:32.537378    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:29:41.340107 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:29:41.340069    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:29:51.046314 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:29:51.046269    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:30:00.540933 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:30:00.540891    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:30:08.960276 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:30:08.960233    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:30:19.542607 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:30:19.542568    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:30:30.456681 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:30:30.456601    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:30:39.541178 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:30:39.541137    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:30:50.482543 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:30:50.482505    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:31:02.456454 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:31:02.456414    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:31:09.735811 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:31:09.735775    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:31:28.240923 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:31:28.240887    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:31:36.847952 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:31:36.847920    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:31:46.748887 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:31:46.748852    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:31:58.944584 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:31:58.944549    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:32:15.147539 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:32:15.147505    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:32:25.438085 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:32:25.438043    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:32:34.745442 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:32:34.745401    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:32:45.248371 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:32:45.248335    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:32:55.257621 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:32:55.257577    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:33:04.942639 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:33:04.942598    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:33:26.745497 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:33:26.745461    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:33:36.154907 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:33:36.154819    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:33:46.358741 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:33:46.358699    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:33:59.447781 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:33:59.447745    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:34:08.263638 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:34:08.263593    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:34:20.649356 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:34:20.649320    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:34:32.755863 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:34:32.755829    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:34:45.246441 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:34:45.246408    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:34:59.946619 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:34:59.946543    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:35:09.268471 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:35:09.268380    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:35:31.538506 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:35:31.538469    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:35:40.633193 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:35:40.633158    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:37:01.845445 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:37:01.845404    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:37:10.937899 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:37:10.937865    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:37:22.898418 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:37:22.898385    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:37:34.739627 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:37:34.739588    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:38:05.339352 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:38:05.339313    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:38:17.642403 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:38:17.642369    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:38:33.747350 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:38:33.747315    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:38:48.139042 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:38:48.139000    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:39:27.341347 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:39:27.341310    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:39:40.349321 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:39:40.349226    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:39:50.048276 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:39:50.048237    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:40:32.946026 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:40:32.945989    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:40:42.242884 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:40:42.242839    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:41:03.950930 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:03.950846    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"]
Jul 03 14:41:03.954513 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:03.954490    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"
Jul 03 14:41:03.956868 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:03.956846    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-2d96f7\"/\"kube-root-ca.crt\""
Jul 03 14:41:03.957722 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:03.957693    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-2d96f7\"/\"test-model-2d96f7-kserve-self-signed-certs\""
Jul 03 14:41:03.957847 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:03.957700    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-2d96f7\"/\"default-dockercfg-km642\""
Jul 03 14:41:03.957847 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:03.957741    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-2d96f7\"/\"openshift-service-ca.crt\""
Jul 03 14:41:03.964375 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:03.964349    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"]
Jul 03 14:41:04.039602 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:04.039565    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/446afa9f-d004-4073-96b7-e7ef9412d278-home\") pod \"test-model-2d96f7-kserve-dbf8db6c7-q59mw\" (UID: \"446afa9f-d004-4073-96b7-e7ef9412d278\") " pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"
Jul 03 14:41:04.039785 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:04.039620    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/446afa9f-d004-4073-96b7-e7ef9412d278-model-cache\") pod \"test-model-2d96f7-kserve-dbf8db6c7-q59mw\" (UID: \"446afa9f-d004-4073-96b7-e7ef9412d278\") " pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"
Jul 03 14:41:04.039785 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:04.039723    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/446afa9f-d004-4073-96b7-e7ef9412d278-tls-certs\") pod \"test-model-2d96f7-kserve-dbf8db6c7-q59mw\" (UID: \"446afa9f-d004-4073-96b7-e7ef9412d278\") " pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"
Jul 03 14:41:04.039785 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:04.039767    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/446afa9f-d004-4073-96b7-e7ef9412d278-dshm\") pod \"test-model-2d96f7-kserve-dbf8db6c7-q59mw\" (UID: \"446afa9f-d004-4073-96b7-e7ef9412d278\") " pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"
Jul 03 14:41:04.039895 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:04.039802    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24kfb\" (UniqueName: \"kubernetes.io/projected/446afa9f-d004-4073-96b7-e7ef9412d278-kube-api-access-24kfb\") pod \"test-model-2d96f7-kserve-dbf8db6c7-q59mw\" (UID: \"446afa9f-d004-4073-96b7-e7ef9412d278\") " pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"
Jul 03 14:41:04.039895 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:04.039853    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/446afa9f-d004-4073-96b7-e7ef9412d278-tmp-dir\") pod \"test-model-2d96f7-kserve-dbf8db6c7-q59mw\" (UID: \"446afa9f-d004-4073-96b7-e7ef9412d278\") " pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"
Jul 03 14:41:04.140858 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:04.140816    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/446afa9f-d004-4073-96b7-e7ef9412d278-dshm\") pod \"test-model-2d96f7-kserve-dbf8db6c7-q59mw\" (UID: \"446afa9f-d004-4073-96b7-e7ef9412d278\") " pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"
Jul 03 14:41:04.140858 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:04.140864    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24kfb\" (UniqueName: \"kubernetes.io/projected/446afa9f-d004-4073-96b7-e7ef9412d278-kube-api-access-24kfb\") pod \"test-model-2d96f7-kserve-dbf8db6c7-q59mw\" (UID: \"446afa9f-d004-4073-96b7-e7ef9412d278\") " pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"
Jul 03 14:41:04.141120 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:04.140908    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/446afa9f-d004-4073-96b7-e7ef9412d278-tmp-dir\") pod \"test-model-2d96f7-kserve-dbf8db6c7-q59mw\" (UID: \"446afa9f-d004-4073-96b7-e7ef9412d278\") " pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"
Jul 03 14:41:04.141120 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:04.141023    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/446afa9f-d004-4073-96b7-e7ef9412d278-home\") pod \"test-model-2d96f7-kserve-dbf8db6c7-q59mw\" (UID: \"446afa9f-d004-4073-96b7-e7ef9412d278\") " pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"
Jul 03 14:41:04.141231 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:04.141123    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/446afa9f-d004-4073-96b7-e7ef9412d278-model-cache\") pod \"test-model-2d96f7-kserve-dbf8db6c7-q59mw\" (UID: \"446afa9f-d004-4073-96b7-e7ef9412d278\") " pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"
Jul 03 14:41:04.141231 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:04.141217    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/446afa9f-d004-4073-96b7-e7ef9412d278-tls-certs\") pod \"test-model-2d96f7-kserve-dbf8db6c7-q59mw\" (UID: \"446afa9f-d004-4073-96b7-e7ef9412d278\") " pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"
Jul 03 14:41:04.141380 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:04.141359    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/446afa9f-d004-4073-96b7-e7ef9412d278-tmp-dir\") pod \"test-model-2d96f7-kserve-dbf8db6c7-q59mw\" (UID: \"446afa9f-d004-4073-96b7-e7ef9412d278\") " pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"
Jul 03 14:41:04.141537 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:04.141512    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/446afa9f-d004-4073-96b7-e7ef9412d278-model-cache\") pod \"test-model-2d96f7-kserve-dbf8db6c7-q59mw\" (UID: \"446afa9f-d004-4073-96b7-e7ef9412d278\") " pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"
Jul 03 14:41:04.141762 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:04.141741    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/446afa9f-d004-4073-96b7-e7ef9412d278-home\") pod \"test-model-2d96f7-kserve-dbf8db6c7-q59mw\" (UID: \"446afa9f-d004-4073-96b7-e7ef9412d278\") " pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"
Jul 03 14:41:04.143278 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:04.143259    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/446afa9f-d004-4073-96b7-e7ef9412d278-dshm\") pod \"test-model-2d96f7-kserve-dbf8db6c7-q59mw\" (UID: \"446afa9f-d004-4073-96b7-e7ef9412d278\") " pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"
Jul 03 14:41:04.143587 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:04.143568    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/446afa9f-d004-4073-96b7-e7ef9412d278-tls-certs\") pod \"test-model-2d96f7-kserve-dbf8db6c7-q59mw\" (UID: \"446afa9f-d004-4073-96b7-e7ef9412d278\") " pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"
Jul 03 14:41:04.148868 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:04.148842    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24kfb\" (UniqueName: \"kubernetes.io/projected/446afa9f-d004-4073-96b7-e7ef9412d278-kube-api-access-24kfb\") pod \"test-model-2d96f7-kserve-dbf8db6c7-q59mw\" (UID: \"446afa9f-d004-4073-96b7-e7ef9412d278\") " pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"
Jul 03 14:41:04.265291 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:04.265195    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"
Jul 03 14:41:04.412819 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:41:04.412786    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod446afa9f_d004_4073_96b7_e7ef9412d278.slice/crio-ab38c2bca8c87716990d8b8ad6052ea8a5a9f3668c82fbaa3c2fb47ae1151618 WatchSource:0}: Error finding container ab38c2bca8c87716990d8b8ad6052ea8a5a9f3668c82fbaa3c2fb47ae1151618: Status 404 returned error can't find the container with id ab38c2bca8c87716990d8b8ad6052ea8a5a9f3668c82fbaa3c2fb47ae1151618
Jul 03 14:41:04.413335 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:04.413233    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"]
Jul 03 14:41:05.387806 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:05.387766    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw" event={"ID":"446afa9f-d004-4073-96b7-e7ef9412d278","Type":"ContainerStarted","Data":"8a8c2c7264c63a429f402ee4e05d3bf39d4e1a1555b06dc495b9e5856c585bbf"}
Jul 03 14:41:05.388213 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:05.387813    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw" event={"ID":"446afa9f-d004-4073-96b7-e7ef9412d278","Type":"ContainerStarted","Data":"ab38c2bca8c87716990d8b8ad6052ea8a5a9f3668c82fbaa3c2fb47ae1151618"}
Jul 03 14:41:05.405271 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:05.405216    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw" podStartSLOduration=2.405195591 podStartE2EDuration="2.405195591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:41:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:41:05.403621408 +0000 UTC m=+1978.853754864" watchObservedRunningTime="2026-07-03 14:41:05.405195591 +0000 UTC m=+1978.855329032"
Jul 03 14:41:07.940209 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:07.940167    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:41:14.265549 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:14.265516    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"
Jul 03 14:41:14.266024 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:14.265568    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"
Jul 03 14:41:14.278069 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:14.278046    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"
Jul 03 14:41:14.448078 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:14.448050    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"
Jul 03 14:41:43.743883 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:43.743843    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"]
Jul 03 14:41:43.748092 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:43.748072    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"
Jul 03 14:41:43.750524 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:43.750501    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-7195a2\"/\"default-dockercfg-zvxmx\""
Jul 03 14:41:43.751787 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:43.751760    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-7195a2\"/\"kube-root-ca.crt\""
Jul 03 14:41:43.751916 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:43.751797    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-7195a2\"/\"openshift-service-ca.crt\""
Jul 03 14:41:43.751916 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:43.751818    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-7195a2\"/\"test-model-7195a2-kserve-self-signed-certs\""
Jul 03 14:41:43.757964 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:43.757937    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"]
Jul 03 14:41:43.912846 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:43.912816    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee9db423-cc69-4d5f-9b49-a5488772d405-tls-certs\") pod \"test-model-7195a2-kserve-bd8c7ffbc-mv68b\" (UID: \"ee9db423-cc69-4d5f-9b49-a5488772d405\") " pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"
Jul 03 14:41:43.912846 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:43.912849    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee9db423-cc69-4d5f-9b49-a5488772d405-home\") pod \"test-model-7195a2-kserve-bd8c7ffbc-mv68b\" (UID: \"ee9db423-cc69-4d5f-9b49-a5488772d405\") " pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"
Jul 03 14:41:43.913128 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:43.913018    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgtdq\" (UniqueName: \"kubernetes.io/projected/ee9db423-cc69-4d5f-9b49-a5488772d405-kube-api-access-lgtdq\") pod \"test-model-7195a2-kserve-bd8c7ffbc-mv68b\" (UID: \"ee9db423-cc69-4d5f-9b49-a5488772d405\") " pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"
Jul 03 14:41:43.913128 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:43.913057    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee9db423-cc69-4d5f-9b49-a5488772d405-model-cache\") pod \"test-model-7195a2-kserve-bd8c7ffbc-mv68b\" (UID: \"ee9db423-cc69-4d5f-9b49-a5488772d405\") " pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"
Jul 03 14:41:43.913128 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:43.913108    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee9db423-cc69-4d5f-9b49-a5488772d405-dshm\") pod \"test-model-7195a2-kserve-bd8c7ffbc-mv68b\" (UID: \"ee9db423-cc69-4d5f-9b49-a5488772d405\") " pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"
Jul 03 14:41:43.913288 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:43.913186    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee9db423-cc69-4d5f-9b49-a5488772d405-tmp-dir\") pod \"test-model-7195a2-kserve-bd8c7ffbc-mv68b\" (UID: \"ee9db423-cc69-4d5f-9b49-a5488772d405\") " pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"
Jul 03 14:41:44.014200 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:44.014121    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgtdq\" (UniqueName: \"kubernetes.io/projected/ee9db423-cc69-4d5f-9b49-a5488772d405-kube-api-access-lgtdq\") pod \"test-model-7195a2-kserve-bd8c7ffbc-mv68b\" (UID: \"ee9db423-cc69-4d5f-9b49-a5488772d405\") " pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"
Jul 03 14:41:44.014200 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:44.014153    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee9db423-cc69-4d5f-9b49-a5488772d405-model-cache\") pod \"test-model-7195a2-kserve-bd8c7ffbc-mv68b\" (UID: \"ee9db423-cc69-4d5f-9b49-a5488772d405\") " pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"
Jul 03 14:41:44.014200 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:44.014191    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee9db423-cc69-4d5f-9b49-a5488772d405-dshm\") pod \"test-model-7195a2-kserve-bd8c7ffbc-mv68b\" (UID: \"ee9db423-cc69-4d5f-9b49-a5488772d405\") " pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"
Jul 03 14:41:44.014486 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:44.014235    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee9db423-cc69-4d5f-9b49-a5488772d405-tmp-dir\") pod \"test-model-7195a2-kserve-bd8c7ffbc-mv68b\" (UID: \"ee9db423-cc69-4d5f-9b49-a5488772d405\") " pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"
Jul 03 14:41:44.014486 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:44.014256    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee9db423-cc69-4d5f-9b49-a5488772d405-tls-certs\") pod \"test-model-7195a2-kserve-bd8c7ffbc-mv68b\" (UID: \"ee9db423-cc69-4d5f-9b49-a5488772d405\") " pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"
Jul 03 14:41:44.014486 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:44.014271    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee9db423-cc69-4d5f-9b49-a5488772d405-home\") pod \"test-model-7195a2-kserve-bd8c7ffbc-mv68b\" (UID: \"ee9db423-cc69-4d5f-9b49-a5488772d405\") " pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"
Jul 03 14:41:44.014647 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:44.014605    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee9db423-cc69-4d5f-9b49-a5488772d405-model-cache\") pod \"test-model-7195a2-kserve-bd8c7ffbc-mv68b\" (UID: \"ee9db423-cc69-4d5f-9b49-a5488772d405\") " pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"
Jul 03 14:41:44.014647 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:44.014614    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee9db423-cc69-4d5f-9b49-a5488772d405-tmp-dir\") pod \"test-model-7195a2-kserve-bd8c7ffbc-mv68b\" (UID: \"ee9db423-cc69-4d5f-9b49-a5488772d405\") " pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"
Jul 03 14:41:44.014758 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:44.014740    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee9db423-cc69-4d5f-9b49-a5488772d405-home\") pod \"test-model-7195a2-kserve-bd8c7ffbc-mv68b\" (UID: \"ee9db423-cc69-4d5f-9b49-a5488772d405\") " pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"
Jul 03 14:41:44.016460 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:44.016442    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee9db423-cc69-4d5f-9b49-a5488772d405-dshm\") pod \"test-model-7195a2-kserve-bd8c7ffbc-mv68b\" (UID: \"ee9db423-cc69-4d5f-9b49-a5488772d405\") " pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"
Jul 03 14:41:44.016751 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:44.016730    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee9db423-cc69-4d5f-9b49-a5488772d405-tls-certs\") pod \"test-model-7195a2-kserve-bd8c7ffbc-mv68b\" (UID: \"ee9db423-cc69-4d5f-9b49-a5488772d405\") " pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"
Jul 03 14:41:44.022383 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:44.022363    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgtdq\" (UniqueName: \"kubernetes.io/projected/ee9db423-cc69-4d5f-9b49-a5488772d405-kube-api-access-lgtdq\") pod \"test-model-7195a2-kserve-bd8c7ffbc-mv68b\" (UID: \"ee9db423-cc69-4d5f-9b49-a5488772d405\") " pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"
Jul 03 14:41:44.059631 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:44.059596    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"
Jul 03 14:41:44.190939 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:44.190911    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"]
Jul 03 14:41:44.193399 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:41:44.193367    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee9db423_cc69_4d5f_9b49_a5488772d405.slice/crio-3775ecb5edc5a448d17417d8f153acc573c4cfe95ad12fbc7aa6ec5f29929e14 WatchSource:0}: Error finding container 3775ecb5edc5a448d17417d8f153acc573c4cfe95ad12fbc7aa6ec5f29929e14: Status 404 returned error can't find the container with id 3775ecb5edc5a448d17417d8f153acc573c4cfe95ad12fbc7aa6ec5f29929e14
Jul 03 14:41:44.576912 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:44.576873    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b" event={"ID":"ee9db423-cc69-4d5f-9b49-a5488772d405","Type":"ContainerStarted","Data":"ec4cb6cbca8ad80a1f9b041dded5d8abb543a898a6cec96b52b2677fedfa0fa9"}
Jul 03 14:41:44.576912 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:44.576920    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b" event={"ID":"ee9db423-cc69-4d5f-9b49-a5488772d405","Type":"ContainerStarted","Data":"3775ecb5edc5a448d17417d8f153acc573c4cfe95ad12fbc7aa6ec5f29929e14"}
Jul 03 14:41:44.595509 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:44.595462    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b" podStartSLOduration=1.595445821 podStartE2EDuration="1.595445821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:41:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:41:44.593764384 +0000 UTC m=+2018.043897843" watchObservedRunningTime="2026-07-03 14:41:44.595445821 +0000 UTC m=+2018.045579260"
Jul 03 14:41:48.235249 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:48.235209    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:41:54.060632 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:54.060592    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"
Jul 03 14:41:54.061096 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:54.060821    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"
Jul 03 14:41:54.072907 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:54.072885    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"
Jul 03 14:41:54.636059 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:41:54.636032    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"
Jul 03 14:42:02.534547 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:02.534503    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:42:28.238341 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:28.238309    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"]
Jul 03 14:42:29.221108 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:29.221021    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"]
Jul 03 14:42:32.542929 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:32.542895    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:42:43.258240 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:43.258175    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw" podUID="446afa9f-d004-4073-96b7-e7ef9412d278" containerName="main" containerID="cri-o://8a8c2c7264c63a429f402ee4e05d3bf39d4e1a1555b06dc495b9e5856c585bbf" gracePeriod=45
Jul 03 14:42:43.505842 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:43.505816    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"
Jul 03 14:42:43.594312 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:43.594210    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/446afa9f-d004-4073-96b7-e7ef9412d278-dshm\") pod \"446afa9f-d004-4073-96b7-e7ef9412d278\" (UID: \"446afa9f-d004-4073-96b7-e7ef9412d278\") "
Jul 03 14:42:43.594312 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:43.594274    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/446afa9f-d004-4073-96b7-e7ef9412d278-model-cache\") pod \"446afa9f-d004-4073-96b7-e7ef9412d278\" (UID: \"446afa9f-d004-4073-96b7-e7ef9412d278\") "
Jul 03 14:42:43.594312 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:43.594300    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/446afa9f-d004-4073-96b7-e7ef9412d278-tmp-dir\") pod \"446afa9f-d004-4073-96b7-e7ef9412d278\" (UID: \"446afa9f-d004-4073-96b7-e7ef9412d278\") "
Jul 03 14:42:43.594618 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:43.594330    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24kfb\" (UniqueName: \"kubernetes.io/projected/446afa9f-d004-4073-96b7-e7ef9412d278-kube-api-access-24kfb\") pod \"446afa9f-d004-4073-96b7-e7ef9412d278\" (UID: \"446afa9f-d004-4073-96b7-e7ef9412d278\") "
Jul 03 14:42:43.594618 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:43.594364    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/446afa9f-d004-4073-96b7-e7ef9412d278-home\") pod \"446afa9f-d004-4073-96b7-e7ef9412d278\" (UID: \"446afa9f-d004-4073-96b7-e7ef9412d278\") "
Jul 03 14:42:43.594618 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:43.594397    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/446afa9f-d004-4073-96b7-e7ef9412d278-tls-certs\") pod \"446afa9f-d004-4073-96b7-e7ef9412d278\" (UID: \"446afa9f-d004-4073-96b7-e7ef9412d278\") "
Jul 03 14:42:43.594618 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:43.594583    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/446afa9f-d004-4073-96b7-e7ef9412d278-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "446afa9f-d004-4073-96b7-e7ef9412d278" (UID: "446afa9f-d004-4073-96b7-e7ef9412d278"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:42:43.594826 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:43.594609    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/446afa9f-d004-4073-96b7-e7ef9412d278-model-cache" (OuterVolumeSpecName: "model-cache") pod "446afa9f-d004-4073-96b7-e7ef9412d278" (UID: "446afa9f-d004-4073-96b7-e7ef9412d278"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:42:43.594826 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:43.594658    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/446afa9f-d004-4073-96b7-e7ef9412d278-home" (OuterVolumeSpecName: "home") pod "446afa9f-d004-4073-96b7-e7ef9412d278" (UID: "446afa9f-d004-4073-96b7-e7ef9412d278"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:42:43.594899 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:43.594842    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/446afa9f-d004-4073-96b7-e7ef9412d278-model-cache\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:42:43.594899 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:43.594861    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/446afa9f-d004-4073-96b7-e7ef9412d278-tmp-dir\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:42:43.594899 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:43.594875    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/446afa9f-d004-4073-96b7-e7ef9412d278-home\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:42:43.596552 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:43.596519    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/446afa9f-d004-4073-96b7-e7ef9412d278-dshm" (OuterVolumeSpecName: "dshm") pod "446afa9f-d004-4073-96b7-e7ef9412d278" (UID: "446afa9f-d004-4073-96b7-e7ef9412d278"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:42:43.596656 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:43.596585    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/446afa9f-d004-4073-96b7-e7ef9412d278-kube-api-access-24kfb" (OuterVolumeSpecName: "kube-api-access-24kfb") pod "446afa9f-d004-4073-96b7-e7ef9412d278" (UID: "446afa9f-d004-4073-96b7-e7ef9412d278"). InnerVolumeSpecName "kube-api-access-24kfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:42:43.596656 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:43.596587    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/446afa9f-d004-4073-96b7-e7ef9412d278-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "446afa9f-d004-4073-96b7-e7ef9412d278" (UID: "446afa9f-d004-4073-96b7-e7ef9412d278"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:42:43.695945 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:43.695903    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/446afa9f-d004-4073-96b7-e7ef9412d278-dshm\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:42:43.695945 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:43.695938    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24kfb\" (UniqueName: \"kubernetes.io/projected/446afa9f-d004-4073-96b7-e7ef9412d278-kube-api-access-24kfb\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:42:43.695945 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:43.695954    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/446afa9f-d004-4073-96b7-e7ef9412d278-tls-certs\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:42:43.861726 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:43.861696    2563 generic.go:356] "Generic (PLEG): container finished" podID="446afa9f-d004-4073-96b7-e7ef9412d278" containerID="8a8c2c7264c63a429f402ee4e05d3bf39d4e1a1555b06dc495b9e5856c585bbf" exitCode=0
Jul 03 14:42:43.861889 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:43.861737    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"
Jul 03 14:42:43.861889 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:43.861777    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw" event={"ID":"446afa9f-d004-4073-96b7-e7ef9412d278","Type":"ContainerDied","Data":"8a8c2c7264c63a429f402ee4e05d3bf39d4e1a1555b06dc495b9e5856c585bbf"}
Jul 03 14:42:43.861889 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:43.861820    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw" event={"ID":"446afa9f-d004-4073-96b7-e7ef9412d278","Type":"ContainerDied","Data":"ab38c2bca8c87716990d8b8ad6052ea8a5a9f3668c82fbaa3c2fb47ae1151618"}
Jul 03 14:42:43.861889 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:43.861838    2563 scope.go:117] "RemoveContainer" containerID="8a8c2c7264c63a429f402ee4e05d3bf39d4e1a1555b06dc495b9e5856c585bbf"
Jul 03 14:42:43.873064 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:43.873043    2563 scope.go:117] "RemoveContainer" containerID="8a8c2c7264c63a429f402ee4e05d3bf39d4e1a1555b06dc495b9e5856c585bbf"
Jul 03 14:42:43.873354 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:42:43.873334    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a8c2c7264c63a429f402ee4e05d3bf39d4e1a1555b06dc495b9e5856c585bbf\": container with ID starting with 8a8c2c7264c63a429f402ee4e05d3bf39d4e1a1555b06dc495b9e5856c585bbf not found: ID does not exist" containerID="8a8c2c7264c63a429f402ee4e05d3bf39d4e1a1555b06dc495b9e5856c585bbf"
Jul 03 14:42:43.873404 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:43.873364    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a8c2c7264c63a429f402ee4e05d3bf39d4e1a1555b06dc495b9e5856c585bbf"} err="failed to get container status \"8a8c2c7264c63a429f402ee4e05d3bf39d4e1a1555b06dc495b9e5856c585bbf\": rpc error: code = NotFound desc = could not find container \"8a8c2c7264c63a429f402ee4e05d3bf39d4e1a1555b06dc495b9e5856c585bbf\": container with ID starting with 8a8c2c7264c63a429f402ee4e05d3bf39d4e1a1555b06dc495b9e5856c585bbf not found: ID does not exist"
Jul 03 14:42:43.885254 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:43.885231    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"]
Jul 03 14:42:43.889033 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:43.889014    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-2d96f7/test-model-2d96f7-kserve-dbf8db6c7-q59mw"]
Jul 03 14:42:44.241434 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:44.241352    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b" podUID="ee9db423-cc69-4d5f-9b49-a5488772d405" containerName="main" containerID="cri-o://ec4cb6cbca8ad80a1f9b041dded5d8abb543a898a6cec96b52b2677fedfa0fa9" gracePeriod=45
Jul 03 14:42:44.499246 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:44.499192    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"
Jul 03 14:42:44.604325 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:44.604295    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee9db423-cc69-4d5f-9b49-a5488772d405-tls-certs\") pod \"ee9db423-cc69-4d5f-9b49-a5488772d405\" (UID: \"ee9db423-cc69-4d5f-9b49-a5488772d405\") "
Jul 03 14:42:44.604515 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:44.604337    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee9db423-cc69-4d5f-9b49-a5488772d405-home\") pod \"ee9db423-cc69-4d5f-9b49-a5488772d405\" (UID: \"ee9db423-cc69-4d5f-9b49-a5488772d405\") "
Jul 03 14:42:44.604515 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:44.604379    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee9db423-cc69-4d5f-9b49-a5488772d405-model-cache\") pod \"ee9db423-cc69-4d5f-9b49-a5488772d405\" (UID: \"ee9db423-cc69-4d5f-9b49-a5488772d405\") "
Jul 03 14:42:44.604515 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:44.604406    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee9db423-cc69-4d5f-9b49-a5488772d405-dshm\") pod \"ee9db423-cc69-4d5f-9b49-a5488772d405\" (UID: \"ee9db423-cc69-4d5f-9b49-a5488772d405\") "
Jul 03 14:42:44.604515 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:44.604430    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee9db423-cc69-4d5f-9b49-a5488772d405-tmp-dir\") pod \"ee9db423-cc69-4d5f-9b49-a5488772d405\" (UID: \"ee9db423-cc69-4d5f-9b49-a5488772d405\") "
Jul 03 14:42:44.604515 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:44.604497    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lgtdq\" (UniqueName: \"kubernetes.io/projected/ee9db423-cc69-4d5f-9b49-a5488772d405-kube-api-access-lgtdq\") pod \"ee9db423-cc69-4d5f-9b49-a5488772d405\" (UID: \"ee9db423-cc69-4d5f-9b49-a5488772d405\") "
Jul 03 14:42:44.604777 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:44.604690    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee9db423-cc69-4d5f-9b49-a5488772d405-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ee9db423-cc69-4d5f-9b49-a5488772d405" (UID: "ee9db423-cc69-4d5f-9b49-a5488772d405"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:42:44.604777 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:44.604704    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee9db423-cc69-4d5f-9b49-a5488772d405-model-cache" (OuterVolumeSpecName: "model-cache") pod "ee9db423-cc69-4d5f-9b49-a5488772d405" (UID: "ee9db423-cc69-4d5f-9b49-a5488772d405"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:42:44.604777 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:44.604717    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee9db423-cc69-4d5f-9b49-a5488772d405-home" (OuterVolumeSpecName: "home") pod "ee9db423-cc69-4d5f-9b49-a5488772d405" (UID: "ee9db423-cc69-4d5f-9b49-a5488772d405"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:42:44.604883 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:44.604831    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee9db423-cc69-4d5f-9b49-a5488772d405-home\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:42:44.604883 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:44.604849    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee9db423-cc69-4d5f-9b49-a5488772d405-model-cache\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:42:44.604883 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:44.604863    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee9db423-cc69-4d5f-9b49-a5488772d405-tmp-dir\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:42:44.606772 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:44.606742    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee9db423-cc69-4d5f-9b49-a5488772d405-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ee9db423-cc69-4d5f-9b49-a5488772d405" (UID: "ee9db423-cc69-4d5f-9b49-a5488772d405"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:42:44.606893 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:44.606817    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee9db423-cc69-4d5f-9b49-a5488772d405-dshm" (OuterVolumeSpecName: "dshm") pod "ee9db423-cc69-4d5f-9b49-a5488772d405" (UID: "ee9db423-cc69-4d5f-9b49-a5488772d405"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:42:44.606893 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:44.606861    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee9db423-cc69-4d5f-9b49-a5488772d405-kube-api-access-lgtdq" (OuterVolumeSpecName: "kube-api-access-lgtdq") pod "ee9db423-cc69-4d5f-9b49-a5488772d405" (UID: "ee9db423-cc69-4d5f-9b49-a5488772d405"). InnerVolumeSpecName "kube-api-access-lgtdq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:42:44.705589 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:44.705544    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee9db423-cc69-4d5f-9b49-a5488772d405-tls-certs\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:42:44.705589 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:44.705579    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee9db423-cc69-4d5f-9b49-a5488772d405-dshm\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:42:44.705589 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:44.705592    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgtdq\" (UniqueName: \"kubernetes.io/projected/ee9db423-cc69-4d5f-9b49-a5488772d405-kube-api-access-lgtdq\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:42:44.868902 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:44.868870    2563 generic.go:356] "Generic (PLEG): container finished" podID="ee9db423-cc69-4d5f-9b49-a5488772d405" containerID="ec4cb6cbca8ad80a1f9b041dded5d8abb543a898a6cec96b52b2677fedfa0fa9" exitCode=0
Jul 03 14:42:44.869088 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:44.868915    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"
Jul 03 14:42:44.869088 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:44.868954    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b" event={"ID":"ee9db423-cc69-4d5f-9b49-a5488772d405","Type":"ContainerDied","Data":"ec4cb6cbca8ad80a1f9b041dded5d8abb543a898a6cec96b52b2677fedfa0fa9"}
Jul 03 14:42:44.869088 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:44.869006    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b" event={"ID":"ee9db423-cc69-4d5f-9b49-a5488772d405","Type":"ContainerDied","Data":"3775ecb5edc5a448d17417d8f153acc573c4cfe95ad12fbc7aa6ec5f29929e14"}
Jul 03 14:42:44.869088 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:44.869026    2563 scope.go:117] "RemoveContainer" containerID="ec4cb6cbca8ad80a1f9b041dded5d8abb543a898a6cec96b52b2677fedfa0fa9"
Jul 03 14:42:44.878005 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:44.877963    2563 scope.go:117] "RemoveContainer" containerID="ec4cb6cbca8ad80a1f9b041dded5d8abb543a898a6cec96b52b2677fedfa0fa9"
Jul 03 14:42:44.878260 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:42:44.878240    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec4cb6cbca8ad80a1f9b041dded5d8abb543a898a6cec96b52b2677fedfa0fa9\": container with ID starting with ec4cb6cbca8ad80a1f9b041dded5d8abb543a898a6cec96b52b2677fedfa0fa9 not found: ID does not exist" containerID="ec4cb6cbca8ad80a1f9b041dded5d8abb543a898a6cec96b52b2677fedfa0fa9"
Jul 03 14:42:44.878318 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:44.878266    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec4cb6cbca8ad80a1f9b041dded5d8abb543a898a6cec96b52b2677fedfa0fa9"} err="failed to get container status \"ec4cb6cbca8ad80a1f9b041dded5d8abb543a898a6cec96b52b2677fedfa0fa9\": rpc error: code = NotFound desc = could not find container \"ec4cb6cbca8ad80a1f9b041dded5d8abb543a898a6cec96b52b2677fedfa0fa9\": container with ID starting with ec4cb6cbca8ad80a1f9b041dded5d8abb543a898a6cec96b52b2677fedfa0fa9 not found: ID does not exist"
Jul 03 14:42:44.890036 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:44.890006    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"]
Jul 03 14:42:44.893735 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:44.893713    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-7195a2/test-model-7195a2-kserve-bd8c7ffbc-mv68b"]
Jul 03 14:42:45.058377 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:45.058345    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="446afa9f-d004-4073-96b7-e7ef9412d278" path="/var/lib/kubelet/pods/446afa9f-d004-4073-96b7-e7ef9412d278/volumes"
Jul 03 14:42:45.058765 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:45.058752    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee9db423-cc69-4d5f-9b49-a5488772d405" path="/var/lib/kubelet/pods/ee9db423-cc69-4d5f-9b49-a5488772d405/volumes"
Jul 03 14:42:46.240191 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:42:46.240156    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:43:16.042278 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:43:16.042239    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:43:30.145623 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:43:30.145584    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:43:47.046197 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:43:47.046166    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:44:02.456740 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:02.456662    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:44:17.349069 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:17.349031    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:44:21.158807 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.158775    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"]
Jul 03 14:44:21.159287 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.159178    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="446afa9f-d004-4073-96b7-e7ef9412d278" containerName="main"
Jul 03 14:44:21.159287 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.159192    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="446afa9f-d004-4073-96b7-e7ef9412d278" containerName="main"
Jul 03 14:44:21.159287 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.159218    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee9db423-cc69-4d5f-9b49-a5488772d405" containerName="main"
Jul 03 14:44:21.159287 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.159223    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee9db423-cc69-4d5f-9b49-a5488772d405" containerName="main"
Jul 03 14:44:21.159287 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.159278    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee9db423-cc69-4d5f-9b49-a5488772d405" containerName="main"
Jul 03 14:44:21.159287 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.159287    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="446afa9f-d004-4073-96b7-e7ef9412d278" containerName="main"
Jul 03 14:44:21.162473 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.162453    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"
Jul 03 14:44:21.165772 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.165746    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5f1e04\"/\"auth-test-model-5f1e04-kserve-self-signed-certs\""
Jul 03 14:44:21.165913 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.165810    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5f1e04\"/\"kube-root-ca.crt\""
Jul 03 14:44:21.166018 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.165995    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5f1e04\"/\"default-dockercfg-b7x8t\""
Jul 03 14:44:21.166088 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.166024    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5f1e04\"/\"openshift-service-ca.crt\""
Jul 03 14:44:21.172542 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.172521    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"]
Jul 03 14:44:21.218705 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.218671    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d91e107-bda9-43c6-a631-2a1bf17fc0af-home\") pod \"auth-test-model-5f1e04-kserve-68c475bfd7-p727p\" (UID: \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\") " pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"
Jul 03 14:44:21.218705 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.218710    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d91e107-bda9-43c6-a631-2a1bf17fc0af-model-cache\") pod \"auth-test-model-5f1e04-kserve-68c475bfd7-p727p\" (UID: \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\") " pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"
Jul 03 14:44:21.218914 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.218768    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmmf8\" (UniqueName: \"kubernetes.io/projected/1d91e107-bda9-43c6-a631-2a1bf17fc0af-kube-api-access-fmmf8\") pod \"auth-test-model-5f1e04-kserve-68c475bfd7-p727p\" (UID: \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\") " pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"
Jul 03 14:44:21.218914 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.218806    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d91e107-bda9-43c6-a631-2a1bf17fc0af-tls-certs\") pod \"auth-test-model-5f1e04-kserve-68c475bfd7-p727p\" (UID: \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\") " pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"
Jul 03 14:44:21.218914 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.218853    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d91e107-bda9-43c6-a631-2a1bf17fc0af-dshm\") pod \"auth-test-model-5f1e04-kserve-68c475bfd7-p727p\" (UID: \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\") " pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"
Jul 03 14:44:21.219072 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.218940    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d91e107-bda9-43c6-a631-2a1bf17fc0af-tmp-dir\") pod \"auth-test-model-5f1e04-kserve-68c475bfd7-p727p\" (UID: \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\") " pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"
Jul 03 14:44:21.319605 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.319566    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmmf8\" (UniqueName: \"kubernetes.io/projected/1d91e107-bda9-43c6-a631-2a1bf17fc0af-kube-api-access-fmmf8\") pod \"auth-test-model-5f1e04-kserve-68c475bfd7-p727p\" (UID: \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\") " pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"
Jul 03 14:44:21.319605 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.319606    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d91e107-bda9-43c6-a631-2a1bf17fc0af-tls-certs\") pod \"auth-test-model-5f1e04-kserve-68c475bfd7-p727p\" (UID: \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\") " pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"
Jul 03 14:44:21.319855 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.319631    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d91e107-bda9-43c6-a631-2a1bf17fc0af-dshm\") pod \"auth-test-model-5f1e04-kserve-68c475bfd7-p727p\" (UID: \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\") " pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"
Jul 03 14:44:21.319855 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.319727    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d91e107-bda9-43c6-a631-2a1bf17fc0af-tmp-dir\") pod \"auth-test-model-5f1e04-kserve-68c475bfd7-p727p\" (UID: \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\") " pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"
Jul 03 14:44:21.319855 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.319774    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d91e107-bda9-43c6-a631-2a1bf17fc0af-home\") pod \"auth-test-model-5f1e04-kserve-68c475bfd7-p727p\" (UID: \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\") " pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"
Jul 03 14:44:21.319855 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.319809    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d91e107-bda9-43c6-a631-2a1bf17fc0af-model-cache\") pod \"auth-test-model-5f1e04-kserve-68c475bfd7-p727p\" (UID: \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\") " pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"
Jul 03 14:44:21.320169 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.320138    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d91e107-bda9-43c6-a631-2a1bf17fc0af-home\") pod \"auth-test-model-5f1e04-kserve-68c475bfd7-p727p\" (UID: \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\") " pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"
Jul 03 14:44:21.320292 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.320174    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d91e107-bda9-43c6-a631-2a1bf17fc0af-tmp-dir\") pod \"auth-test-model-5f1e04-kserve-68c475bfd7-p727p\" (UID: \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\") " pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"
Jul 03 14:44:21.320292 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.320206    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d91e107-bda9-43c6-a631-2a1bf17fc0af-model-cache\") pod \"auth-test-model-5f1e04-kserve-68c475bfd7-p727p\" (UID: \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\") " pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"
Jul 03 14:44:21.321881 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.321856    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d91e107-bda9-43c6-a631-2a1bf17fc0af-dshm\") pod \"auth-test-model-5f1e04-kserve-68c475bfd7-p727p\" (UID: \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\") " pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"
Jul 03 14:44:21.322172 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.322157    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d91e107-bda9-43c6-a631-2a1bf17fc0af-tls-certs\") pod \"auth-test-model-5f1e04-kserve-68c475bfd7-p727p\" (UID: \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\") " pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"
Jul 03 14:44:21.326747 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.326728    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmmf8\" (UniqueName: \"kubernetes.io/projected/1d91e107-bda9-43c6-a631-2a1bf17fc0af-kube-api-access-fmmf8\") pod \"auth-test-model-5f1e04-kserve-68c475bfd7-p727p\" (UID: \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\") " pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"
Jul 03 14:44:21.473469 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.473386    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"
Jul 03 14:44:21.606474 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:21.606451    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"]
Jul 03 14:44:21.608324 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:44:21.608294    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d91e107_bda9_43c6_a631_2a1bf17fc0af.slice/crio-082cb4e2992a6fcb27523b962c56ce07d3799d663272357b433701f17ffa6c3e WatchSource:0}: Error finding container 082cb4e2992a6fcb27523b962c56ce07d3799d663272357b433701f17ffa6c3e: Status 404 returned error can't find the container with id 082cb4e2992a6fcb27523b962c56ce07d3799d663272357b433701f17ffa6c3e
Jul 03 14:44:22.336680 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:22.336648    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p" event={"ID":"1d91e107-bda9-43c6-a631-2a1bf17fc0af","Type":"ContainerStarted","Data":"f455a97504b42a37c7b74d8eca722c2b9978c8961325322b6715d2a19f70fd77"}
Jul 03 14:44:22.336680 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:22.336690    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p" event={"ID":"1d91e107-bda9-43c6-a631-2a1bf17fc0af","Type":"ContainerStarted","Data":"082cb4e2992a6fcb27523b962c56ce07d3799d663272357b433701f17ffa6c3e"}
Jul 03 14:44:22.354857 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:22.354809    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p" podStartSLOduration=1.354794686 podStartE2EDuration="1.354794686s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:44:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:44:22.353098016 +0000 UTC m=+2175.803231471" watchObservedRunningTime="2026-07-03 14:44:22.354794686 +0000 UTC m=+2175.804928126"
Jul 03 14:44:31.474398 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:31.474347    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"
Jul 03 14:44:31.474857 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:31.474412    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"
Jul 03 14:44:31.487132 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:31.487109    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"
Jul 03 14:44:32.394372 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:32.394341    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"
Jul 03 14:44:34.045840 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:44:34.045813    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:45:00.152638 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:00.152598    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-5f1e04-29718165-7cm2n"]
Jul 03 14:45:00.155964 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:00.155948    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-5f1e04-29718165-7cm2n"
Jul 03 14:45:00.159366 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:00.159340    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-f66fv\""
Jul 03 14:45:00.167730 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:00.167701    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-5f1e04-29718165-7cm2n"]
Jul 03 14:45:00.194322 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:00.194291    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9vh5\" (UniqueName: \"kubernetes.io/projected/cbc78d57-b903-480e-8f86-801352bff8fa-kube-api-access-m9vh5\") pod \"maas-api-key-cleanup-e2e-shared-a-5f1e04-29718165-7cm2n\" (UID: \"cbc78d57-b903-480e-8f86-801352bff8fa\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-5f1e04-29718165-7cm2n"
Jul 03 14:45:00.295647 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:00.295606    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9vh5\" (UniqueName: \"kubernetes.io/projected/cbc78d57-b903-480e-8f86-801352bff8fa-kube-api-access-m9vh5\") pod \"maas-api-key-cleanup-e2e-shared-a-5f1e04-29718165-7cm2n\" (UID: \"cbc78d57-b903-480e-8f86-801352bff8fa\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-5f1e04-29718165-7cm2n"
Jul 03 14:45:00.304993 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:00.304935    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9vh5\" (UniqueName: \"kubernetes.io/projected/cbc78d57-b903-480e-8f86-801352bff8fa-kube-api-access-m9vh5\") pod \"maas-api-key-cleanup-e2e-shared-a-5f1e04-29718165-7cm2n\" (UID: \"cbc78d57-b903-480e-8f86-801352bff8fa\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-5f1e04-29718165-7cm2n"
Jul 03 14:45:00.469286 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:00.469198    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-5f1e04-29718165-7cm2n"
Jul 03 14:45:00.604989 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:00.604944    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-5f1e04-29718165-7cm2n"]
Jul 03 14:45:00.607265 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:45:00.607237    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbc78d57_b903_480e_8f86_801352bff8fa.slice/crio-27a45770df4772eecfd179ce5b3798f829e8d3940711c84cb55059835692a4f4 WatchSource:0}: Error finding container 27a45770df4772eecfd179ce5b3798f829e8d3940711c84cb55059835692a4f4: Status 404 returned error can't find the container with id 27a45770df4772eecfd179ce5b3798f829e8d3940711c84cb55059835692a4f4
Jul 03 14:45:00.609286 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:00.609265    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 03 14:45:01.529040 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:01.528994    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-5f1e04-29718165-7cm2n" event={"ID":"cbc78d57-b903-480e-8f86-801352bff8fa","Type":"ContainerStarted","Data":"27a45770df4772eecfd179ce5b3798f829e8d3940711c84cb55059835692a4f4"}
Jul 03 14:45:03.541909 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:03.541875    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-5f1e04-29718165-7cm2n" event={"ID":"cbc78d57-b903-480e-8f86-801352bff8fa","Type":"ContainerStarted","Data":"0edfbc7566626f3cf2c0153c83d5be4f61523af1d385285e930b6cfdc05a8205"}
Jul 03 14:45:03.558304 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:03.558252    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-5f1e04-29718165-7cm2n" podStartSLOduration=1.439215614 podStartE2EDuration="3.558234049s" totalImagesPullingTime="2.119018435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:45:03.555437498 +0000 UTC m=+2217.005570940" watchObservedRunningTime="2026-07-03 14:45:03.558234049 +0000 UTC m=+2217.008367489"
Jul 03 14:45:04.548432 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:04.548398    2563 generic.go:356] "Generic (PLEG): container finished" podID="cbc78d57-b903-480e-8f86-801352bff8fa" containerID="0edfbc7566626f3cf2c0153c83d5be4f61523af1d385285e930b6cfdc05a8205" exitCode=0
Jul 03 14:45:04.548842 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:04.548437    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-5f1e04-29718165-7cm2n" event={"ID":"cbc78d57-b903-480e-8f86-801352bff8fa","Type":"ContainerDied","Data":"0edfbc7566626f3cf2c0153c83d5be4f61523af1d385285e930b6cfdc05a8205"}
Jul 03 14:45:05.696852 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:05.696824    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-5f1e04-29718165-7cm2n"
Jul 03 14:45:05.747374 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:05.747338    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9vh5\" (UniqueName: \"kubernetes.io/projected/cbc78d57-b903-480e-8f86-801352bff8fa-kube-api-access-m9vh5\") pod \"cbc78d57-b903-480e-8f86-801352bff8fa\" (UID: \"cbc78d57-b903-480e-8f86-801352bff8fa\") "
Jul 03 14:45:05.749528 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:05.749500    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbc78d57-b903-480e-8f86-801352bff8fa-kube-api-access-m9vh5" (OuterVolumeSpecName: "kube-api-access-m9vh5") pod "cbc78d57-b903-480e-8f86-801352bff8fa" (UID: "cbc78d57-b903-480e-8f86-801352bff8fa"). InnerVolumeSpecName "kube-api-access-m9vh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:45:05.848615 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:05.848526    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9vh5\" (UniqueName: \"kubernetes.io/projected/cbc78d57-b903-480e-8f86-801352bff8fa-kube-api-access-m9vh5\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:45:06.559616 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:06.559583    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-5f1e04-29718165-7cm2n"
Jul 03 14:45:06.559767 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:06.559617    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-5f1e04-29718165-7cm2n" event={"ID":"cbc78d57-b903-480e-8f86-801352bff8fa","Type":"ContainerDied","Data":"27a45770df4772eecfd179ce5b3798f829e8d3940711c84cb55059835692a4f4"}
Jul 03 14:45:06.559767 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:06.559665    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="27a45770df4772eecfd179ce5b3798f829e8d3940711c84cb55059835692a4f4"
Jul 03 14:45:07.137992 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.137931    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"]
Jul 03 14:45:07.138526 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.138506    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbc78d57-b903-480e-8f86-801352bff8fa" containerName="cleanup"
Jul 03 14:45:07.138602 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.138530    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbc78d57-b903-480e-8f86-801352bff8fa" containerName="cleanup"
Jul 03 14:45:07.138653 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.138630    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbc78d57-b903-480e-8f86-801352bff8fa" containerName="cleanup"
Jul 03 14:45:07.141718 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.141699    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"
Jul 03 14:45:07.144120 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.144101    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-28686a\"/\"openshift-service-ca.crt\""
Jul 03 14:45:07.144219 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.144144    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-28686a\"/\"default-dockercfg-84tzh\""
Jul 03 14:45:07.144875 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.144857    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-28686a\"/\"kube-root-ca.crt\""
Jul 03 14:45:07.144930 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.144884    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-28686a\"/\"auth-test-model-28686a-kserve-self-signed-certs\""
Jul 03 14:45:07.150904 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.150882    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"]
Jul 03 14:45:07.162499 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.162465    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/afbf7939-571b-4dd0-b64e-b904fa963402-tmp-dir\") pod \"auth-test-model-28686a-kserve-7d8bcc5479-q7pjq\" (UID: \"afbf7939-571b-4dd0-b64e-b904fa963402\") " pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"
Jul 03 14:45:07.162668 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.162647    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/afbf7939-571b-4dd0-b64e-b904fa963402-tls-certs\") pod \"auth-test-model-28686a-kserve-7d8bcc5479-q7pjq\" (UID: \"afbf7939-571b-4dd0-b64e-b904fa963402\") " pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"
Jul 03 14:45:07.162815 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.162793    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/afbf7939-571b-4dd0-b64e-b904fa963402-home\") pod \"auth-test-model-28686a-kserve-7d8bcc5479-q7pjq\" (UID: \"afbf7939-571b-4dd0-b64e-b904fa963402\") " pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"
Jul 03 14:45:07.162908 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.162844    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/afbf7939-571b-4dd0-b64e-b904fa963402-model-cache\") pod \"auth-test-model-28686a-kserve-7d8bcc5479-q7pjq\" (UID: \"afbf7939-571b-4dd0-b64e-b904fa963402\") " pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"
Jul 03 14:45:07.162908 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.162870    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxnnr\" (UniqueName: \"kubernetes.io/projected/afbf7939-571b-4dd0-b64e-b904fa963402-kube-api-access-zxnnr\") pod \"auth-test-model-28686a-kserve-7d8bcc5479-q7pjq\" (UID: \"afbf7939-571b-4dd0-b64e-b904fa963402\") " pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"
Jul 03 14:45:07.163066 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.162910    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/afbf7939-571b-4dd0-b64e-b904fa963402-dshm\") pod \"auth-test-model-28686a-kserve-7d8bcc5479-q7pjq\" (UID: \"afbf7939-571b-4dd0-b64e-b904fa963402\") " pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"
Jul 03 14:45:07.264271 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.264241    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/afbf7939-571b-4dd0-b64e-b904fa963402-home\") pod \"auth-test-model-28686a-kserve-7d8bcc5479-q7pjq\" (UID: \"afbf7939-571b-4dd0-b64e-b904fa963402\") " pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"
Jul 03 14:45:07.264452 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.264279    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/afbf7939-571b-4dd0-b64e-b904fa963402-model-cache\") pod \"auth-test-model-28686a-kserve-7d8bcc5479-q7pjq\" (UID: \"afbf7939-571b-4dd0-b64e-b904fa963402\") " pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"
Jul 03 14:45:07.264452 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.264297    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxnnr\" (UniqueName: \"kubernetes.io/projected/afbf7939-571b-4dd0-b64e-b904fa963402-kube-api-access-zxnnr\") pod \"auth-test-model-28686a-kserve-7d8bcc5479-q7pjq\" (UID: \"afbf7939-571b-4dd0-b64e-b904fa963402\") " pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"
Jul 03 14:45:07.264452 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.264430    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/afbf7939-571b-4dd0-b64e-b904fa963402-dshm\") pod \"auth-test-model-28686a-kserve-7d8bcc5479-q7pjq\" (UID: \"afbf7939-571b-4dd0-b64e-b904fa963402\") " pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"
Jul 03 14:45:07.264625 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.264579    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/afbf7939-571b-4dd0-b64e-b904fa963402-tmp-dir\") pod \"auth-test-model-28686a-kserve-7d8bcc5479-q7pjq\" (UID: \"afbf7939-571b-4dd0-b64e-b904fa963402\") " pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"
Jul 03 14:45:07.264625 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.264618    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/afbf7939-571b-4dd0-b64e-b904fa963402-tls-certs\") pod \"auth-test-model-28686a-kserve-7d8bcc5479-q7pjq\" (UID: \"afbf7939-571b-4dd0-b64e-b904fa963402\") " pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"
Jul 03 14:45:07.264732 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.264649    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/afbf7939-571b-4dd0-b64e-b904fa963402-home\") pod \"auth-test-model-28686a-kserve-7d8bcc5479-q7pjq\" (UID: \"afbf7939-571b-4dd0-b64e-b904fa963402\") " pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"
Jul 03 14:45:07.264732 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.264696    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/afbf7939-571b-4dd0-b64e-b904fa963402-model-cache\") pod \"auth-test-model-28686a-kserve-7d8bcc5479-q7pjq\" (UID: \"afbf7939-571b-4dd0-b64e-b904fa963402\") " pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"
Jul 03 14:45:07.264871 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.264854    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/afbf7939-571b-4dd0-b64e-b904fa963402-tmp-dir\") pod \"auth-test-model-28686a-kserve-7d8bcc5479-q7pjq\" (UID: \"afbf7939-571b-4dd0-b64e-b904fa963402\") " pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"
Jul 03 14:45:07.266636 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.266619    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/afbf7939-571b-4dd0-b64e-b904fa963402-dshm\") pod \"auth-test-model-28686a-kserve-7d8bcc5479-q7pjq\" (UID: \"afbf7939-571b-4dd0-b64e-b904fa963402\") " pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"
Jul 03 14:45:07.266886 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.266869    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/afbf7939-571b-4dd0-b64e-b904fa963402-tls-certs\") pod \"auth-test-model-28686a-kserve-7d8bcc5479-q7pjq\" (UID: \"afbf7939-571b-4dd0-b64e-b904fa963402\") " pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"
Jul 03 14:45:07.272401 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.272373    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxnnr\" (UniqueName: \"kubernetes.io/projected/afbf7939-571b-4dd0-b64e-b904fa963402-kube-api-access-zxnnr\") pod \"auth-test-model-28686a-kserve-7d8bcc5479-q7pjq\" (UID: \"afbf7939-571b-4dd0-b64e-b904fa963402\") " pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"
Jul 03 14:45:07.451243 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.451142    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"
Jul 03 14:45:07.588660 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:07.588630    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"]
Jul 03 14:45:07.590832 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:45:07.590788    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podafbf7939_571b_4dd0_b64e_b904fa963402.slice/crio-458b63d85f957ceff5496f0cb22d510be5bc261d864a16fcf8458b6d14a5ac53 WatchSource:0}: Error finding container 458b63d85f957ceff5496f0cb22d510be5bc261d864a16fcf8458b6d14a5ac53: Status 404 returned error can't find the container with id 458b63d85f957ceff5496f0cb22d510be5bc261d864a16fcf8458b6d14a5ac53
Jul 03 14:45:08.572329 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:08.572290    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq" event={"ID":"afbf7939-571b-4dd0-b64e-b904fa963402","Type":"ContainerStarted","Data":"4a23b75b82723a6855d2006f98a441fffd6d1c22fb4c138cf552a45ee18c9fc9"}
Jul 03 14:45:08.572329 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:08.572334    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq" event={"ID":"afbf7939-571b-4dd0-b64e-b904fa963402","Type":"ContainerStarted","Data":"458b63d85f957ceff5496f0cb22d510be5bc261d864a16fcf8458b6d14a5ac53"}
Jul 03 14:45:08.591712 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:08.591654    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq" podStartSLOduration=1.59163751 podStartE2EDuration="1.59163751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:45:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:45:08.59020427 +0000 UTC m=+2222.040337725" watchObservedRunningTime="2026-07-03 14:45:08.59163751 +0000 UTC m=+2222.041770951"
Jul 03 14:45:10.747257 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:10.747218    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:45:17.451677 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:17.451634    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"
Jul 03 14:45:17.451677 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:17.451685    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"
Jul 03 14:45:17.464540 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:17.464513    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"
Jul 03 14:45:17.627733 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:17.627703    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"
Jul 03 14:45:26.041856 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:26.041820    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:45:44.943486 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:44.943453    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:45:59.734521 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:45:59.734478    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:46:13.936633 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:46:13.936600    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:46:31.189713 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:46:31.189677    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:46:48.639491 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:46:48.639450    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:47:01.744730 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:01.744641    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"]
Jul 03 14:47:02.635329 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:02.635291    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"]
Jul 03 14:47:04.935810 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:04.935776    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:47:16.764794 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:16.764750    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p" podUID="1d91e107-bda9-43c6-a631-2a1bf17fc0af" containerName="main" containerID="cri-o://f455a97504b42a37c7b74d8eca722c2b9978c8961325322b6715d2a19f70fd77" gracePeriod=45
Jul 03 14:47:17.011651 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.011623    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"
Jul 03 14:47:17.121452 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.121423    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d91e107-bda9-43c6-a631-2a1bf17fc0af-tmp-dir\") pod \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\" (UID: \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\") "
Jul 03 14:47:17.121625 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.121460    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d91e107-bda9-43c6-a631-2a1bf17fc0af-dshm\") pod \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\" (UID: \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\") "
Jul 03 14:47:17.121625 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.121510    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmmf8\" (UniqueName: \"kubernetes.io/projected/1d91e107-bda9-43c6-a631-2a1bf17fc0af-kube-api-access-fmmf8\") pod \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\" (UID: \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\") "
Jul 03 14:47:17.121625 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.121555    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d91e107-bda9-43c6-a631-2a1bf17fc0af-model-cache\") pod \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\" (UID: \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\") "
Jul 03 14:47:17.121625 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.121572    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d91e107-bda9-43c6-a631-2a1bf17fc0af-tls-certs\") pod \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\" (UID: \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\") "
Jul 03 14:47:17.121625 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.121621    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d91e107-bda9-43c6-a631-2a1bf17fc0af-home\") pod \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\" (UID: \"1d91e107-bda9-43c6-a631-2a1bf17fc0af\") "
Jul 03 14:47:17.122343 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.121741    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d91e107-bda9-43c6-a631-2a1bf17fc0af-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1d91e107-bda9-43c6-a631-2a1bf17fc0af" (UID: "1d91e107-bda9-43c6-a631-2a1bf17fc0af"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:47:17.122343 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.121859    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d91e107-bda9-43c6-a631-2a1bf17fc0af-model-cache" (OuterVolumeSpecName: "model-cache") pod "1d91e107-bda9-43c6-a631-2a1bf17fc0af" (UID: "1d91e107-bda9-43c6-a631-2a1bf17fc0af"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:47:17.122343 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.122038    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d91e107-bda9-43c6-a631-2a1bf17fc0af-model-cache\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:47:17.122343 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.122060    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d91e107-bda9-43c6-a631-2a1bf17fc0af-tmp-dir\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:47:17.122343 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.122072    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d91e107-bda9-43c6-a631-2a1bf17fc0af-home" (OuterVolumeSpecName: "home") pod "1d91e107-bda9-43c6-a631-2a1bf17fc0af" (UID: "1d91e107-bda9-43c6-a631-2a1bf17fc0af"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:47:17.123936 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.123901    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d91e107-bda9-43c6-a631-2a1bf17fc0af-dshm" (OuterVolumeSpecName: "dshm") pod "1d91e107-bda9-43c6-a631-2a1bf17fc0af" (UID: "1d91e107-bda9-43c6-a631-2a1bf17fc0af"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:47:17.124106 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.123992    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d91e107-bda9-43c6-a631-2a1bf17fc0af-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1d91e107-bda9-43c6-a631-2a1bf17fc0af" (UID: "1d91e107-bda9-43c6-a631-2a1bf17fc0af"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:47:17.124178 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.124111    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d91e107-bda9-43c6-a631-2a1bf17fc0af-kube-api-access-fmmf8" (OuterVolumeSpecName: "kube-api-access-fmmf8") pod "1d91e107-bda9-43c6-a631-2a1bf17fc0af" (UID: "1d91e107-bda9-43c6-a631-2a1bf17fc0af"). InnerVolumeSpecName "kube-api-access-fmmf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:47:17.211636 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.211597    2563 generic.go:356] "Generic (PLEG): container finished" podID="1d91e107-bda9-43c6-a631-2a1bf17fc0af" containerID="f455a97504b42a37c7b74d8eca722c2b9978c8961325322b6715d2a19f70fd77" exitCode=0
Jul 03 14:47:17.211778 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.211642    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"
Jul 03 14:47:17.211778 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.211685    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p" event={"ID":"1d91e107-bda9-43c6-a631-2a1bf17fc0af","Type":"ContainerDied","Data":"f455a97504b42a37c7b74d8eca722c2b9978c8961325322b6715d2a19f70fd77"}
Jul 03 14:47:17.211778 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.211730    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p" event={"ID":"1d91e107-bda9-43c6-a631-2a1bf17fc0af","Type":"ContainerDied","Data":"082cb4e2992a6fcb27523b962c56ce07d3799d663272357b433701f17ffa6c3e"}
Jul 03 14:47:17.211778 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.211747    2563 scope.go:117] "RemoveContainer" containerID="f455a97504b42a37c7b74d8eca722c2b9978c8961325322b6715d2a19f70fd77"
Jul 03 14:47:17.221425 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.221405    2563 scope.go:117] "RemoveContainer" containerID="f455a97504b42a37c7b74d8eca722c2b9978c8961325322b6715d2a19f70fd77"
Jul 03 14:47:17.221697 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:47:17.221674    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f455a97504b42a37c7b74d8eca722c2b9978c8961325322b6715d2a19f70fd77\": container with ID starting with f455a97504b42a37c7b74d8eca722c2b9978c8961325322b6715d2a19f70fd77 not found: ID does not exist" containerID="f455a97504b42a37c7b74d8eca722c2b9978c8961325322b6715d2a19f70fd77"
Jul 03 14:47:17.221776 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.221707    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f455a97504b42a37c7b74d8eca722c2b9978c8961325322b6715d2a19f70fd77"} err="failed to get container status \"f455a97504b42a37c7b74d8eca722c2b9978c8961325322b6715d2a19f70fd77\": rpc error: code = NotFound desc = could not find container \"f455a97504b42a37c7b74d8eca722c2b9978c8961325322b6715d2a19f70fd77\": container with ID starting with f455a97504b42a37c7b74d8eca722c2b9978c8961325322b6715d2a19f70fd77 not found: ID does not exist"
Jul 03 14:47:17.223022 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.222981    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d91e107-bda9-43c6-a631-2a1bf17fc0af-dshm\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:47:17.223117 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.223025    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmmf8\" (UniqueName: \"kubernetes.io/projected/1d91e107-bda9-43c6-a631-2a1bf17fc0af-kube-api-access-fmmf8\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:47:17.223117 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.223040    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d91e107-bda9-43c6-a631-2a1bf17fc0af-tls-certs\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:47:17.223117 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.223051    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d91e107-bda9-43c6-a631-2a1bf17fc0af-home\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:47:17.232730 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.232708    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"]
Jul 03 14:47:17.240963 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.240942    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5f1e04/auth-test-model-5f1e04-kserve-68c475bfd7-p727p"]
Jul 03 14:47:17.656060 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.656019    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq" podUID="afbf7939-571b-4dd0-b64e-b904fa963402" containerName="main" containerID="cri-o://4a23b75b82723a6855d2006f98a441fffd6d1c22fb4c138cf552a45ee18c9fc9" gracePeriod=45
Jul 03 14:47:17.910295 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:17.910233    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"
Jul 03 14:47:18.030723 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:18.030691    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/afbf7939-571b-4dd0-b64e-b904fa963402-home\") pod \"afbf7939-571b-4dd0-b64e-b904fa963402\" (UID: \"afbf7939-571b-4dd0-b64e-b904fa963402\") "
Jul 03 14:47:18.030917 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:18.030738    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/afbf7939-571b-4dd0-b64e-b904fa963402-tls-certs\") pod \"afbf7939-571b-4dd0-b64e-b904fa963402\" (UID: \"afbf7939-571b-4dd0-b64e-b904fa963402\") "
Jul 03 14:47:18.030917 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:18.030813    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/afbf7939-571b-4dd0-b64e-b904fa963402-dshm\") pod \"afbf7939-571b-4dd0-b64e-b904fa963402\" (UID: \"afbf7939-571b-4dd0-b64e-b904fa963402\") "
Jul 03 14:47:18.030917 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:18.030860    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxnnr\" (UniqueName: \"kubernetes.io/projected/afbf7939-571b-4dd0-b64e-b904fa963402-kube-api-access-zxnnr\") pod \"afbf7939-571b-4dd0-b64e-b904fa963402\" (UID: \"afbf7939-571b-4dd0-b64e-b904fa963402\") "
Jul 03 14:47:18.030917 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:18.030907    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/afbf7939-571b-4dd0-b64e-b904fa963402-tmp-dir\") pod \"afbf7939-571b-4dd0-b64e-b904fa963402\" (UID: \"afbf7939-571b-4dd0-b64e-b904fa963402\") "
Jul 03 14:47:18.031184 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:18.031019    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/afbf7939-571b-4dd0-b64e-b904fa963402-model-cache\") pod \"afbf7939-571b-4dd0-b64e-b904fa963402\" (UID: \"afbf7939-571b-4dd0-b64e-b904fa963402\") "
Jul 03 14:47:18.031184 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:18.031031    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afbf7939-571b-4dd0-b64e-b904fa963402-home" (OuterVolumeSpecName: "home") pod "afbf7939-571b-4dd0-b64e-b904fa963402" (UID: "afbf7939-571b-4dd0-b64e-b904fa963402"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:47:18.031297 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:18.031233    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afbf7939-571b-4dd0-b64e-b904fa963402-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "afbf7939-571b-4dd0-b64e-b904fa963402" (UID: "afbf7939-571b-4dd0-b64e-b904fa963402"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:47:18.031351 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:18.031322    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afbf7939-571b-4dd0-b64e-b904fa963402-model-cache" (OuterVolumeSpecName: "model-cache") pod "afbf7939-571b-4dd0-b64e-b904fa963402" (UID: "afbf7939-571b-4dd0-b64e-b904fa963402"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:47:18.031408 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:18.031385    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/afbf7939-571b-4dd0-b64e-b904fa963402-tmp-dir\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:47:18.031408 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:18.031400    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/afbf7939-571b-4dd0-b64e-b904fa963402-home\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:47:18.033005 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:18.032959    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afbf7939-571b-4dd0-b64e-b904fa963402-dshm" (OuterVolumeSpecName: "dshm") pod "afbf7939-571b-4dd0-b64e-b904fa963402" (UID: "afbf7939-571b-4dd0-b64e-b904fa963402"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:47:18.033111 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:18.033055    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afbf7939-571b-4dd0-b64e-b904fa963402-kube-api-access-zxnnr" (OuterVolumeSpecName: "kube-api-access-zxnnr") pod "afbf7939-571b-4dd0-b64e-b904fa963402" (UID: "afbf7939-571b-4dd0-b64e-b904fa963402"). InnerVolumeSpecName "kube-api-access-zxnnr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:47:18.033111 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:18.033062    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afbf7939-571b-4dd0-b64e-b904fa963402-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "afbf7939-571b-4dd0-b64e-b904fa963402" (UID: "afbf7939-571b-4dd0-b64e-b904fa963402"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:47:18.132285 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:18.132248    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/afbf7939-571b-4dd0-b64e-b904fa963402-model-cache\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:47:18.132285 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:18.132277    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/afbf7939-571b-4dd0-b64e-b904fa963402-tls-certs\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:47:18.132285 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:18.132285    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/afbf7939-571b-4dd0-b64e-b904fa963402-dshm\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:47:18.132285 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:18.132295    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxnnr\" (UniqueName: \"kubernetes.io/projected/afbf7939-571b-4dd0-b64e-b904fa963402-kube-api-access-zxnnr\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:47:18.218843 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:18.218759    2563 generic.go:356] "Generic (PLEG): container finished" podID="afbf7939-571b-4dd0-b64e-b904fa963402" containerID="4a23b75b82723a6855d2006f98a441fffd6d1c22fb4c138cf552a45ee18c9fc9" exitCode=0
Jul 03 14:47:18.218843 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:18.218809    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"
Jul 03 14:47:18.219076 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:18.218842    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq" event={"ID":"afbf7939-571b-4dd0-b64e-b904fa963402","Type":"ContainerDied","Data":"4a23b75b82723a6855d2006f98a441fffd6d1c22fb4c138cf552a45ee18c9fc9"}
Jul 03 14:47:18.219076 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:18.218890    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq" event={"ID":"afbf7939-571b-4dd0-b64e-b904fa963402","Type":"ContainerDied","Data":"458b63d85f957ceff5496f0cb22d510be5bc261d864a16fcf8458b6d14a5ac53"}
Jul 03 14:47:18.219076 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:18.218909    2563 scope.go:117] "RemoveContainer" containerID="4a23b75b82723a6855d2006f98a441fffd6d1c22fb4c138cf552a45ee18c9fc9"
Jul 03 14:47:18.228094 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:18.228073    2563 scope.go:117] "RemoveContainer" containerID="4a23b75b82723a6855d2006f98a441fffd6d1c22fb4c138cf552a45ee18c9fc9"
Jul 03 14:47:18.228333 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:47:18.228315    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a23b75b82723a6855d2006f98a441fffd6d1c22fb4c138cf552a45ee18c9fc9\": container with ID starting with 4a23b75b82723a6855d2006f98a441fffd6d1c22fb4c138cf552a45ee18c9fc9 not found: ID does not exist" containerID="4a23b75b82723a6855d2006f98a441fffd6d1c22fb4c138cf552a45ee18c9fc9"
Jul 03 14:47:18.228382 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:18.228340    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a23b75b82723a6855d2006f98a441fffd6d1c22fb4c138cf552a45ee18c9fc9"} err="failed to get container status \"4a23b75b82723a6855d2006f98a441fffd6d1c22fb4c138cf552a45ee18c9fc9\": rpc error: code = NotFound desc = could not find container \"4a23b75b82723a6855d2006f98a441fffd6d1c22fb4c138cf552a45ee18c9fc9\": container with ID starting with 4a23b75b82723a6855d2006f98a441fffd6d1c22fb4c138cf552a45ee18c9fc9 not found: ID does not exist"
Jul 03 14:47:18.239638 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:18.239615    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"]
Jul 03 14:47:18.243153 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:18.243132    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-28686a/auth-test-model-28686a-kserve-7d8bcc5479-q7pjq"]
Jul 03 14:47:19.059296 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:19.059262    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d91e107-bda9-43c6-a631-2a1bf17fc0af" path="/var/lib/kubelet/pods/1d91e107-bda9-43c6-a631-2a1bf17fc0af/volumes"
Jul 03 14:47:19.059657 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:19.059643    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afbf7939-571b-4dd0-b64e-b904fa963402" path="/var/lib/kubelet/pods/afbf7939-571b-4dd0-b64e-b904fa963402/volumes"
Jul 03 14:47:30.449119 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.449088    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"]
Jul 03 14:47:30.449477 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.449467    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afbf7939-571b-4dd0-b64e-b904fa963402" containerName="main"
Jul 03 14:47:30.449531 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.449478    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="afbf7939-571b-4dd0-b64e-b904fa963402" containerName="main"
Jul 03 14:47:30.449531 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.449497    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d91e107-bda9-43c6-a631-2a1bf17fc0af" containerName="main"
Jul 03 14:47:30.449531 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.449503    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d91e107-bda9-43c6-a631-2a1bf17fc0af" containerName="main"
Jul 03 14:47:30.449629 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.449573    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="afbf7939-571b-4dd0-b64e-b904fa963402" containerName="main"
Jul 03 14:47:30.449629 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.449583    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d91e107-bda9-43c6-a631-2a1bf17fc0af" containerName="main"
Jul 03 14:47:30.454053 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.454034    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"
Jul 03 14:47:30.456463 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.456441    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5f1e04\"/\"openshift-service-ca.crt\""
Jul 03 14:47:30.457440 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.457423    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5f1e04\"/\"kube-root-ca.crt\""
Jul 03 14:47:30.457535 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.457456    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5f1e04\"/\"sub-test-model-5f1e04-kserve-self-signed-certs\""
Jul 03 14:47:30.457605 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.457544    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5f1e04\"/\"default-dockercfg-b7x8t\""
Jul 03 14:47:30.462738 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.462710    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"]
Jul 03 14:47:30.556781 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.556744    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-tls-certs\") pod \"sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r\" (UID: \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\") " pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"
Jul 03 14:47:30.556948 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.556860    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-home\") pod \"sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r\" (UID: \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\") " pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"
Jul 03 14:47:30.557083 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.556962    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-tmp-dir\") pod \"sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r\" (UID: \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\") " pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"
Jul 03 14:47:30.557083 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.557046    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-dshm\") pod \"sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r\" (UID: \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\") " pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"
Jul 03 14:47:30.557083 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.557072    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-model-cache\") pod \"sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r\" (UID: \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\") " pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"
Jul 03 14:47:30.557197 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.557129    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxdbq\" (UniqueName: \"kubernetes.io/projected/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-kube-api-access-mxdbq\") pod \"sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r\" (UID: \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\") " pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"
Jul 03 14:47:30.658448 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.658409    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-home\") pod \"sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r\" (UID: \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\") " pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"
Jul 03 14:47:30.658643 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.658466    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-tmp-dir\") pod \"sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r\" (UID: \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\") " pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"
Jul 03 14:47:30.658643 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.658494    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-dshm\") pod \"sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r\" (UID: \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\") " pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"
Jul 03 14:47:30.658643 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.658509    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-model-cache\") pod \"sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r\" (UID: \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\") " pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"
Jul 03 14:47:30.658643 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.658538    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxdbq\" (UniqueName: \"kubernetes.io/projected/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-kube-api-access-mxdbq\") pod \"sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r\" (UID: \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\") " pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"
Jul 03 14:47:30.658643 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.658562    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-tls-certs\") pod \"sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r\" (UID: \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\") " pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"
Jul 03 14:47:30.658904 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.658874    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-home\") pod \"sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r\" (UID: \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\") " pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"
Jul 03 14:47:30.658963 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.658886    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-tmp-dir\") pod \"sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r\" (UID: \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\") " pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"
Jul 03 14:47:30.658963 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.658954    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-model-cache\") pod \"sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r\" (UID: \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\") " pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"
Jul 03 14:47:30.660828 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.660809    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-dshm\") pod \"sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r\" (UID: \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\") " pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"
Jul 03 14:47:30.661020 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.661003    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-tls-certs\") pod \"sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r\" (UID: \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\") " pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"
Jul 03 14:47:30.666461 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.666436    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxdbq\" (UniqueName: \"kubernetes.io/projected/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-kube-api-access-mxdbq\") pod \"sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r\" (UID: \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\") " pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"
Jul 03 14:47:30.764537 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.764436    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"
Jul 03 14:47:30.899324 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:30.899298    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"]
Jul 03 14:47:30.901880 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:47:30.901852    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf66eb3d_6a80_4b99_81e9_28cc9c2ed6db.slice/crio-379e0ec89d9235d20c5603232b6ca90d136b2817ac3506026d2394426896e725 WatchSource:0}: Error finding container 379e0ec89d9235d20c5603232b6ca90d136b2817ac3506026d2394426896e725: Status 404 returned error can't find the container with id 379e0ec89d9235d20c5603232b6ca90d136b2817ac3506026d2394426896e725
Jul 03 14:47:31.283436 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:31.283398    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r" event={"ID":"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db","Type":"ContainerStarted","Data":"a76047493f652fd3b7bb0c2f1425e91ff4c6449c6075acc7b9504c9264b0feeb"}
Jul 03 14:47:31.283436 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:31.283440    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r" event={"ID":"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db","Type":"ContainerStarted","Data":"379e0ec89d9235d20c5603232b6ca90d136b2817ac3506026d2394426896e725"}
Jul 03 14:47:31.301000 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:31.300937    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r" podStartSLOduration=1.300905511 podStartE2EDuration="1.300905511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:47:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:47:31.298963265 +0000 UTC m=+2364.749096706" watchObservedRunningTime="2026-07-03 14:47:31.300905511 +0000 UTC m=+2364.751038950"
Jul 03 14:47:34.248232 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:34.248197    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:47:40.765395 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:40.765355    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"
Jul 03 14:47:40.765894 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:40.765417    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"
Jul 03 14:47:40.777817 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:40.777785    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"
Jul 03 14:47:41.345311 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:41.345276    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"
Jul 03 14:47:46.739404 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:46.739374    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"]
Jul 03 14:47:46.743100 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:46.743080    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"
Jul 03 14:47:46.745407 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:46.745384    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-28686a\"/\"kube-root-ca.crt\""
Jul 03 14:47:46.746430 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:46.746404    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-28686a\"/\"default-dockercfg-84tzh\""
Jul 03 14:47:46.746528 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:46.746428    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-28686a\"/\"openshift-service-ca.crt\""
Jul 03 14:47:46.746572 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:46.746527    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-28686a\"/\"sub-test-model-28686a-kserve-self-signed-certs\""
Jul 03 14:47:46.751459 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:46.751436    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"]
Jul 03 14:47:46.808016 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:46.807959    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-home\") pod \"sub-test-model-28686a-kserve-76b7757dbf-m28nm\" (UID: \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\") " pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"
Jul 03 14:47:46.808200 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:46.808032    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-dshm\") pod \"sub-test-model-28686a-kserve-76b7757dbf-m28nm\" (UID: \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\") " pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"
Jul 03 14:47:46.808200 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:46.808090    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-tls-certs\") pod \"sub-test-model-28686a-kserve-76b7757dbf-m28nm\" (UID: \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\") " pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"
Jul 03 14:47:46.808200 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:46.808154    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-st7w4\" (UniqueName: \"kubernetes.io/projected/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-kube-api-access-st7w4\") pod \"sub-test-model-28686a-kserve-76b7757dbf-m28nm\" (UID: \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\") " pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"
Jul 03 14:47:46.808200 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:46.808175    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-model-cache\") pod \"sub-test-model-28686a-kserve-76b7757dbf-m28nm\" (UID: \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\") " pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"
Jul 03 14:47:46.808200 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:46.808194    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-tmp-dir\") pod \"sub-test-model-28686a-kserve-76b7757dbf-m28nm\" (UID: \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\") " pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"
Jul 03 14:47:46.909638 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:46.909606    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-home\") pod \"sub-test-model-28686a-kserve-76b7757dbf-m28nm\" (UID: \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\") " pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"
Jul 03 14:47:46.909797 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:46.909655    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-dshm\") pod \"sub-test-model-28686a-kserve-76b7757dbf-m28nm\" (UID: \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\") " pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"
Jul 03 14:47:46.909797 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:46.909764    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-tls-certs\") pod \"sub-test-model-28686a-kserve-76b7757dbf-m28nm\" (UID: \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\") " pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"
Jul 03 14:47:46.909882 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:46.909821    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st7w4\" (UniqueName: \"kubernetes.io/projected/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-kube-api-access-st7w4\") pod \"sub-test-model-28686a-kserve-76b7757dbf-m28nm\" (UID: \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\") " pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"
Jul 03 14:47:46.909882 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:46.909852    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-model-cache\") pod \"sub-test-model-28686a-kserve-76b7757dbf-m28nm\" (UID: \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\") " pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"
Jul 03 14:47:46.909882 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:46.909872    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-tmp-dir\") pod \"sub-test-model-28686a-kserve-76b7757dbf-m28nm\" (UID: \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\") " pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"
Jul 03 14:47:46.910063 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:46.910007    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-home\") pod \"sub-test-model-28686a-kserve-76b7757dbf-m28nm\" (UID: \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\") " pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"
Jul 03 14:47:46.910229 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:46.910206    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-model-cache\") pod \"sub-test-model-28686a-kserve-76b7757dbf-m28nm\" (UID: \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\") " pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"
Jul 03 14:47:46.910350 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:46.910329    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-tmp-dir\") pod \"sub-test-model-28686a-kserve-76b7757dbf-m28nm\" (UID: \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\") " pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"
Jul 03 14:47:46.912043 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:46.912023    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-dshm\") pod \"sub-test-model-28686a-kserve-76b7757dbf-m28nm\" (UID: \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\") " pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"
Jul 03 14:47:46.912178 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:46.912161    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-tls-certs\") pod \"sub-test-model-28686a-kserve-76b7757dbf-m28nm\" (UID: \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\") " pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"
Jul 03 14:47:46.917795 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:46.917771    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-st7w4\" (UniqueName: \"kubernetes.io/projected/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-kube-api-access-st7w4\") pod \"sub-test-model-28686a-kserve-76b7757dbf-m28nm\" (UID: \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\") " pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"
Jul 03 14:47:47.054393 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:47.054292    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"
Jul 03 14:47:47.190016 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:47.189991    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"]
Jul 03 14:47:47.191738 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:47:47.191712    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaf5c598f_23e6_4d32_80c2_b1ffbaacf2af.slice/crio-44ec8a75285ec374976058024f64d3441e0f3663f58ee139a1471fc335af8229 WatchSource:0}: Error finding container 44ec8a75285ec374976058024f64d3441e0f3663f58ee139a1471fc335af8229: Status 404 returned error can't find the container with id 44ec8a75285ec374976058024f64d3441e0f3663f58ee139a1471fc335af8229
Jul 03 14:47:47.366150 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:47.366111    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm" event={"ID":"af5c598f-23e6-4d32-80c2-b1ffbaacf2af","Type":"ContainerStarted","Data":"3a5984cf6dff757e8c7f9dbc3784dd7f1eb785bb6e15a7466de6afc132fc03b6"}
Jul 03 14:47:47.366350 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:47.366160    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm" event={"ID":"af5c598f-23e6-4d32-80c2-b1ffbaacf2af","Type":"ContainerStarted","Data":"44ec8a75285ec374976058024f64d3441e0f3663f58ee139a1471fc335af8229"}
Jul 03 14:47:47.390344 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:47.390280    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm" podStartSLOduration=1.390263984 podStartE2EDuration="1.390263984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:47:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:47:47.389781421 +0000 UTC m=+2380.839914862" watchObservedRunningTime="2026-07-03 14:47:47.390263984 +0000 UTC m=+2380.840397417"
Jul 03 14:47:53.168487 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:53.168452    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:47:57.059720 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:57.059689    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"
Jul 03 14:47:57.060143 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:57.059739    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"
Jul 03 14:47:57.068382 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:57.068361    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"
Jul 03 14:47:57.427646 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:47:57.427614    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"
Jul 03 14:48:15.735141 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:15.735107    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:48:32.243836 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:32.243751    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:48:37.880640 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:37.880608    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"]
Jul 03 14:48:38.748802 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:38.748764    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"]
Jul 03 14:48:42.941551 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:42.941514    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:48:52.900705 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:52.900658    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r" podUID="df66eb3d-6a80-4b99-81e9-28cc9c2ed6db" containerName="main" containerID="cri-o://a76047493f652fd3b7bb0c2f1425e91ff4c6449c6075acc7b9504c9264b0feeb" gracePeriod=45
Jul 03 14:48:53.161360 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:53.161300    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"
Jul 03 14:48:53.261103 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:53.261064    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxdbq\" (UniqueName: \"kubernetes.io/projected/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-kube-api-access-mxdbq\") pod \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\" (UID: \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\") "
Jul 03 14:48:53.261103 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:53.261108    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-home\") pod \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\" (UID: \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\") "
Jul 03 14:48:53.261377 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:53.261135    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-model-cache\") pod \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\" (UID: \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\") "
Jul 03 14:48:53.261377 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:53.261171    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-tls-certs\") pod \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\" (UID: \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\") "
Jul 03 14:48:53.261377 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:53.261238    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-tmp-dir\") pod \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\" (UID: \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\") "
Jul 03 14:48:53.261535 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:53.261387    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-dshm\") pod \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\" (UID: \"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db\") "
Jul 03 14:48:53.261535 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:53.261388    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-home" (OuterVolumeSpecName: "home") pod "df66eb3d-6a80-4b99-81e9-28cc9c2ed6db" (UID: "df66eb3d-6a80-4b99-81e9-28cc9c2ed6db"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:48:53.261535 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:53.261472    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-model-cache" (OuterVolumeSpecName: "model-cache") pod "df66eb3d-6a80-4b99-81e9-28cc9c2ed6db" (UID: "df66eb3d-6a80-4b99-81e9-28cc9c2ed6db"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:48:53.261535 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:53.261485    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "df66eb3d-6a80-4b99-81e9-28cc9c2ed6db" (UID: "df66eb3d-6a80-4b99-81e9-28cc9c2ed6db"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:48:53.261776 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:53.261756    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-tmp-dir\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:48:53.261843 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:53.261777    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-home\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:48:53.261843 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:53.261787    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-model-cache\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:48:53.263298 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:53.263273    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-kube-api-access-mxdbq" (OuterVolumeSpecName: "kube-api-access-mxdbq") pod "df66eb3d-6a80-4b99-81e9-28cc9c2ed6db" (UID: "df66eb3d-6a80-4b99-81e9-28cc9c2ed6db"). InnerVolumeSpecName "kube-api-access-mxdbq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:48:53.263391 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:53.263337    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "df66eb3d-6a80-4b99-81e9-28cc9c2ed6db" (UID: "df66eb3d-6a80-4b99-81e9-28cc9c2ed6db"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:48:53.263391 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:53.263346    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-dshm" (OuterVolumeSpecName: "dshm") pod "df66eb3d-6a80-4b99-81e9-28cc9c2ed6db" (UID: "df66eb3d-6a80-4b99-81e9-28cc9c2ed6db"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:48:53.363155 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:53.363103    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-dshm\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:48:53.363155 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:53.363151    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxdbq\" (UniqueName: \"kubernetes.io/projected/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-kube-api-access-mxdbq\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:48:53.363155 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:53.363163    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db-tls-certs\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:48:53.693260 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:53.693225    2563 generic.go:356] "Generic (PLEG): container finished" podID="df66eb3d-6a80-4b99-81e9-28cc9c2ed6db" containerID="a76047493f652fd3b7bb0c2f1425e91ff4c6449c6075acc7b9504c9264b0feeb" exitCode=0
Jul 03 14:48:53.693462 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:53.693271    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"
Jul 03 14:48:53.693462 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:53.693281    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r" event={"ID":"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db","Type":"ContainerDied","Data":"a76047493f652fd3b7bb0c2f1425e91ff4c6449c6075acc7b9504c9264b0feeb"}
Jul 03 14:48:53.693462 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:53.693313    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r" event={"ID":"df66eb3d-6a80-4b99-81e9-28cc9c2ed6db","Type":"ContainerDied","Data":"379e0ec89d9235d20c5603232b6ca90d136b2817ac3506026d2394426896e725"}
Jul 03 14:48:53.693462 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:53.693330    2563 scope.go:117] "RemoveContainer" containerID="a76047493f652fd3b7bb0c2f1425e91ff4c6449c6075acc7b9504c9264b0feeb"
Jul 03 14:48:53.703142 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:53.703116    2563 scope.go:117] "RemoveContainer" containerID="a76047493f652fd3b7bb0c2f1425e91ff4c6449c6075acc7b9504c9264b0feeb"
Jul 03 14:48:53.703378 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:48:53.703360    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a76047493f652fd3b7bb0c2f1425e91ff4c6449c6075acc7b9504c9264b0feeb\": container with ID starting with a76047493f652fd3b7bb0c2f1425e91ff4c6449c6075acc7b9504c9264b0feeb not found: ID does not exist" containerID="a76047493f652fd3b7bb0c2f1425e91ff4c6449c6075acc7b9504c9264b0feeb"
Jul 03 14:48:53.703438 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:53.703387    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a76047493f652fd3b7bb0c2f1425e91ff4c6449c6075acc7b9504c9264b0feeb"} err="failed to get container status \"a76047493f652fd3b7bb0c2f1425e91ff4c6449c6075acc7b9504c9264b0feeb\": rpc error: code = NotFound desc = could not find container \"a76047493f652fd3b7bb0c2f1425e91ff4c6449c6075acc7b9504c9264b0feeb\": container with ID starting with a76047493f652fd3b7bb0c2f1425e91ff4c6449c6075acc7b9504c9264b0feeb not found: ID does not exist"
Jul 03 14:48:53.714624 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:53.714600    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"]
Jul 03 14:48:53.719403 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:53.719383    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5f1e04/sub-test-model-5f1e04-kserve-7d5c4c8fbf-hzp2r"]
Jul 03 14:48:53.769378 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:53.769343    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm" podUID="af5c598f-23e6-4d32-80c2-b1ffbaacf2af" containerName="main" containerID="cri-o://3a5984cf6dff757e8c7f9dbc3784dd7f1eb785bb6e15a7466de6afc132fc03b6" gracePeriod=45
Jul 03 14:48:54.025226 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:54.025204    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"
Jul 03 14:48:54.069285 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:54.069258    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-tls-certs\") pod \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\" (UID: \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\") "
Jul 03 14:48:54.069450 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:54.069289    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-st7w4\" (UniqueName: \"kubernetes.io/projected/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-kube-api-access-st7w4\") pod \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\" (UID: \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\") "
Jul 03 14:48:54.069450 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:54.069366    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-home\") pod \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\" (UID: \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\") "
Jul 03 14:48:54.069450 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:54.069388    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-tmp-dir\") pod \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\" (UID: \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\") "
Jul 03 14:48:54.069450 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:54.069423    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-dshm\") pod \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\" (UID: \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\") "
Jul 03 14:48:54.069744 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:54.069457    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-model-cache\") pod \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\" (UID: \"af5c598f-23e6-4d32-80c2-b1ffbaacf2af\") "
Jul 03 14:48:54.069744 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:54.069670    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-home" (OuterVolumeSpecName: "home") pod "af5c598f-23e6-4d32-80c2-b1ffbaacf2af" (UID: "af5c598f-23e6-4d32-80c2-b1ffbaacf2af"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:48:54.069744 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:54.069695    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "af5c598f-23e6-4d32-80c2-b1ffbaacf2af" (UID: "af5c598f-23e6-4d32-80c2-b1ffbaacf2af"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:48:54.069950 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:54.069798    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-model-cache" (OuterVolumeSpecName: "model-cache") pod "af5c598f-23e6-4d32-80c2-b1ffbaacf2af" (UID: "af5c598f-23e6-4d32-80c2-b1ffbaacf2af"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:48:54.069950 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:54.069829    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-home\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:48:54.069950 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:54.069847    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-tmp-dir\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:48:54.071473 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:54.071438    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-kube-api-access-st7w4" (OuterVolumeSpecName: "kube-api-access-st7w4") pod "af5c598f-23e6-4d32-80c2-b1ffbaacf2af" (UID: "af5c598f-23e6-4d32-80c2-b1ffbaacf2af"). InnerVolumeSpecName "kube-api-access-st7w4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:48:54.071473 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:54.071450    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "af5c598f-23e6-4d32-80c2-b1ffbaacf2af" (UID: "af5c598f-23e6-4d32-80c2-b1ffbaacf2af"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:48:54.071625 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:54.071517    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-dshm" (OuterVolumeSpecName: "dshm") pod "af5c598f-23e6-4d32-80c2-b1ffbaacf2af" (UID: "af5c598f-23e6-4d32-80c2-b1ffbaacf2af"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:48:54.171218 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:54.171170    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-dshm\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:48:54.171218 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:54.171211    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-model-cache\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:48:54.171218 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:54.171222    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-tls-certs\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:48:54.171218 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:54.171231    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-st7w4\" (UniqueName: \"kubernetes.io/projected/af5c598f-23e6-4d32-80c2-b1ffbaacf2af-kube-api-access-st7w4\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:48:54.700162 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:54.700124    2563 generic.go:356] "Generic (PLEG): container finished" podID="af5c598f-23e6-4d32-80c2-b1ffbaacf2af" containerID="3a5984cf6dff757e8c7f9dbc3784dd7f1eb785bb6e15a7466de6afc132fc03b6" exitCode=0
Jul 03 14:48:54.700162 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:54.700165    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"
Jul 03 14:48:54.700445 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:54.700209    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm" event={"ID":"af5c598f-23e6-4d32-80c2-b1ffbaacf2af","Type":"ContainerDied","Data":"3a5984cf6dff757e8c7f9dbc3784dd7f1eb785bb6e15a7466de6afc132fc03b6"}
Jul 03 14:48:54.700445 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:54.700255    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm" event={"ID":"af5c598f-23e6-4d32-80c2-b1ffbaacf2af","Type":"ContainerDied","Data":"44ec8a75285ec374976058024f64d3441e0f3663f58ee139a1471fc335af8229"}
Jul 03 14:48:54.700445 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:54.700280    2563 scope.go:117] "RemoveContainer" containerID="3a5984cf6dff757e8c7f9dbc3784dd7f1eb785bb6e15a7466de6afc132fc03b6"
Jul 03 14:48:54.709479 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:54.709456    2563 scope.go:117] "RemoveContainer" containerID="3a5984cf6dff757e8c7f9dbc3784dd7f1eb785bb6e15a7466de6afc132fc03b6"
Jul 03 14:48:54.709765 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:48:54.709742    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a5984cf6dff757e8c7f9dbc3784dd7f1eb785bb6e15a7466de6afc132fc03b6\": container with ID starting with 3a5984cf6dff757e8c7f9dbc3784dd7f1eb785bb6e15a7466de6afc132fc03b6 not found: ID does not exist" containerID="3a5984cf6dff757e8c7f9dbc3784dd7f1eb785bb6e15a7466de6afc132fc03b6"
Jul 03 14:48:54.709849 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:54.709777    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a5984cf6dff757e8c7f9dbc3784dd7f1eb785bb6e15a7466de6afc132fc03b6"} err="failed to get container status \"3a5984cf6dff757e8c7f9dbc3784dd7f1eb785bb6e15a7466de6afc132fc03b6\": rpc error: code = NotFound desc = could not find container \"3a5984cf6dff757e8c7f9dbc3784dd7f1eb785bb6e15a7466de6afc132fc03b6\": container with ID starting with 3a5984cf6dff757e8c7f9dbc3784dd7f1eb785bb6e15a7466de6afc132fc03b6 not found: ID does not exist"
Jul 03 14:48:54.722599 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:54.722569    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"]
Jul 03 14:48:54.728509 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:54.728484    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-28686a/sub-test-model-28686a-kserve-76b7757dbf-m28nm"]
Jul 03 14:48:54.745816 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:48:54.745789    2563 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaf5c598f_23e6_4d32_80c2_b1ffbaacf2af.slice/crio-44ec8a75285ec374976058024f64d3441e0f3663f58ee139a1471fc335af8229\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaf5c598f_23e6_4d32_80c2_b1ffbaacf2af.slice\": RecentStats: unable to find data in memory cache]"
Jul 03 14:48:55.059109 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:55.059034    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af5c598f-23e6-4d32-80c2-b1ffbaacf2af" path="/var/lib/kubelet/pods/af5c598f-23e6-4d32-80c2-b1ffbaacf2af/volumes"
Jul 03 14:48:55.059453 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:48:55.059419    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df66eb3d-6a80-4b99-81e9-28cc9c2ed6db" path="/var/lib/kubelet/pods/df66eb3d-6a80-4b99-81e9-28cc9c2ed6db/volumes"
Jul 03 14:49:02.234497 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.234457    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"]
Jul 03 14:49:02.234886 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.234850    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df66eb3d-6a80-4b99-81e9-28cc9c2ed6db" containerName="main"
Jul 03 14:49:02.234886 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.234863    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="df66eb3d-6a80-4b99-81e9-28cc9c2ed6db" containerName="main"
Jul 03 14:49:02.234886 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.234881    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af5c598f-23e6-4d32-80c2-b1ffbaacf2af" containerName="main"
Jul 03 14:49:02.235017 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.234887    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="af5c598f-23e6-4d32-80c2-b1ffbaacf2af" containerName="main"
Jul 03 14:49:02.235017 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.234946    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="df66eb3d-6a80-4b99-81e9-28cc9c2ed6db" containerName="main"
Jul 03 14:49:02.235017 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.234953    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="af5c598f-23e6-4d32-80c2-b1ffbaacf2af" containerName="main"
Jul 03 14:49:02.239221 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.239205    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"
Jul 03 14:49:02.241893 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.241874    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5f1e04\"/\"rate-test-model-5f1e04-kserve-self-signed-certs\""
Jul 03 14:49:02.242683 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.242663    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5f1e04\"/\"default-dockercfg-b7x8t\""
Jul 03 14:49:02.242782 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.242685    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5f1e04\"/\"kube-root-ca.crt\""
Jul 03 14:49:02.242782 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.242666    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5f1e04\"/\"openshift-service-ca.crt\""
Jul 03 14:49:02.249153 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.249129    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"]
Jul 03 14:49:02.349400 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.349352    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4759ebfe-926b-4964-82b9-58cdf515bb0b-model-cache\") pod \"rate-test-model-5f1e04-kserve-844c78948c-nz7g4\" (UID: \"4759ebfe-926b-4964-82b9-58cdf515bb0b\") " pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"
Jul 03 14:49:02.349588 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.349416    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4759ebfe-926b-4964-82b9-58cdf515bb0b-dshm\") pod \"rate-test-model-5f1e04-kserve-844c78948c-nz7g4\" (UID: \"4759ebfe-926b-4964-82b9-58cdf515bb0b\") " pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"
Jul 03 14:49:02.349588 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.349451    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4759ebfe-926b-4964-82b9-58cdf515bb0b-tls-certs\") pod \"rate-test-model-5f1e04-kserve-844c78948c-nz7g4\" (UID: \"4759ebfe-926b-4964-82b9-58cdf515bb0b\") " pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"
Jul 03 14:49:02.349588 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.349541    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpmzk\" (UniqueName: \"kubernetes.io/projected/4759ebfe-926b-4964-82b9-58cdf515bb0b-kube-api-access-hpmzk\") pod \"rate-test-model-5f1e04-kserve-844c78948c-nz7g4\" (UID: \"4759ebfe-926b-4964-82b9-58cdf515bb0b\") " pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"
Jul 03 14:49:02.349588 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.349566    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4759ebfe-926b-4964-82b9-58cdf515bb0b-home\") pod \"rate-test-model-5f1e04-kserve-844c78948c-nz7g4\" (UID: \"4759ebfe-926b-4964-82b9-58cdf515bb0b\") " pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"
Jul 03 14:49:02.349726 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.349590    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4759ebfe-926b-4964-82b9-58cdf515bb0b-tmp-dir\") pod \"rate-test-model-5f1e04-kserve-844c78948c-nz7g4\" (UID: \"4759ebfe-926b-4964-82b9-58cdf515bb0b\") " pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"
Jul 03 14:49:02.450549 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.450507    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpmzk\" (UniqueName: \"kubernetes.io/projected/4759ebfe-926b-4964-82b9-58cdf515bb0b-kube-api-access-hpmzk\") pod \"rate-test-model-5f1e04-kserve-844c78948c-nz7g4\" (UID: \"4759ebfe-926b-4964-82b9-58cdf515bb0b\") " pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"
Jul 03 14:49:02.450549 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.450547    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4759ebfe-926b-4964-82b9-58cdf515bb0b-home\") pod \"rate-test-model-5f1e04-kserve-844c78948c-nz7g4\" (UID: \"4759ebfe-926b-4964-82b9-58cdf515bb0b\") " pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"
Jul 03 14:49:02.450828 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.450571    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4759ebfe-926b-4964-82b9-58cdf515bb0b-tmp-dir\") pod \"rate-test-model-5f1e04-kserve-844c78948c-nz7g4\" (UID: \"4759ebfe-926b-4964-82b9-58cdf515bb0b\") " pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"
Jul 03 14:49:02.450828 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.450601    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4759ebfe-926b-4964-82b9-58cdf515bb0b-model-cache\") pod \"rate-test-model-5f1e04-kserve-844c78948c-nz7g4\" (UID: \"4759ebfe-926b-4964-82b9-58cdf515bb0b\") " pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"
Jul 03 14:49:02.450828 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.450643    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4759ebfe-926b-4964-82b9-58cdf515bb0b-dshm\") pod \"rate-test-model-5f1e04-kserve-844c78948c-nz7g4\" (UID: \"4759ebfe-926b-4964-82b9-58cdf515bb0b\") " pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"
Jul 03 14:49:02.450828 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.450683    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4759ebfe-926b-4964-82b9-58cdf515bb0b-tls-certs\") pod \"rate-test-model-5f1e04-kserve-844c78948c-nz7g4\" (UID: \"4759ebfe-926b-4964-82b9-58cdf515bb0b\") " pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"
Jul 03 14:49:02.451038 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.451003    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4759ebfe-926b-4964-82b9-58cdf515bb0b-home\") pod \"rate-test-model-5f1e04-kserve-844c78948c-nz7g4\" (UID: \"4759ebfe-926b-4964-82b9-58cdf515bb0b\") " pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"
Jul 03 14:49:02.451080 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.451043    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4759ebfe-926b-4964-82b9-58cdf515bb0b-tmp-dir\") pod \"rate-test-model-5f1e04-kserve-844c78948c-nz7g4\" (UID: \"4759ebfe-926b-4964-82b9-58cdf515bb0b\") " pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"
Jul 03 14:49:02.451080 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.451069    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4759ebfe-926b-4964-82b9-58cdf515bb0b-model-cache\") pod \"rate-test-model-5f1e04-kserve-844c78948c-nz7g4\" (UID: \"4759ebfe-926b-4964-82b9-58cdf515bb0b\") " pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"
Jul 03 14:49:02.452998 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.452960    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4759ebfe-926b-4964-82b9-58cdf515bb0b-dshm\") pod \"rate-test-model-5f1e04-kserve-844c78948c-nz7g4\" (UID: \"4759ebfe-926b-4964-82b9-58cdf515bb0b\") " pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"
Jul 03 14:49:02.453173 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.453155    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4759ebfe-926b-4964-82b9-58cdf515bb0b-tls-certs\") pod \"rate-test-model-5f1e04-kserve-844c78948c-nz7g4\" (UID: \"4759ebfe-926b-4964-82b9-58cdf515bb0b\") " pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"
Jul 03 14:49:02.459428 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.459402    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpmzk\" (UniqueName: \"kubernetes.io/projected/4759ebfe-926b-4964-82b9-58cdf515bb0b-kube-api-access-hpmzk\") pod \"rate-test-model-5f1e04-kserve-844c78948c-nz7g4\" (UID: \"4759ebfe-926b-4964-82b9-58cdf515bb0b\") " pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"
Jul 03 14:49:02.548531 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.548445    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"
Jul 03 14:49:02.684643 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.684613    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"]
Jul 03 14:49:02.686426 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:49:02.686385    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4759ebfe_926b_4964_82b9_58cdf515bb0b.slice/crio-cdca3ca23d615bbad425208a8ec045ac1cf5fc73b0926ed4efc9222e0aa8c5ce WatchSource:0}: Error finding container cdca3ca23d615bbad425208a8ec045ac1cf5fc73b0926ed4efc9222e0aa8c5ce: Status 404 returned error can't find the container with id cdca3ca23d615bbad425208a8ec045ac1cf5fc73b0926ed4efc9222e0aa8c5ce
Jul 03 14:49:02.741939 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:02.741909    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4" event={"ID":"4759ebfe-926b-4964-82b9-58cdf515bb0b","Type":"ContainerStarted","Data":"cdca3ca23d615bbad425208a8ec045ac1cf5fc73b0926ed4efc9222e0aa8c5ce"}
Jul 03 14:49:03.749026 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:03.748985    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4" event={"ID":"4759ebfe-926b-4964-82b9-58cdf515bb0b","Type":"ContainerStarted","Data":"c139d2f2b787936146f9256bcbb4949e8a321efd7a87ccb6dc45e25070e996f3"}
Jul 03 14:49:03.765633 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:03.765569    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4" podStartSLOduration=1.7655492590000001 podStartE2EDuration="1.765549259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:49:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:49:03.765204181 +0000 UTC m=+2457.215337635" watchObservedRunningTime="2026-07-03 14:49:03.765549259 +0000 UTC m=+2457.215682721"
Jul 03 14:49:06.043415 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:06.043377    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:49:12.548850 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:12.548816    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"
Jul 03 14:49:12.548850 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:12.548856    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"
Jul 03 14:49:12.561089 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:12.561063    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"
Jul 03 14:49:12.805216 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:12.805131    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"
Jul 03 14:49:41.345184 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.345150    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"]
Jul 03 14:49:41.348948 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.348931    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"
Jul 03 14:49:41.352105 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.352082    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-28686a\"/\"openshift-service-ca.crt\""
Jul 03 14:49:41.352235 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.352172    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-28686a\"/\"rate-test-model-28686a-kserve-self-signed-certs\""
Jul 03 14:49:41.352235 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.352213    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-28686a\"/\"kube-root-ca.crt\""
Jul 03 14:49:41.352235 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.352223    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-28686a\"/\"default-dockercfg-84tzh\""
Jul 03 14:49:41.357947 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.357922    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"]
Jul 03 14:49:41.433210 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.433176    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60d8d98d-917a-4ede-bf53-36e48321adad-home\") pod \"rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q\" (UID: \"60d8d98d-917a-4ede-bf53-36e48321adad\") " pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"
Jul 03 14:49:41.433210 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.433213    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60d8d98d-917a-4ede-bf53-36e48321adad-tmp-dir\") pod \"rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q\" (UID: \"60d8d98d-917a-4ede-bf53-36e48321adad\") " pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"
Jul 03 14:49:41.433427 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.433238    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60d8d98d-917a-4ede-bf53-36e48321adad-tls-certs\") pod \"rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q\" (UID: \"60d8d98d-917a-4ede-bf53-36e48321adad\") " pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"
Jul 03 14:49:41.433427 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.433292    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60d8d98d-917a-4ede-bf53-36e48321adad-model-cache\") pod \"rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q\" (UID: \"60d8d98d-917a-4ede-bf53-36e48321adad\") " pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"
Jul 03 14:49:41.433427 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.433343    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60d8d98d-917a-4ede-bf53-36e48321adad-dshm\") pod \"rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q\" (UID: \"60d8d98d-917a-4ede-bf53-36e48321adad\") " pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"
Jul 03 14:49:41.433427 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.433401    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf2vv\" (UniqueName: \"kubernetes.io/projected/60d8d98d-917a-4ede-bf53-36e48321adad-kube-api-access-mf2vv\") pod \"rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q\" (UID: \"60d8d98d-917a-4ede-bf53-36e48321adad\") " pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"
Jul 03 14:49:41.534824 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.534790    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf2vv\" (UniqueName: \"kubernetes.io/projected/60d8d98d-917a-4ede-bf53-36e48321adad-kube-api-access-mf2vv\") pod \"rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q\" (UID: \"60d8d98d-917a-4ede-bf53-36e48321adad\") " pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"
Jul 03 14:49:41.535029 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.534948    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60d8d98d-917a-4ede-bf53-36e48321adad-home\") pod \"rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q\" (UID: \"60d8d98d-917a-4ede-bf53-36e48321adad\") " pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"
Jul 03 14:49:41.535029 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.535016    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60d8d98d-917a-4ede-bf53-36e48321adad-tmp-dir\") pod \"rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q\" (UID: \"60d8d98d-917a-4ede-bf53-36e48321adad\") " pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"
Jul 03 14:49:41.535129 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.535045    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60d8d98d-917a-4ede-bf53-36e48321adad-tls-certs\") pod \"rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q\" (UID: \"60d8d98d-917a-4ede-bf53-36e48321adad\") " pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"
Jul 03 14:49:41.535129 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.535068    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60d8d98d-917a-4ede-bf53-36e48321adad-model-cache\") pod \"rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q\" (UID: \"60d8d98d-917a-4ede-bf53-36e48321adad\") " pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"
Jul 03 14:49:41.535129 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.535091    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60d8d98d-917a-4ede-bf53-36e48321adad-dshm\") pod \"rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q\" (UID: \"60d8d98d-917a-4ede-bf53-36e48321adad\") " pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"
Jul 03 14:49:41.535399 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.535379    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60d8d98d-917a-4ede-bf53-36e48321adad-home\") pod \"rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q\" (UID: \"60d8d98d-917a-4ede-bf53-36e48321adad\") " pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"
Jul 03 14:49:41.535446 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.535399    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60d8d98d-917a-4ede-bf53-36e48321adad-tmp-dir\") pod \"rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q\" (UID: \"60d8d98d-917a-4ede-bf53-36e48321adad\") " pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"
Jul 03 14:49:41.535506 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.535489    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60d8d98d-917a-4ede-bf53-36e48321adad-model-cache\") pod \"rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q\" (UID: \"60d8d98d-917a-4ede-bf53-36e48321adad\") " pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"
Jul 03 14:49:41.537327 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.537307    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60d8d98d-917a-4ede-bf53-36e48321adad-dshm\") pod \"rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q\" (UID: \"60d8d98d-917a-4ede-bf53-36e48321adad\") " pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"
Jul 03 14:49:41.537578 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.537560    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60d8d98d-917a-4ede-bf53-36e48321adad-tls-certs\") pod \"rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q\" (UID: \"60d8d98d-917a-4ede-bf53-36e48321adad\") " pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"
Jul 03 14:49:41.546154 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.546105    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf2vv\" (UniqueName: \"kubernetes.io/projected/60d8d98d-917a-4ede-bf53-36e48321adad-kube-api-access-mf2vv\") pod \"rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q\" (UID: \"60d8d98d-917a-4ede-bf53-36e48321adad\") " pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"
Jul 03 14:49:41.659258 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.659226    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"
Jul 03 14:49:41.792662 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.792640    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"]
Jul 03 14:49:41.795245 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:49:41.795218    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod60d8d98d_917a_4ede_bf53_36e48321adad.slice/crio-223691e59e98eba0c4440b1e1de0116ce61de3168a998f25db0fa98ef5211133 WatchSource:0}: Error finding container 223691e59e98eba0c4440b1e1de0116ce61de3168a998f25db0fa98ef5211133: Status 404 returned error can't find the container with id 223691e59e98eba0c4440b1e1de0116ce61de3168a998f25db0fa98ef5211133
Jul 03 14:49:41.934225 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.934132    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q" event={"ID":"60d8d98d-917a-4ede-bf53-36e48321adad","Type":"ContainerStarted","Data":"f2f2a3543d0857fd8848dd459f9536bd305744aaca558db35b4075a2500b5aad"}
Jul 03 14:49:41.934225 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.934188    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q" event={"ID":"60d8d98d-917a-4ede-bf53-36e48321adad","Type":"ContainerStarted","Data":"223691e59e98eba0c4440b1e1de0116ce61de3168a998f25db0fa98ef5211133"}
Jul 03 14:49:41.952488 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:41.952423    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q" podStartSLOduration=0.952402592 podStartE2EDuration="952.402592ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 14:49:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:49:41.952193887 +0000 UTC m=+2495.402327347" watchObservedRunningTime="2026-07-03 14:49:41.952402592 +0000 UTC m=+2495.402536030"
Jul 03 14:49:46.551353 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:46.551316    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:49:51.660316 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:51.660272    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"
Jul 03 14:49:51.660316 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:51.660323    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"
Jul 03 14:49:51.673303 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:51.673271    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"
Jul 03 14:49:51.995145 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:49:51.995059    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"
Jul 03 14:50:02.947007 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:02.946953    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:50:22.050140 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:22.050104    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:50:26.566767 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:26.566733    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"]
Jul 03 14:50:27.525535 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:27.525506    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"]
Jul 03 14:50:31.052489 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:31.052449    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7123-139783418424768"]
Jul 03 14:50:31.056331 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:31.056304    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783418424768"
Jul 03 14:50:31.059339 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:31.059313    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-l8qw4\""
Jul 03 14:50:31.067849 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:31.067821    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7123-139783418424768"]
Jul 03 14:50:31.234841 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:31.234798    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjks5\" (UniqueName: \"kubernetes.io/projected/9f1e1c94-dc73-409e-b3e6-5d9421543043-kube-api-access-tjks5\") pod \"test-curl-7123-139783418424768\" (UID: \"9f1e1c94-dc73-409e-b3e6-5d9421543043\") " pod="opendatahub/test-curl-7123-139783418424768"
Jul 03 14:50:31.336268 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:31.336186    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjks5\" (UniqueName: \"kubernetes.io/projected/9f1e1c94-dc73-409e-b3e6-5d9421543043-kube-api-access-tjks5\") pod \"test-curl-7123-139783418424768\" (UID: \"9f1e1c94-dc73-409e-b3e6-5d9421543043\") " pod="opendatahub/test-curl-7123-139783418424768"
Jul 03 14:50:31.344933 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:31.344899    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjks5\" (UniqueName: \"kubernetes.io/projected/9f1e1c94-dc73-409e-b3e6-5d9421543043-kube-api-access-tjks5\") pod \"test-curl-7123-139783418424768\" (UID: \"9f1e1c94-dc73-409e-b3e6-5d9421543043\") " pod="opendatahub/test-curl-7123-139783418424768"
Jul 03 14:50:31.373738 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:31.373708    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783418424768"
Jul 03 14:50:31.710542 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:31.710520    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7123-139783418424768"]
Jul 03 14:50:31.712488 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:50:31.712461    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f1e1c94_dc73_409e_b3e6_5d9421543043.slice/crio-4df9a712785184c902fd4fd8ed77cb9b03fd7ccfe7e8aeb91302cfab7f628f21 WatchSource:0}: Error finding container 4df9a712785184c902fd4fd8ed77cb9b03fd7ccfe7e8aeb91302cfab7f628f21: Status 404 returned error can't find the container with id 4df9a712785184c902fd4fd8ed77cb9b03fd7ccfe7e8aeb91302cfab7f628f21
Jul 03 14:50:31.713623 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:31.713606    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 03 14:50:32.182354 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:32.182317    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7123-139783418424768" event={"ID":"9f1e1c94-dc73-409e-b3e6-5d9421543043","Type":"ContainerStarted","Data":"4df9a712785184c902fd4fd8ed77cb9b03fd7ccfe7e8aeb91302cfab7f628f21"}
Jul 03 14:50:33.188121 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:33.188084    2563 generic.go:356] "Generic (PLEG): container finished" podID="9f1e1c94-dc73-409e-b3e6-5d9421543043" containerID="f20926a9d90a69379dd6ef4829acfdb77afda66fb17a7d16e8b9bc39b3f5581e" exitCode=0
Jul 03 14:50:33.188513 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:33.188170    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7123-139783418424768" event={"ID":"9f1e1c94-dc73-409e-b3e6-5d9421543043","Type":"ContainerDied","Data":"f20926a9d90a69379dd6ef4829acfdb77afda66fb17a7d16e8b9bc39b3f5581e"}
Jul 03 14:50:34.327918 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:34.327893    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783418424768"
Jul 03 14:50:34.362998 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:34.362943    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7123-139783418424768_9f1e1c94-dc73-409e-b3e6-5d9421543043/test-curl-7123-139783418424768/0.log"
Jul 03 14:50:34.387277 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:34.387247    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7123-139783418424768"]
Jul 03 14:50:34.390705 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:34.390679    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7123-139783418424768"]
Jul 03 14:50:34.468283 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:34.468199    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjks5\" (UniqueName: \"kubernetes.io/projected/9f1e1c94-dc73-409e-b3e6-5d9421543043-kube-api-access-tjks5\") pod \"9f1e1c94-dc73-409e-b3e6-5d9421543043\" (UID: \"9f1e1c94-dc73-409e-b3e6-5d9421543043\") "
Jul 03 14:50:34.470378 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:34.470350    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f1e1c94-dc73-409e-b3e6-5d9421543043-kube-api-access-tjks5" (OuterVolumeSpecName: "kube-api-access-tjks5") pod "9f1e1c94-dc73-409e-b3e6-5d9421543043" (UID: "9f1e1c94-dc73-409e-b3e6-5d9421543043"). InnerVolumeSpecName "kube-api-access-tjks5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:50:34.512702 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:34.512670    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7123-139783418424880"]
Jul 03 14:50:34.513084 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:34.513070    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f1e1c94-dc73-409e-b3e6-5d9421543043" containerName="test-curl-7123-139783418424768"
Jul 03 14:50:34.513137 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:34.513086    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f1e1c94-dc73-409e-b3e6-5d9421543043" containerName="test-curl-7123-139783418424768"
Jul 03 14:50:34.513173 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:34.513158    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f1e1c94-dc73-409e-b3e6-5d9421543043" containerName="test-curl-7123-139783418424768"
Jul 03 14:50:34.516191 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:34.516177    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783418424880"
Jul 03 14:50:34.522756 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:34.522734    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7123-139783418424880"]
Jul 03 14:50:34.569691 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:34.569657    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjks5\" (UniqueName: \"kubernetes.io/projected/9f1e1c94-dc73-409e-b3e6-5d9421543043-kube-api-access-tjks5\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:50:34.670545 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:34.670509    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5j4l\" (UniqueName: \"kubernetes.io/projected/05da8896-b009-4b15-8e91-72ff3554caa0-kube-api-access-r5j4l\") pod \"test-curl-7123-139783418424880\" (UID: \"05da8896-b009-4b15-8e91-72ff3554caa0\") " pod="opendatahub/test-curl-7123-139783418424880"
Jul 03 14:50:34.771309 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:34.771231    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5j4l\" (UniqueName: \"kubernetes.io/projected/05da8896-b009-4b15-8e91-72ff3554caa0-kube-api-access-r5j4l\") pod \"test-curl-7123-139783418424880\" (UID: \"05da8896-b009-4b15-8e91-72ff3554caa0\") " pod="opendatahub/test-curl-7123-139783418424880"
Jul 03 14:50:34.781368 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:34.781337    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5j4l\" (UniqueName: \"kubernetes.io/projected/05da8896-b009-4b15-8e91-72ff3554caa0-kube-api-access-r5j4l\") pod \"test-curl-7123-139783418424880\" (UID: \"05da8896-b009-4b15-8e91-72ff3554caa0\") " pod="opendatahub/test-curl-7123-139783418424880"
Jul 03 14:50:34.826273 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:34.826230    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783418424880"
Jul 03 14:50:34.960997 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:34.960949    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7123-139783418424880"]
Jul 03 14:50:34.964072 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:50:34.964042    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod05da8896_b009_4b15_8e91_72ff3554caa0.slice/crio-b48f5b4f95fcb28b199a20c8f4312b4ea0d57f135c44422d36f41db6748eb514 WatchSource:0}: Error finding container b48f5b4f95fcb28b199a20c8f4312b4ea0d57f135c44422d36f41db6748eb514: Status 404 returned error can't find the container with id b48f5b4f95fcb28b199a20c8f4312b4ea0d57f135c44422d36f41db6748eb514
Jul 03 14:50:35.065601 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:35.065519    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f1e1c94-dc73-409e-b3e6-5d9421543043" path="/var/lib/kubelet/pods/9f1e1c94-dc73-409e-b3e6-5d9421543043/volumes"
Jul 03 14:50:35.199011 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:35.198957    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783418424768"
Jul 03 14:50:35.199289 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:35.199033    2563 scope.go:117] "RemoveContainer" containerID="f20926a9d90a69379dd6ef4829acfdb77afda66fb17a7d16e8b9bc39b3f5581e"
Jul 03 14:50:35.200356 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:35.200325    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7123-139783418424880" event={"ID":"05da8896-b009-4b15-8e91-72ff3554caa0","Type":"ContainerStarted","Data":"b48f5b4f95fcb28b199a20c8f4312b4ea0d57f135c44422d36f41db6748eb514"}
Jul 03 14:50:36.208897 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:36.208864    2563 generic.go:356] "Generic (PLEG): container finished" podID="05da8896-b009-4b15-8e91-72ff3554caa0" containerID="d4f72d5569336dfe9a6bb06fb817107647daee242426556b1e50e801a267cf49" exitCode=0
Jul 03 14:50:36.209305 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:36.208956    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7123-139783418424880" event={"ID":"05da8896-b009-4b15-8e91-72ff3554caa0","Type":"ContainerDied","Data":"d4f72d5569336dfe9a6bb06fb817107647daee242426556b1e50e801a267cf49"}
Jul 03 14:50:37.354018 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:37.353994    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783418424880"
Jul 03 14:50:37.377603 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:37.377571    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7123-139783418424880_05da8896-b009-4b15-8e91-72ff3554caa0/test-curl-7123-139783418424880/0.log"
Jul 03 14:50:37.404458 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:37.404419    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7123-139783418424880"]
Jul 03 14:50:37.408906 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:37.408877    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7123-139783418424880"]
Jul 03 14:50:37.494494 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:37.494399    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5j4l\" (UniqueName: \"kubernetes.io/projected/05da8896-b009-4b15-8e91-72ff3554caa0-kube-api-access-r5j4l\") pod \"05da8896-b009-4b15-8e91-72ff3554caa0\" (UID: \"05da8896-b009-4b15-8e91-72ff3554caa0\") "
Jul 03 14:50:37.496446 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:37.496420    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05da8896-b009-4b15-8e91-72ff3554caa0-kube-api-access-r5j4l" (OuterVolumeSpecName: "kube-api-access-r5j4l") pod "05da8896-b009-4b15-8e91-72ff3554caa0" (UID: "05da8896-b009-4b15-8e91-72ff3554caa0"). InnerVolumeSpecName "kube-api-access-r5j4l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:50:37.596036 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:37.596004    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5j4l\" (UniqueName: \"kubernetes.io/projected/05da8896-b009-4b15-8e91-72ff3554caa0-kube-api-access-r5j4l\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:50:38.220232 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:38.220201    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b48f5b4f95fcb28b199a20c8f4312b4ea0d57f135c44422d36f41db6748eb514"
Jul 03 14:50:38.220232 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:38.220206    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783418424880"
Jul 03 14:50:38.230179 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:50:38.230148    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7123-139783418424880\" is forbidden: User \"system:node:ip-10-0-142-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-233.ec2.internal' and this object" podUID="05da8896-b009-4b15-8e91-72ff3554caa0" pod="opendatahub/test-curl-7123-139783418424880"
Jul 03 14:50:38.741616 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:38.741579    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:50:39.059181 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:39.059099    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05da8896-b009-4b15-8e91-72ff3554caa0" path="/var/lib/kubelet/pods/05da8896-b009-4b15-8e91-72ff3554caa0/volumes"
Jul 03 14:50:40.142915 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:40.142877    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7123-139783413036496"]
Jul 03 14:50:40.143347 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:40.143279    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05da8896-b009-4b15-8e91-72ff3554caa0" containerName="test-curl-7123-139783418424880"
Jul 03 14:50:40.143347 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:40.143294    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="05da8896-b009-4b15-8e91-72ff3554caa0" containerName="test-curl-7123-139783418424880"
Jul 03 14:50:40.143435 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:40.143367    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="05da8896-b009-4b15-8e91-72ff3554caa0" containerName="test-curl-7123-139783418424880"
Jul 03 14:50:40.146352 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:40.146336    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413036496"
Jul 03 14:50:40.148778 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:40.148756    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-l8qw4\""
Jul 03 14:50:40.153366 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:40.153159    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7123-139783413036496"]
Jul 03 14:50:40.219445 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:40.219408    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kxjd\" (UniqueName: \"kubernetes.io/projected/47409e59-b67f-4289-b5f0-52fb080eb9b5-kube-api-access-2kxjd\") pod \"test-curl-7123-139783413036496\" (UID: \"47409e59-b67f-4289-b5f0-52fb080eb9b5\") " pod="opendatahub/test-curl-7123-139783413036496"
Jul 03 14:50:40.320270 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:40.320226    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kxjd\" (UniqueName: \"kubernetes.io/projected/47409e59-b67f-4289-b5f0-52fb080eb9b5-kube-api-access-2kxjd\") pod \"test-curl-7123-139783413036496\" (UID: \"47409e59-b67f-4289-b5f0-52fb080eb9b5\") " pod="opendatahub/test-curl-7123-139783413036496"
Jul 03 14:50:40.328757 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:40.328726    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kxjd\" (UniqueName: \"kubernetes.io/projected/47409e59-b67f-4289-b5f0-52fb080eb9b5-kube-api-access-2kxjd\") pod \"test-curl-7123-139783413036496\" (UID: \"47409e59-b67f-4289-b5f0-52fb080eb9b5\") " pod="opendatahub/test-curl-7123-139783413036496"
Jul 03 14:50:40.456331 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:40.456244    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413036496"
Jul 03 14:50:40.588330 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:40.588305    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7123-139783413036496"]
Jul 03 14:50:40.591179 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:50:40.591148    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47409e59_b67f_4289_b5f0_52fb080eb9b5.slice/crio-ec35bb6773cc172580983164b9be6695adf2b13f8737ef42dd040a6570da4e07 WatchSource:0}: Error finding container ec35bb6773cc172580983164b9be6695adf2b13f8737ef42dd040a6570da4e07: Status 404 returned error can't find the container with id ec35bb6773cc172580983164b9be6695adf2b13f8737ef42dd040a6570da4e07
Jul 03 14:50:41.236477 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:41.236382    2563 generic.go:356] "Generic (PLEG): container finished" podID="47409e59-b67f-4289-b5f0-52fb080eb9b5" containerID="3fea55b072193abb9963e75f34879faedba06fc140d799e7314893edbbcf87bd" exitCode=0
Jul 03 14:50:41.236853 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:41.236475    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7123-139783413036496" event={"ID":"47409e59-b67f-4289-b5f0-52fb080eb9b5","Type":"ContainerDied","Data":"3fea55b072193abb9963e75f34879faedba06fc140d799e7314893edbbcf87bd"}
Jul 03 14:50:41.236853 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:41.236526    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7123-139783413036496" event={"ID":"47409e59-b67f-4289-b5f0-52fb080eb9b5","Type":"ContainerStarted","Data":"ec35bb6773cc172580983164b9be6695adf2b13f8737ef42dd040a6570da4e07"}
Jul 03 14:50:41.586786 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:41.586669    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4" podUID="4759ebfe-926b-4964-82b9-58cdf515bb0b" containerName="main" containerID="cri-o://c139d2f2b787936146f9256bcbb4949e8a321efd7a87ccb6dc45e25070e996f3" gracePeriod=45
Jul 03 14:50:41.847140 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:41.847069    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"
Jul 03 14:50:41.936693 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:41.936641    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4759ebfe-926b-4964-82b9-58cdf515bb0b-home\") pod \"4759ebfe-926b-4964-82b9-58cdf515bb0b\" (UID: \"4759ebfe-926b-4964-82b9-58cdf515bb0b\") "
Jul 03 14:50:41.936901 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:41.936750    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4759ebfe-926b-4964-82b9-58cdf515bb0b-dshm\") pod \"4759ebfe-926b-4964-82b9-58cdf515bb0b\" (UID: \"4759ebfe-926b-4964-82b9-58cdf515bb0b\") "
Jul 03 14:50:41.936901 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:41.936788    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4759ebfe-926b-4964-82b9-58cdf515bb0b-model-cache\") pod \"4759ebfe-926b-4964-82b9-58cdf515bb0b\" (UID: \"4759ebfe-926b-4964-82b9-58cdf515bb0b\") "
Jul 03 14:50:41.936901 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:41.936820    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4759ebfe-926b-4964-82b9-58cdf515bb0b-tls-certs\") pod \"4759ebfe-926b-4964-82b9-58cdf515bb0b\" (UID: \"4759ebfe-926b-4964-82b9-58cdf515bb0b\") "
Jul 03 14:50:41.936901 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:41.936876    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpmzk\" (UniqueName: \"kubernetes.io/projected/4759ebfe-926b-4964-82b9-58cdf515bb0b-kube-api-access-hpmzk\") pod \"4759ebfe-926b-4964-82b9-58cdf515bb0b\" (UID: \"4759ebfe-926b-4964-82b9-58cdf515bb0b\") "
Jul 03 14:50:41.937175 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:41.936915    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4759ebfe-926b-4964-82b9-58cdf515bb0b-home" (OuterVolumeSpecName: "home") pod "4759ebfe-926b-4964-82b9-58cdf515bb0b" (UID: "4759ebfe-926b-4964-82b9-58cdf515bb0b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:50:41.937175 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:41.936926    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4759ebfe-926b-4964-82b9-58cdf515bb0b-tmp-dir\") pod \"4759ebfe-926b-4964-82b9-58cdf515bb0b\" (UID: \"4759ebfe-926b-4964-82b9-58cdf515bb0b\") "
Jul 03 14:50:41.937175 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:41.937104    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4759ebfe-926b-4964-82b9-58cdf515bb0b-model-cache" (OuterVolumeSpecName: "model-cache") pod "4759ebfe-926b-4964-82b9-58cdf515bb0b" (UID: "4759ebfe-926b-4964-82b9-58cdf515bb0b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:50:41.937175 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:41.937140    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4759ebfe-926b-4964-82b9-58cdf515bb0b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4759ebfe-926b-4964-82b9-58cdf515bb0b" (UID: "4759ebfe-926b-4964-82b9-58cdf515bb0b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:50:41.937382 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:41.937360    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4759ebfe-926b-4964-82b9-58cdf515bb0b-tmp-dir\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:50:41.937432 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:41.937388    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4759ebfe-926b-4964-82b9-58cdf515bb0b-home\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:50:41.937432 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:41.937402    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4759ebfe-926b-4964-82b9-58cdf515bb0b-model-cache\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:50:41.939047 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:41.939027    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4759ebfe-926b-4964-82b9-58cdf515bb0b-dshm" (OuterVolumeSpecName: "dshm") pod "4759ebfe-926b-4964-82b9-58cdf515bb0b" (UID: "4759ebfe-926b-4964-82b9-58cdf515bb0b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:50:41.939114 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:41.939044    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4759ebfe-926b-4964-82b9-58cdf515bb0b-kube-api-access-hpmzk" (OuterVolumeSpecName: "kube-api-access-hpmzk") pod "4759ebfe-926b-4964-82b9-58cdf515bb0b" (UID: "4759ebfe-926b-4964-82b9-58cdf515bb0b"). InnerVolumeSpecName "kube-api-access-hpmzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:50:41.939156 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:41.939133    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4759ebfe-926b-4964-82b9-58cdf515bb0b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4759ebfe-926b-4964-82b9-58cdf515bb0b" (UID: "4759ebfe-926b-4964-82b9-58cdf515bb0b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:50:42.038075 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.038038    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4759ebfe-926b-4964-82b9-58cdf515bb0b-dshm\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:50:42.038075 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.038070    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4759ebfe-926b-4964-82b9-58cdf515bb0b-tls-certs\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:50:42.038075 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.038081    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpmzk\" (UniqueName: \"kubernetes.io/projected/4759ebfe-926b-4964-82b9-58cdf515bb0b-kube-api-access-hpmzk\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:50:42.243247 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.243211    2563 generic.go:356] "Generic (PLEG): container finished" podID="4759ebfe-926b-4964-82b9-58cdf515bb0b" containerID="c139d2f2b787936146f9256bcbb4949e8a321efd7a87ccb6dc45e25070e996f3" exitCode=0
Jul 03 14:50:42.243740 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.243294    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4" event={"ID":"4759ebfe-926b-4964-82b9-58cdf515bb0b","Type":"ContainerDied","Data":"c139d2f2b787936146f9256bcbb4949e8a321efd7a87ccb6dc45e25070e996f3"}
Jul 03 14:50:42.243740 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.243328    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"
Jul 03 14:50:42.243740 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.243348    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4" event={"ID":"4759ebfe-926b-4964-82b9-58cdf515bb0b","Type":"ContainerDied","Data":"cdca3ca23d615bbad425208a8ec045ac1cf5fc73b0926ed4efc9222e0aa8c5ce"}
Jul 03 14:50:42.243740 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.243374    2563 scope.go:117] "RemoveContainer" containerID="c139d2f2b787936146f9256bcbb4949e8a321efd7a87ccb6dc45e25070e996f3"
Jul 03 14:50:42.252791 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.252766    2563 scope.go:117] "RemoveContainer" containerID="c139d2f2b787936146f9256bcbb4949e8a321efd7a87ccb6dc45e25070e996f3"
Jul 03 14:50:42.253210 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:50:42.253185    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c139d2f2b787936146f9256bcbb4949e8a321efd7a87ccb6dc45e25070e996f3\": container with ID starting with c139d2f2b787936146f9256bcbb4949e8a321efd7a87ccb6dc45e25070e996f3 not found: ID does not exist" containerID="c139d2f2b787936146f9256bcbb4949e8a321efd7a87ccb6dc45e25070e996f3"
Jul 03 14:50:42.253340 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.253220    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c139d2f2b787936146f9256bcbb4949e8a321efd7a87ccb6dc45e25070e996f3"} err="failed to get container status \"c139d2f2b787936146f9256bcbb4949e8a321efd7a87ccb6dc45e25070e996f3\": rpc error: code = NotFound desc = could not find container \"c139d2f2b787936146f9256bcbb4949e8a321efd7a87ccb6dc45e25070e996f3\": container with ID starting with c139d2f2b787936146f9256bcbb4949e8a321efd7a87ccb6dc45e25070e996f3 not found: ID does not exist"
Jul 03 14:50:42.269881 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.269843    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"]
Jul 03 14:50:42.274507 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.274477    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5f1e04/rate-test-model-5f1e04-kserve-844c78948c-nz7g4"]
Jul 03 14:50:42.387756 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.387727    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413036496"
Jul 03 14:50:42.410065 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.410033    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7123-139783413036496_47409e59-b67f-4289-b5f0-52fb080eb9b5/test-curl-7123-139783413036496/0.log"
Jul 03 14:50:42.433599 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.433566    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7123-139783413036496"]
Jul 03 14:50:42.436265 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.436233    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7123-139783413036496"]
Jul 03 14:50:42.441324 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.441296    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kxjd\" (UniqueName: \"kubernetes.io/projected/47409e59-b67f-4289-b5f0-52fb080eb9b5-kube-api-access-2kxjd\") pod \"47409e59-b67f-4289-b5f0-52fb080eb9b5\" (UID: \"47409e59-b67f-4289-b5f0-52fb080eb9b5\") "
Jul 03 14:50:42.443325 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.443301    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47409e59-b67f-4289-b5f0-52fb080eb9b5-kube-api-access-2kxjd" (OuterVolumeSpecName: "kube-api-access-2kxjd") pod "47409e59-b67f-4289-b5f0-52fb080eb9b5" (UID: "47409e59-b67f-4289-b5f0-52fb080eb9b5"). InnerVolumeSpecName "kube-api-access-2kxjd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:50:42.543348 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.543254    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kxjd\" (UniqueName: \"kubernetes.io/projected/47409e59-b67f-4289-b5f0-52fb080eb9b5-kube-api-access-2kxjd\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:50:42.545503 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.545456    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q" podUID="60d8d98d-917a-4ede-bf53-36e48321adad" containerName="main" containerID="cri-o://f2f2a3543d0857fd8848dd459f9536bd305744aaca558db35b4075a2500b5aad" gracePeriod=45
Jul 03 14:50:42.560053 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.559936    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7123-139783413038368"]
Jul 03 14:50:42.560406 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.560383    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4759ebfe-926b-4964-82b9-58cdf515bb0b" containerName="main"
Jul 03 14:50:42.560406 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.560406    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4759ebfe-926b-4964-82b9-58cdf515bb0b" containerName="main"
Jul 03 14:50:42.560574 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.560422    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47409e59-b67f-4289-b5f0-52fb080eb9b5" containerName="test-curl-7123-139783413036496"
Jul 03 14:50:42.560574 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.560431    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="47409e59-b67f-4289-b5f0-52fb080eb9b5" containerName="test-curl-7123-139783413036496"
Jul 03 14:50:42.560574 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.560569    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="47409e59-b67f-4289-b5f0-52fb080eb9b5" containerName="test-curl-7123-139783413036496"
Jul 03 14:50:42.560724 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.560591    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4759ebfe-926b-4964-82b9-58cdf515bb0b" containerName="main"
Jul 03 14:50:42.563650 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.563630    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413038368"
Jul 03 14:50:42.570411 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.570389    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7123-139783413038368"]
Jul 03 14:50:42.644465 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.644422    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb8gs\" (UniqueName: \"kubernetes.io/projected/a3d33f4b-bd85-450c-a527-4db656094f47-kube-api-access-bb8gs\") pod \"test-curl-7123-139783413038368\" (UID: \"a3d33f4b-bd85-450c-a527-4db656094f47\") " pod="opendatahub/test-curl-7123-139783413038368"
Jul 03 14:50:42.745860 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.745820    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb8gs\" (UniqueName: \"kubernetes.io/projected/a3d33f4b-bd85-450c-a527-4db656094f47-kube-api-access-bb8gs\") pod \"test-curl-7123-139783413038368\" (UID: \"a3d33f4b-bd85-450c-a527-4db656094f47\") " pod="opendatahub/test-curl-7123-139783413038368"
Jul 03 14:50:42.754099 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.754070    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb8gs\" (UniqueName: \"kubernetes.io/projected/a3d33f4b-bd85-450c-a527-4db656094f47-kube-api-access-bb8gs\") pod \"test-curl-7123-139783413038368\" (UID: \"a3d33f4b-bd85-450c-a527-4db656094f47\") " pod="opendatahub/test-curl-7123-139783413038368"
Jul 03 14:50:42.787816 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.787792    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"
Jul 03 14:50:42.847202 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.847118    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60d8d98d-917a-4ede-bf53-36e48321adad-dshm\") pod \"60d8d98d-917a-4ede-bf53-36e48321adad\" (UID: \"60d8d98d-917a-4ede-bf53-36e48321adad\") "
Jul 03 14:50:42.847202 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.847173    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60d8d98d-917a-4ede-bf53-36e48321adad-model-cache\") pod \"60d8d98d-917a-4ede-bf53-36e48321adad\" (UID: \"60d8d98d-917a-4ede-bf53-36e48321adad\") "
Jul 03 14:50:42.847202 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.847193    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60d8d98d-917a-4ede-bf53-36e48321adad-tls-certs\") pod \"60d8d98d-917a-4ede-bf53-36e48321adad\" (UID: \"60d8d98d-917a-4ede-bf53-36e48321adad\") "
Jul 03 14:50:42.847483 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.847209    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60d8d98d-917a-4ede-bf53-36e48321adad-home\") pod \"60d8d98d-917a-4ede-bf53-36e48321adad\" (UID: \"60d8d98d-917a-4ede-bf53-36e48321adad\") "
Jul 03 14:50:42.847483 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.847262    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mf2vv\" (UniqueName: \"kubernetes.io/projected/60d8d98d-917a-4ede-bf53-36e48321adad-kube-api-access-mf2vv\") pod \"60d8d98d-917a-4ede-bf53-36e48321adad\" (UID: \"60d8d98d-917a-4ede-bf53-36e48321adad\") "
Jul 03 14:50:42.847483 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.847293    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60d8d98d-917a-4ede-bf53-36e48321adad-tmp-dir\") pod \"60d8d98d-917a-4ede-bf53-36e48321adad\" (UID: \"60d8d98d-917a-4ede-bf53-36e48321adad\") "
Jul 03 14:50:42.847483 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.847442    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60d8d98d-917a-4ede-bf53-36e48321adad-model-cache" (OuterVolumeSpecName: "model-cache") pod "60d8d98d-917a-4ede-bf53-36e48321adad" (UID: "60d8d98d-917a-4ede-bf53-36e48321adad"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:50:42.847692 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.847481    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60d8d98d-917a-4ede-bf53-36e48321adad-home" (OuterVolumeSpecName: "home") pod "60d8d98d-917a-4ede-bf53-36e48321adad" (UID: "60d8d98d-917a-4ede-bf53-36e48321adad"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:50:42.847692 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.847617    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60d8d98d-917a-4ede-bf53-36e48321adad-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "60d8d98d-917a-4ede-bf53-36e48321adad" (UID: "60d8d98d-917a-4ede-bf53-36e48321adad"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:50:42.847692 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.847685    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60d8d98d-917a-4ede-bf53-36e48321adad-model-cache\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:50:42.847845 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.847704    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60d8d98d-917a-4ede-bf53-36e48321adad-home\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:50:42.849352 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.849322    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60d8d98d-917a-4ede-bf53-36e48321adad-kube-api-access-mf2vv" (OuterVolumeSpecName: "kube-api-access-mf2vv") pod "60d8d98d-917a-4ede-bf53-36e48321adad" (UID: "60d8d98d-917a-4ede-bf53-36e48321adad"). InnerVolumeSpecName "kube-api-access-mf2vv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:50:42.849758 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.849742    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60d8d98d-917a-4ede-bf53-36e48321adad-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "60d8d98d-917a-4ede-bf53-36e48321adad" (UID: "60d8d98d-917a-4ede-bf53-36e48321adad"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 03 14:50:42.849803 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.849757    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60d8d98d-917a-4ede-bf53-36e48321adad-dshm" (OuterVolumeSpecName: "dshm") pod "60d8d98d-917a-4ede-bf53-36e48321adad" (UID: "60d8d98d-917a-4ede-bf53-36e48321adad"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 03 14:50:42.911582 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.911548    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413038368"
Jul 03 14:50:42.948704 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.948661    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mf2vv\" (UniqueName: \"kubernetes.io/projected/60d8d98d-917a-4ede-bf53-36e48321adad-kube-api-access-mf2vv\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:50:42.948704 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.948704    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60d8d98d-917a-4ede-bf53-36e48321adad-tmp-dir\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:50:42.948904 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.948722    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60d8d98d-917a-4ede-bf53-36e48321adad-dshm\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:50:42.948904 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:42.948735    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60d8d98d-917a-4ede-bf53-36e48321adad-tls-certs\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:50:43.045472 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:43.045445    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7123-139783413038368"]
Jul 03 14:50:43.047225 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:50:43.047198    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda3d33f4b_bd85_450c_a527_4db656094f47.slice/crio-a7a370270907b051583199432c049dfe5e0a69d6fd7a89afd5cb15a1825c4100 WatchSource:0}: Error finding container a7a370270907b051583199432c049dfe5e0a69d6fd7a89afd5cb15a1825c4100: Status 404 returned error can't find the container with id a7a370270907b051583199432c049dfe5e0a69d6fd7a89afd5cb15a1825c4100
Jul 03 14:50:43.060200 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:43.060173    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47409e59-b67f-4289-b5f0-52fb080eb9b5" path="/var/lib/kubelet/pods/47409e59-b67f-4289-b5f0-52fb080eb9b5/volumes"
Jul 03 14:50:43.060607 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:43.060594    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4759ebfe-926b-4964-82b9-58cdf515bb0b" path="/var/lib/kubelet/pods/4759ebfe-926b-4964-82b9-58cdf515bb0b/volumes"
Jul 03 14:50:43.253345 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:43.253310    2563 generic.go:356] "Generic (PLEG): container finished" podID="60d8d98d-917a-4ede-bf53-36e48321adad" containerID="f2f2a3543d0857fd8848dd459f9536bd305744aaca558db35b4075a2500b5aad" exitCode=0
Jul 03 14:50:43.253825 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:43.253366    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"
Jul 03 14:50:43.253825 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:43.253399    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q" event={"ID":"60d8d98d-917a-4ede-bf53-36e48321adad","Type":"ContainerDied","Data":"f2f2a3543d0857fd8848dd459f9536bd305744aaca558db35b4075a2500b5aad"}
Jul 03 14:50:43.253825 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:43.253456    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q" event={"ID":"60d8d98d-917a-4ede-bf53-36e48321adad","Type":"ContainerDied","Data":"223691e59e98eba0c4440b1e1de0116ce61de3168a998f25db0fa98ef5211133"}
Jul 03 14:50:43.253825 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:43.253484    2563 scope.go:117] "RemoveContainer" containerID="f2f2a3543d0857fd8848dd459f9536bd305744aaca558db35b4075a2500b5aad"
Jul 03 14:50:43.254937 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:43.254911    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413036496"
Jul 03 14:50:43.256598 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:43.256570    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7123-139783413038368" event={"ID":"a3d33f4b-bd85-450c-a527-4db656094f47","Type":"ContainerStarted","Data":"a7a370270907b051583199432c049dfe5e0a69d6fd7a89afd5cb15a1825c4100"}
Jul 03 14:50:43.264634 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:43.264616    2563 scope.go:117] "RemoveContainer" containerID="f2f2a3543d0857fd8848dd459f9536bd305744aaca558db35b4075a2500b5aad"
Jul 03 14:50:43.264937 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:50:43.264917    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2f2a3543d0857fd8848dd459f9536bd305744aaca558db35b4075a2500b5aad\": container with ID starting with f2f2a3543d0857fd8848dd459f9536bd305744aaca558db35b4075a2500b5aad not found: ID does not exist" containerID="f2f2a3543d0857fd8848dd459f9536bd305744aaca558db35b4075a2500b5aad"
Jul 03 14:50:43.265012 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:43.264945    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2f2a3543d0857fd8848dd459f9536bd305744aaca558db35b4075a2500b5aad"} err="failed to get container status \"f2f2a3543d0857fd8848dd459f9536bd305744aaca558db35b4075a2500b5aad\": rpc error: code = NotFound desc = could not find container \"f2f2a3543d0857fd8848dd459f9536bd305744aaca558db35b4075a2500b5aad\": container with ID starting with f2f2a3543d0857fd8848dd459f9536bd305744aaca558db35b4075a2500b5aad not found: ID does not exist"
Jul 03 14:50:43.265012 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:43.264962    2563 scope.go:117] "RemoveContainer" containerID="3fea55b072193abb9963e75f34879faedba06fc140d799e7314893edbbcf87bd"
Jul 03 14:50:43.271478 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:43.271454    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"]
Jul 03 14:50:43.276236 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:43.276211    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-28686a/rate-test-model-28686a-kserve-9ccdfd6bc-qdx5q"]
Jul 03 14:50:44.266417 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:44.266383    2563 generic.go:356] "Generic (PLEG): container finished" podID="a3d33f4b-bd85-450c-a527-4db656094f47" containerID="860d38f1037126475ea9fb61a55ca126b3bea602d7a68cda5b75bbe5e8aabf6d" exitCode=0
Jul 03 14:50:44.266825 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:44.266477    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7123-139783413038368" event={"ID":"a3d33f4b-bd85-450c-a527-4db656094f47","Type":"ContainerDied","Data":"860d38f1037126475ea9fb61a55ca126b3bea602d7a68cda5b75bbe5e8aabf6d"}
Jul 03 14:50:45.059494 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:45.059464    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60d8d98d-917a-4ede-bf53-36e48321adad" path="/var/lib/kubelet/pods/60d8d98d-917a-4ede-bf53-36e48321adad/volumes"
Jul 03 14:50:45.410804 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:45.410779    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413038368"
Jul 03 14:50:45.431670 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:45.431644    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7123-139783413038368_a3d33f4b-bd85-450c-a527-4db656094f47/test-curl-7123-139783413038368/0.log"
Jul 03 14:50:45.457213 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:45.457179    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7123-139783413038368"]
Jul 03 14:50:45.461452 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:45.461428    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7123-139783413038368"]
Jul 03 14:50:45.471761 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:45.471741    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bb8gs\" (UniqueName: \"kubernetes.io/projected/a3d33f4b-bd85-450c-a527-4db656094f47-kube-api-access-bb8gs\") pod \"a3d33f4b-bd85-450c-a527-4db656094f47\" (UID: \"a3d33f4b-bd85-450c-a527-4db656094f47\") "
Jul 03 14:50:45.473718 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:45.473696    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3d33f4b-bd85-450c-a527-4db656094f47-kube-api-access-bb8gs" (OuterVolumeSpecName: "kube-api-access-bb8gs") pod "a3d33f4b-bd85-450c-a527-4db656094f47" (UID: "a3d33f4b-bd85-450c-a527-4db656094f47"). InnerVolumeSpecName "kube-api-access-bb8gs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:50:45.573337 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:45.573250    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bb8gs\" (UniqueName: \"kubernetes.io/projected/a3d33f4b-bd85-450c-a527-4db656094f47-kube-api-access-bb8gs\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:50:45.586428 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:45.586397    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7123-139783413037072"]
Jul 03 14:50:45.586790 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:45.586777    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3d33f4b-bd85-450c-a527-4db656094f47" containerName="test-curl-7123-139783413038368"
Jul 03 14:50:45.586854 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:45.586791    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3d33f4b-bd85-450c-a527-4db656094f47" containerName="test-curl-7123-139783413038368"
Jul 03 14:50:45.586854 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:45.586801    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60d8d98d-917a-4ede-bf53-36e48321adad" containerName="main"
Jul 03 14:50:45.586854 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:45.586808    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="60d8d98d-917a-4ede-bf53-36e48321adad" containerName="main"
Jul 03 14:50:45.586961 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:45.586881    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3d33f4b-bd85-450c-a527-4db656094f47" containerName="test-curl-7123-139783413038368"
Jul 03 14:50:45.586961 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:45.586889    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="60d8d98d-917a-4ede-bf53-36e48321adad" containerName="main"
Jul 03 14:50:45.589855 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:45.589839    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413037072"
Jul 03 14:50:45.596549 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:45.596528    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7123-139783413037072"]
Jul 03 14:50:45.674760 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:45.674727    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k24r\" (UniqueName: \"kubernetes.io/projected/52b7a4aa-bd01-47e5-a971-0404029b17ec-kube-api-access-6k24r\") pod \"test-curl-7123-139783413037072\" (UID: \"52b7a4aa-bd01-47e5-a971-0404029b17ec\") " pod="opendatahub/test-curl-7123-139783413037072"
Jul 03 14:50:45.776067 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:45.776016    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k24r\" (UniqueName: \"kubernetes.io/projected/52b7a4aa-bd01-47e5-a971-0404029b17ec-kube-api-access-6k24r\") pod \"test-curl-7123-139783413037072\" (UID: \"52b7a4aa-bd01-47e5-a971-0404029b17ec\") " pod="opendatahub/test-curl-7123-139783413037072"
Jul 03 14:50:45.784682 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:45.784653    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k24r\" (UniqueName: \"kubernetes.io/projected/52b7a4aa-bd01-47e5-a971-0404029b17ec-kube-api-access-6k24r\") pod \"test-curl-7123-139783413037072\" (UID: \"52b7a4aa-bd01-47e5-a971-0404029b17ec\") " pod="opendatahub/test-curl-7123-139783413037072"
Jul 03 14:50:45.899914 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:45.899876    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413037072"
Jul 03 14:50:46.031341 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:46.031312    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7123-139783413037072"]
Jul 03 14:50:46.032987 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:50:46.032942    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52b7a4aa_bd01_47e5_a971_0404029b17ec.slice/crio-eb7d7b502e29ad8a19e3e9f0de0e227033a5e7314f1b197a75234442a473eca6 WatchSource:0}: Error finding container eb7d7b502e29ad8a19e3e9f0de0e227033a5e7314f1b197a75234442a473eca6: Status 404 returned error can't find the container with id eb7d7b502e29ad8a19e3e9f0de0e227033a5e7314f1b197a75234442a473eca6
Jul 03 14:50:46.277962 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:46.277875    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413038368"
Jul 03 14:50:46.277962 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:46.277912    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a7a370270907b051583199432c049dfe5e0a69d6fd7a89afd5cb15a1825c4100"
Jul 03 14:50:46.279154 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:46.279129    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7123-139783413037072" event={"ID":"52b7a4aa-bd01-47e5-a971-0404029b17ec","Type":"ContainerStarted","Data":"eb7d7b502e29ad8a19e3e9f0de0e227033a5e7314f1b197a75234442a473eca6"}
Jul 03 14:50:46.288184 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:50:46.288156    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7123-139783413038368\" is forbidden: User \"system:node:ip-10-0-142-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-233.ec2.internal' and this object" podUID="a3d33f4b-bd85-450c-a527-4db656094f47" pod="opendatahub/test-curl-7123-139783413038368"
Jul 03 14:50:47.060745 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:50:47.060703    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7123-139783413038368\" is forbidden: User \"system:node:ip-10-0-142-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-233.ec2.internal' and this object" podUID="a3d33f4b-bd85-450c-a527-4db656094f47" pod="opendatahub/test-curl-7123-139783413038368"
Jul 03 14:50:47.065208 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:47.065173    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3d33f4b-bd85-450c-a527-4db656094f47" path="/var/lib/kubelet/pods/a3d33f4b-bd85-450c-a527-4db656094f47/volumes"
Jul 03 14:50:47.285374 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:47.285336    2563 generic.go:356] "Generic (PLEG): container finished" podID="52b7a4aa-bd01-47e5-a971-0404029b17ec" containerID="e566ba42f7bcec57787a0bbac7c0bc323847986dd4df742df921323efeb0ce9e" exitCode=0
Jul 03 14:50:47.285528 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:47.285429    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7123-139783413037072" event={"ID":"52b7a4aa-bd01-47e5-a971-0404029b17ec","Type":"ContainerDied","Data":"e566ba42f7bcec57787a0bbac7c0bc323847986dd4df742df921323efeb0ce9e"}
Jul 03 14:50:48.431960 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:48.431928    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413037072"
Jul 03 14:50:48.451674 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:48.451649    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7123-139783413037072_52b7a4aa-bd01-47e5-a971-0404029b17ec/test-curl-7123-139783413037072/0.log"
Jul 03 14:50:48.477366 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:48.477337    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7123-139783413037072"]
Jul 03 14:50:48.479767 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:48.479741    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7123-139783413037072"]
Jul 03 14:50:48.504476 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:48.504453    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6k24r\" (UniqueName: \"kubernetes.io/projected/52b7a4aa-bd01-47e5-a971-0404029b17ec-kube-api-access-6k24r\") pod \"52b7a4aa-bd01-47e5-a971-0404029b17ec\" (UID: \"52b7a4aa-bd01-47e5-a971-0404029b17ec\") "
Jul 03 14:50:48.506481 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:48.506456    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52b7a4aa-bd01-47e5-a971-0404029b17ec-kube-api-access-6k24r" (OuterVolumeSpecName: "kube-api-access-6k24r") pod "52b7a4aa-bd01-47e5-a971-0404029b17ec" (UID: "52b7a4aa-bd01-47e5-a971-0404029b17ec"). InnerVolumeSpecName "kube-api-access-6k24r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:50:48.597989 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:48.597887    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7123-139783413038656"]
Jul 03 14:50:48.598297 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:48.598284    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52b7a4aa-bd01-47e5-a971-0404029b17ec" containerName="test-curl-7123-139783413037072"
Jul 03 14:50:48.598348 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:48.598299    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="52b7a4aa-bd01-47e5-a971-0404029b17ec" containerName="test-curl-7123-139783413037072"
Jul 03 14:50:48.598391 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:48.598376    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="52b7a4aa-bd01-47e5-a971-0404029b17ec" containerName="test-curl-7123-139783413037072"
Jul 03 14:50:48.601645 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:48.601628    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413038656"
Jul 03 14:50:48.605806 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:48.605779    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6k24r\" (UniqueName: \"kubernetes.io/projected/52b7a4aa-bd01-47e5-a971-0404029b17ec-kube-api-access-6k24r\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:50:48.608196 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:48.608175    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7123-139783413038656"]
Jul 03 14:50:48.706811 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:48.706770    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r28bm\" (UniqueName: \"kubernetes.io/projected/fbd9c5e6-f4c2-42ae-b484-751016680e1a-kube-api-access-r28bm\") pod \"test-curl-7123-139783413038656\" (UID: \"fbd9c5e6-f4c2-42ae-b484-751016680e1a\") " pod="opendatahub/test-curl-7123-139783413038656"
Jul 03 14:50:48.808140 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:48.808094    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r28bm\" (UniqueName: \"kubernetes.io/projected/fbd9c5e6-f4c2-42ae-b484-751016680e1a-kube-api-access-r28bm\") pod \"test-curl-7123-139783413038656\" (UID: \"fbd9c5e6-f4c2-42ae-b484-751016680e1a\") " pod="opendatahub/test-curl-7123-139783413038656"
Jul 03 14:50:48.820211 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:48.820179    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r28bm\" (UniqueName: \"kubernetes.io/projected/fbd9c5e6-f4c2-42ae-b484-751016680e1a-kube-api-access-r28bm\") pod \"test-curl-7123-139783413038656\" (UID: \"fbd9c5e6-f4c2-42ae-b484-751016680e1a\") " pod="opendatahub/test-curl-7123-139783413038656"
Jul 03 14:50:48.912689 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:48.912656    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413038656"
Jul 03 14:50:49.043650 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:49.043605    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7123-139783413038656"]
Jul 03 14:50:49.046574 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:50:49.046549    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfbd9c5e6_f4c2_42ae_b484_751016680e1a.slice/crio-8b1d140bcb897e9d505ca7406f3486ad4fab6a003c83eefe7ce6dcb34f7e204f WatchSource:0}: Error finding container 8b1d140bcb897e9d505ca7406f3486ad4fab6a003c83eefe7ce6dcb34f7e204f: Status 404 returned error can't find the container with id 8b1d140bcb897e9d505ca7406f3486ad4fab6a003c83eefe7ce6dcb34f7e204f
Jul 03 14:50:49.058522 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:49.058492    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52b7a4aa-bd01-47e5-a971-0404029b17ec" path="/var/lib/kubelet/pods/52b7a4aa-bd01-47e5-a971-0404029b17ec/volumes"
Jul 03 14:50:49.301094 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:49.301003    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7123-139783413038656" event={"ID":"fbd9c5e6-f4c2-42ae-b484-751016680e1a","Type":"ContainerStarted","Data":"8b1d140bcb897e9d505ca7406f3486ad4fab6a003c83eefe7ce6dcb34f7e204f"}
Jul 03 14:50:49.302169 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:49.302143    2563 scope.go:117] "RemoveContainer" containerID="e566ba42f7bcec57787a0bbac7c0bc323847986dd4df742df921323efeb0ce9e"
Jul 03 14:50:49.302169 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:49.302160    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413037072"
Jul 03 14:50:50.310841 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:50.310809    2563 generic.go:356] "Generic (PLEG): container finished" podID="fbd9c5e6-f4c2-42ae-b484-751016680e1a" containerID="84c565a149db406015e6127ed7348ca3067e6c48e14ebf73dda5adbac87b3f15" exitCode=0
Jul 03 14:50:50.311284 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:50.310858    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7123-139783413038656" event={"ID":"fbd9c5e6-f4c2-42ae-b484-751016680e1a","Type":"ContainerDied","Data":"84c565a149db406015e6127ed7348ca3067e6c48e14ebf73dda5adbac87b3f15"}
Jul 03 14:50:51.455637 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:51.455604    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413038656"
Jul 03 14:50:51.476383 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:51.476352    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7123-139783413038656_fbd9c5e6-f4c2-42ae-b484-751016680e1a/test-curl-7123-139783413038656/0.log"
Jul 03 14:50:51.499747 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:51.499717    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7123-139783413038656"]
Jul 03 14:50:51.503329 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:51.503304    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7123-139783413038656"]
Jul 03 14:50:51.536792 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:51.536763    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r28bm\" (UniqueName: \"kubernetes.io/projected/fbd9c5e6-f4c2-42ae-b484-751016680e1a-kube-api-access-r28bm\") pod \"fbd9c5e6-f4c2-42ae-b484-751016680e1a\" (UID: \"fbd9c5e6-f4c2-42ae-b484-751016680e1a\") "
Jul 03 14:50:51.538871 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:51.538841    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbd9c5e6-f4c2-42ae-b484-751016680e1a-kube-api-access-r28bm" (OuterVolumeSpecName: "kube-api-access-r28bm") pod "fbd9c5e6-f4c2-42ae-b484-751016680e1a" (UID: "fbd9c5e6-f4c2-42ae-b484-751016680e1a"). InnerVolumeSpecName "kube-api-access-r28bm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:50:51.630769 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:51.630736    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7123-139783413037360"]
Jul 03 14:50:51.631138 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:51.631125    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbd9c5e6-f4c2-42ae-b484-751016680e1a" containerName="test-curl-7123-139783413038656"
Jul 03 14:50:51.631199 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:51.631141    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbd9c5e6-f4c2-42ae-b484-751016680e1a" containerName="test-curl-7123-139783413038656"
Jul 03 14:50:51.631237 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:51.631217    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbd9c5e6-f4c2-42ae-b484-751016680e1a" containerName="test-curl-7123-139783413038656"
Jul 03 14:50:51.634239 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:51.634224    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413037360"
Jul 03 14:50:51.637504 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:51.637481    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r28bm\" (UniqueName: \"kubernetes.io/projected/fbd9c5e6-f4c2-42ae-b484-751016680e1a-kube-api-access-r28bm\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:50:51.640069 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:51.640038    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7123-139783413037360"]
Jul 03 14:50:51.738116 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:51.738086    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qddlc\" (UniqueName: \"kubernetes.io/projected/01065928-9b0d-417e-ba09-b3a4a53c79da-kube-api-access-qddlc\") pod \"test-curl-7123-139783413037360\" (UID: \"01065928-9b0d-417e-ba09-b3a4a53c79da\") " pod="opendatahub/test-curl-7123-139783413037360"
Jul 03 14:50:51.838939 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:51.838898    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qddlc\" (UniqueName: \"kubernetes.io/projected/01065928-9b0d-417e-ba09-b3a4a53c79da-kube-api-access-qddlc\") pod \"test-curl-7123-139783413037360\" (UID: \"01065928-9b0d-417e-ba09-b3a4a53c79da\") " pod="opendatahub/test-curl-7123-139783413037360"
Jul 03 14:50:51.852144 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:51.852111    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qddlc\" (UniqueName: \"kubernetes.io/projected/01065928-9b0d-417e-ba09-b3a4a53c79da-kube-api-access-qddlc\") pod \"test-curl-7123-139783413037360\" (UID: \"01065928-9b0d-417e-ba09-b3a4a53c79da\") " pod="opendatahub/test-curl-7123-139783413037360"
Jul 03 14:50:51.944927 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:51.944836    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413037360"
Jul 03 14:50:52.078451 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:52.078416    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7123-139783413037360"]
Jul 03 14:50:52.081407 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:50:52.081378    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01065928_9b0d_417e_ba09_b3a4a53c79da.slice/crio-5c808a95e70bf52fd12681fa8b95e7cb73e7934318123f7f7b33e75c52b0e482 WatchSource:0}: Error finding container 5c808a95e70bf52fd12681fa8b95e7cb73e7934318123f7f7b33e75c52b0e482: Status 404 returned error can't find the container with id 5c808a95e70bf52fd12681fa8b95e7cb73e7934318123f7f7b33e75c52b0e482
Jul 03 14:50:52.324399 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:52.324316    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7123-139783413037360" event={"ID":"01065928-9b0d-417e-ba09-b3a4a53c79da","Type":"ContainerStarted","Data":"5c808a95e70bf52fd12681fa8b95e7cb73e7934318123f7f7b33e75c52b0e482"}
Jul 03 14:50:52.325635 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:52.325613    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8b1d140bcb897e9d505ca7406f3486ad4fab6a003c83eefe7ce6dcb34f7e204f"
Jul 03 14:50:52.325754 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:52.325643    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413038656"
Jul 03 14:50:52.337248 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:50:52.337222    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7123-139783413038656\" is forbidden: User \"system:node:ip-10-0-142-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-233.ec2.internal' and this object" podUID="fbd9c5e6-f4c2-42ae-b484-751016680e1a" pod="opendatahub/test-curl-7123-139783413038656"
Jul 03 14:50:53.058617 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:53.058581    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbd9c5e6-f4c2-42ae-b484-751016680e1a" path="/var/lib/kubelet/pods/fbd9c5e6-f4c2-42ae-b484-751016680e1a/volumes"
Jul 03 14:50:53.332542 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:53.332451    2563 generic.go:356] "Generic (PLEG): container finished" podID="01065928-9b0d-417e-ba09-b3a4a53c79da" containerID="8f48b4b95ecb213a91f3cec7c08ce293bd074085ee81be2b0b63b8646f98018b" exitCode=0
Jul 03 14:50:53.332691 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:53.332543    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7123-139783413037360" event={"ID":"01065928-9b0d-417e-ba09-b3a4a53c79da","Type":"ContainerDied","Data":"8f48b4b95ecb213a91f3cec7c08ce293bd074085ee81be2b0b63b8646f98018b"}
Jul 03 14:50:54.480565 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:54.480541    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413037360"
Jul 03 14:50:54.500408 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:54.500382    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7123-139783413037360_01065928-9b0d-417e-ba09-b3a4a53c79da/test-curl-7123-139783413037360/0.log"
Jul 03 14:50:54.521651 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:54.521621    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7123-139783413037360"]
Jul 03 14:50:54.526410 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:54.526377    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7123-139783413037360"]
Jul 03 14:50:54.565798 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:54.565770    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qddlc\" (UniqueName: \"kubernetes.io/projected/01065928-9b0d-417e-ba09-b3a4a53c79da-kube-api-access-qddlc\") pod \"01065928-9b0d-417e-ba09-b3a4a53c79da\" (UID: \"01065928-9b0d-417e-ba09-b3a4a53c79da\") "
Jul 03 14:50:54.567816 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:54.567790    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01065928-9b0d-417e-ba09-b3a4a53c79da-kube-api-access-qddlc" (OuterVolumeSpecName: "kube-api-access-qddlc") pod "01065928-9b0d-417e-ba09-b3a4a53c79da" (UID: "01065928-9b0d-417e-ba09-b3a4a53c79da"). InnerVolumeSpecName "kube-api-access-qddlc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:50:54.643227 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:54.643192    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7123-139783413037360"]
Jul 03 14:50:54.643584 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:54.643571    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01065928-9b0d-417e-ba09-b3a4a53c79da" containerName="test-curl-7123-139783413037360"
Jul 03 14:50:54.643624 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:54.643585    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="01065928-9b0d-417e-ba09-b3a4a53c79da" containerName="test-curl-7123-139783413037360"
Jul 03 14:50:54.643670 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:54.643661    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="01065928-9b0d-417e-ba09-b3a4a53c79da" containerName="test-curl-7123-139783413037360"
Jul 03 14:50:54.646646 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:54.646630    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413037360"
Jul 03 14:50:54.651901 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:54.651877    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7123-139783413037360"]
Jul 03 14:50:54.667578 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:54.667549    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qddlc\" (UniqueName: \"kubernetes.io/projected/01065928-9b0d-417e-ba09-b3a4a53c79da-kube-api-access-qddlc\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:50:54.768821 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:54.768784    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhd6d\" (UniqueName: \"kubernetes.io/projected/d9a6b21d-bda4-4d2e-90c9-e09d54f586dc-kube-api-access-fhd6d\") pod \"test-curl-7123-139783413037360\" (UID: \"d9a6b21d-bda4-4d2e-90c9-e09d54f586dc\") " pod="opendatahub/test-curl-7123-139783413037360"
Jul 03 14:50:54.869801 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:54.869760    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhd6d\" (UniqueName: \"kubernetes.io/projected/d9a6b21d-bda4-4d2e-90c9-e09d54f586dc-kube-api-access-fhd6d\") pod \"test-curl-7123-139783413037360\" (UID: \"d9a6b21d-bda4-4d2e-90c9-e09d54f586dc\") " pod="opendatahub/test-curl-7123-139783413037360"
Jul 03 14:50:54.877946 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:54.877924    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhd6d\" (UniqueName: \"kubernetes.io/projected/d9a6b21d-bda4-4d2e-90c9-e09d54f586dc-kube-api-access-fhd6d\") pod \"test-curl-7123-139783413037360\" (UID: \"d9a6b21d-bda4-4d2e-90c9-e09d54f586dc\") " pod="opendatahub/test-curl-7123-139783413037360"
Jul 03 14:50:54.956771 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:54.956655    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413037360"
Jul 03 14:50:55.062595 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:55.062558    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01065928-9b0d-417e-ba09-b3a4a53c79da" path="/var/lib/kubelet/pods/01065928-9b0d-417e-ba09-b3a4a53c79da/volumes"
Jul 03 14:50:55.093173 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:55.093130    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7123-139783413037360"]
Jul 03 14:50:55.094574 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:50:55.094542    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9a6b21d_bda4_4d2e_90c9_e09d54f586dc.slice/crio-cdbab64cfd3131553b647c1a6730a4af091943b9bd735571831549674d12cf62 WatchSource:0}: Error finding container cdbab64cfd3131553b647c1a6730a4af091943b9bd735571831549674d12cf62: Status 404 returned error can't find the container with id cdbab64cfd3131553b647c1a6730a4af091943b9bd735571831549674d12cf62
Jul 03 14:50:55.109373 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:50:55.109336    2563 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01065928_9b0d_417e_ba09_b3a4a53c79da.slice\": RecentStats: unable to find data in memory cache]"
Jul 03 14:50:55.344074 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:55.343990    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413037360"
Jul 03 14:50:55.344074 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:55.343990    2563 scope.go:117] "RemoveContainer" containerID="8f48b4b95ecb213a91f3cec7c08ce293bd074085ee81be2b0b63b8646f98018b"
Jul 03 14:50:55.345463 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:55.345193    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7123-139783413037360" event={"ID":"d9a6b21d-bda4-4d2e-90c9-e09d54f586dc","Type":"ContainerStarted","Data":"cdbab64cfd3131553b647c1a6730a4af091943b9bd735571831549674d12cf62"}
Jul 03 14:50:56.358153 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:56.358120    2563 generic.go:356] "Generic (PLEG): container finished" podID="d9a6b21d-bda4-4d2e-90c9-e09d54f586dc" containerID="158e811109f1f42e999a5a3dc908941ad1d82283906a5b4839aa3a87cdf2a84b" exitCode=0
Jul 03 14:50:56.358571 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:56.358207    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7123-139783413037360" event={"ID":"d9a6b21d-bda4-4d2e-90c9-e09d54f586dc","Type":"ContainerDied","Data":"158e811109f1f42e999a5a3dc908941ad1d82283906a5b4839aa3a87cdf2a84b"}
Jul 03 14:50:57.492694 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:57.492669    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413037360"
Jul 03 14:50:57.514092 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:57.514068    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7123-139783413037360_d9a6b21d-bda4-4d2e-90c9-e09d54f586dc/test-curl-7123-139783413037360/0.log"
Jul 03 14:50:57.538115 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:57.538083    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7123-139783413037360"]
Jul 03 14:50:57.541473 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:57.541449    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7123-139783413037360"]
Jul 03 14:50:57.595738 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:57.595701    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhd6d\" (UniqueName: \"kubernetes.io/projected/d9a6b21d-bda4-4d2e-90c9-e09d54f586dc-kube-api-access-fhd6d\") pod \"d9a6b21d-bda4-4d2e-90c9-e09d54f586dc\" (UID: \"d9a6b21d-bda4-4d2e-90c9-e09d54f586dc\") "
Jul 03 14:50:57.597753 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:57.597723    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9a6b21d-bda4-4d2e-90c9-e09d54f586dc-kube-api-access-fhd6d" (OuterVolumeSpecName: "kube-api-access-fhd6d") pod "d9a6b21d-bda4-4d2e-90c9-e09d54f586dc" (UID: "d9a6b21d-bda4-4d2e-90c9-e09d54f586dc"). InnerVolumeSpecName "kube-api-access-fhd6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:50:57.662061 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:57.661980    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7123-139783413038800"]
Jul 03 14:50:57.662378 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:57.662365    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9a6b21d-bda4-4d2e-90c9-e09d54f586dc" containerName="test-curl-7123-139783413037360"
Jul 03 14:50:57.662416 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:57.662379    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9a6b21d-bda4-4d2e-90c9-e09d54f586dc" containerName="test-curl-7123-139783413037360"
Jul 03 14:50:57.662448 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:57.662441    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9a6b21d-bda4-4d2e-90c9-e09d54f586dc" containerName="test-curl-7123-139783413037360"
Jul 03 14:50:57.665482 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:57.665464    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413038800"
Jul 03 14:50:57.672589 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:57.672423    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7123-139783413038800"]
Jul 03 14:50:57.696691 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:57.696663    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhd6d\" (UniqueName: \"kubernetes.io/projected/d9a6b21d-bda4-4d2e-90c9-e09d54f586dc-kube-api-access-fhd6d\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:50:57.797320 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:57.797285    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8hcj\" (UniqueName: \"kubernetes.io/projected/2d833b21-d911-4fe1-96c0-d72ed4387b87-kube-api-access-z8hcj\") pod \"test-curl-7123-139783413038800\" (UID: \"2d833b21-d911-4fe1-96c0-d72ed4387b87\") " pod="opendatahub/test-curl-7123-139783413038800"
Jul 03 14:50:57.898709 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:57.898670    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8hcj\" (UniqueName: \"kubernetes.io/projected/2d833b21-d911-4fe1-96c0-d72ed4387b87-kube-api-access-z8hcj\") pod \"test-curl-7123-139783413038800\" (UID: \"2d833b21-d911-4fe1-96c0-d72ed4387b87\") " pod="opendatahub/test-curl-7123-139783413038800"
Jul 03 14:50:57.907205 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:57.907179    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8hcj\" (UniqueName: \"kubernetes.io/projected/2d833b21-d911-4fe1-96c0-d72ed4387b87-kube-api-access-z8hcj\") pod \"test-curl-7123-139783413038800\" (UID: \"2d833b21-d911-4fe1-96c0-d72ed4387b87\") " pod="opendatahub/test-curl-7123-139783413038800"
Jul 03 14:50:57.975389 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:57.975300    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413038800"
Jul 03 14:50:58.105018 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:58.104990    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7123-139783413038800"]
Jul 03 14:50:58.107481 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:50:58.107453    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d833b21_d911_4fe1_96c0_d72ed4387b87.slice/crio-674dd6bef623fe027cb058b56671565dffcbb0b797a813191274bad22f069c36 WatchSource:0}: Error finding container 674dd6bef623fe027cb058b56671565dffcbb0b797a813191274bad22f069c36: Status 404 returned error can't find the container with id 674dd6bef623fe027cb058b56671565dffcbb0b797a813191274bad22f069c36
Jul 03 14:50:58.370027 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:58.369996    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413037360"
Jul 03 14:50:58.370027 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:58.370020    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cdbab64cfd3131553b647c1a6730a4af091943b9bd735571831549674d12cf62"
Jul 03 14:50:58.371211 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:58.371187    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7123-139783413038800" event={"ID":"2d833b21-d911-4fe1-96c0-d72ed4387b87","Type":"ContainerStarted","Data":"674dd6bef623fe027cb058b56671565dffcbb0b797a813191274bad22f069c36"}
Jul 03 14:50:58.380821 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:50:58.380789    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7123-139783413037360\" is forbidden: User \"system:node:ip-10-0-142-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-233.ec2.internal' and this object" podUID="d9a6b21d-bda4-4d2e-90c9-e09d54f586dc" pod="opendatahub/test-curl-7123-139783413037360"
Jul 03 14:50:59.058490 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:59.058447    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9a6b21d-bda4-4d2e-90c9-e09d54f586dc" path="/var/lib/kubelet/pods/d9a6b21d-bda4-4d2e-90c9-e09d54f586dc/volumes"
Jul 03 14:50:59.378152 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:59.378120    2563 generic.go:356] "Generic (PLEG): container finished" podID="2d833b21-d911-4fe1-96c0-d72ed4387b87" containerID="0553e5dbd7a974e20691f20f7e5ebde9c860e05b0b437073abf34ff12d7b1e9f" exitCode=0
Jul 03 14:50:59.378342 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:50:59.378206    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7123-139783413038800" event={"ID":"2d833b21-d911-4fe1-96c0-d72ed4387b87","Type":"ContainerDied","Data":"0553e5dbd7a974e20691f20f7e5ebde9c860e05b0b437073abf34ff12d7b1e9f"}
Jul 03 14:51:00.518674 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:00.518649    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413038800"
Jul 03 14:51:00.538356 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:00.538326    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7123-139783413038800_2d833b21-d911-4fe1-96c0-d72ed4387b87/test-curl-7123-139783413038800/0.log"
Jul 03 14:51:00.560519 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:00.560485    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7123-139783413038800"]
Jul 03 14:51:00.563882 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:00.563859    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7123-139783413038800"]
Jul 03 14:51:00.627193 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:00.627161    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8hcj\" (UniqueName: \"kubernetes.io/projected/2d833b21-d911-4fe1-96c0-d72ed4387b87-kube-api-access-z8hcj\") pod \"2d833b21-d911-4fe1-96c0-d72ed4387b87\" (UID: \"2d833b21-d911-4fe1-96c0-d72ed4387b87\") "
Jul 03 14:51:00.629285 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:00.629252    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d833b21-d911-4fe1-96c0-d72ed4387b87-kube-api-access-z8hcj" (OuterVolumeSpecName: "kube-api-access-z8hcj") pod "2d833b21-d911-4fe1-96c0-d72ed4387b87" (UID: "2d833b21-d911-4fe1-96c0-d72ed4387b87"). InnerVolumeSpecName "kube-api-access-z8hcj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:51:00.685927 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:00.685891    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7123-139783413038800"]
Jul 03 14:51:00.686313 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:00.686300    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d833b21-d911-4fe1-96c0-d72ed4387b87" containerName="test-curl-7123-139783413038800"
Jul 03 14:51:00.686353 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:00.686316    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d833b21-d911-4fe1-96c0-d72ed4387b87" containerName="test-curl-7123-139783413038800"
Jul 03 14:51:00.686387 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:00.686376    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d833b21-d911-4fe1-96c0-d72ed4387b87" containerName="test-curl-7123-139783413038800"
Jul 03 14:51:00.689370 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:00.689354    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413038800"
Jul 03 14:51:00.693116 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:00.693093    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7123-139783413038800"]
Jul 03 14:51:00.728691 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:00.728656    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8hcj\" (UniqueName: \"kubernetes.io/projected/2d833b21-d911-4fe1-96c0-d72ed4387b87-kube-api-access-z8hcj\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:51:00.829758 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:00.829719    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv6gz\" (UniqueName: \"kubernetes.io/projected/671c602a-3ff7-45e9-b765-927a8ecbbebb-kube-api-access-zv6gz\") pod \"test-curl-7123-139783413038800\" (UID: \"671c602a-3ff7-45e9-b765-927a8ecbbebb\") " pod="opendatahub/test-curl-7123-139783413038800"
Jul 03 14:51:00.931254 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:00.931154    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv6gz\" (UniqueName: \"kubernetes.io/projected/671c602a-3ff7-45e9-b765-927a8ecbbebb-kube-api-access-zv6gz\") pod \"test-curl-7123-139783413038800\" (UID: \"671c602a-3ff7-45e9-b765-927a8ecbbebb\") " pod="opendatahub/test-curl-7123-139783413038800"
Jul 03 14:51:00.939539 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:00.939506    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv6gz\" (UniqueName: \"kubernetes.io/projected/671c602a-3ff7-45e9-b765-927a8ecbbebb-kube-api-access-zv6gz\") pod \"test-curl-7123-139783413038800\" (UID: \"671c602a-3ff7-45e9-b765-927a8ecbbebb\") " pod="opendatahub/test-curl-7123-139783413038800"
Jul 03 14:51:01.000091 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:01.000052    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413038800"
Jul 03 14:51:01.061385 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:01.061349    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d833b21-d911-4fe1-96c0-d72ed4387b87" path="/var/lib/kubelet/pods/2d833b21-d911-4fe1-96c0-d72ed4387b87/volumes"
Jul 03 14:51:01.135115 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:01.135090    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7123-139783413038800"]
Jul 03 14:51:01.137517 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:51:01.137488    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod671c602a_3ff7_45e9_b765_927a8ecbbebb.slice/crio-96a2413e6a726e1d6269e613bfd05e1d2b7d366421e1b11bbaa2f1c60d0d1b61 WatchSource:0}: Error finding container 96a2413e6a726e1d6269e613bfd05e1d2b7d366421e1b11bbaa2f1c60d0d1b61: Status 404 returned error can't find the container with id 96a2413e6a726e1d6269e613bfd05e1d2b7d366421e1b11bbaa2f1c60d0d1b61
Jul 03 14:51:01.390783 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:01.390742    2563 scope.go:117] "RemoveContainer" containerID="0553e5dbd7a974e20691f20f7e5ebde9c860e05b0b437073abf34ff12d7b1e9f"
Jul 03 14:51:01.390783 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:01.390772    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413038800"
Jul 03 14:51:01.392096 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:01.392069    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7123-139783413038800" event={"ID":"671c602a-3ff7-45e9-b765-927a8ecbbebb","Type":"ContainerStarted","Data":"96a2413e6a726e1d6269e613bfd05e1d2b7d366421e1b11bbaa2f1c60d0d1b61"}
Jul 03 14:51:02.399012 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:02.398964    2563 generic.go:356] "Generic (PLEG): container finished" podID="671c602a-3ff7-45e9-b765-927a8ecbbebb" containerID="7548281da39d8aa53b4724cba5b7fc3c48be5485a7f62d9956673d9510e80d76" exitCode=0
Jul 03 14:51:02.399566 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:02.399068    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7123-139783413038800" event={"ID":"671c602a-3ff7-45e9-b765-927a8ecbbebb","Type":"ContainerDied","Data":"7548281da39d8aa53b4724cba5b7fc3c48be5485a7f62d9956673d9510e80d76"}
Jul 03 14:51:03.555781 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:03.555751    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413038800"
Jul 03 14:51:03.574695 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:03.574665    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7123-139783413038800_671c602a-3ff7-45e9-b765-927a8ecbbebb/test-curl-7123-139783413038800/0.log"
Jul 03 14:51:03.597618 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:03.597563    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7123-139783413038800"]
Jul 03 14:51:03.599627 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:03.599599    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7123-139783413038800"]
Jul 03 14:51:03.658810 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:03.658716    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zv6gz\" (UniqueName: \"kubernetes.io/projected/671c602a-3ff7-45e9-b765-927a8ecbbebb-kube-api-access-zv6gz\") pod \"671c602a-3ff7-45e9-b765-927a8ecbbebb\" (UID: \"671c602a-3ff7-45e9-b765-927a8ecbbebb\") "
Jul 03 14:51:03.660812 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:03.660777    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/671c602a-3ff7-45e9-b765-927a8ecbbebb-kube-api-access-zv6gz" (OuterVolumeSpecName: "kube-api-access-zv6gz") pod "671c602a-3ff7-45e9-b765-927a8ecbbebb" (UID: "671c602a-3ff7-45e9-b765-927a8ecbbebb"). InnerVolumeSpecName "kube-api-access-zv6gz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:51:03.723675 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:03.723638    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7123-139783413037360"]
Jul 03 14:51:03.724044 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:03.724030    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="671c602a-3ff7-45e9-b765-927a8ecbbebb" containerName="test-curl-7123-139783413038800"
Jul 03 14:51:03.724101 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:03.724046    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="671c602a-3ff7-45e9-b765-927a8ecbbebb" containerName="test-curl-7123-139783413038800"
Jul 03 14:51:03.724136 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:03.724125    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="671c602a-3ff7-45e9-b765-927a8ecbbebb" containerName="test-curl-7123-139783413038800"
Jul 03 14:51:03.727138 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:03.727121    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413037360"
Jul 03 14:51:03.733884 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:03.733856    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7123-139783413037360"]
Jul 03 14:51:03.760131 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:03.760097    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zv6gz\" (UniqueName: \"kubernetes.io/projected/671c602a-3ff7-45e9-b765-927a8ecbbebb-kube-api-access-zv6gz\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:51:03.860578 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:03.860542    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5g85\" (UniqueName: \"kubernetes.io/projected/c0062590-15b4-40b7-b38b-438972d58ebf-kube-api-access-c5g85\") pod \"test-curl-7123-139783413037360\" (UID: \"c0062590-15b4-40b7-b38b-438972d58ebf\") " pod="opendatahub/test-curl-7123-139783413037360"
Jul 03 14:51:03.962063 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:03.961929    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5g85\" (UniqueName: \"kubernetes.io/projected/c0062590-15b4-40b7-b38b-438972d58ebf-kube-api-access-c5g85\") pod \"test-curl-7123-139783413037360\" (UID: \"c0062590-15b4-40b7-b38b-438972d58ebf\") " pod="opendatahub/test-curl-7123-139783413037360"
Jul 03 14:51:03.969953 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:03.969921    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5g85\" (UniqueName: \"kubernetes.io/projected/c0062590-15b4-40b7-b38b-438972d58ebf-kube-api-access-c5g85\") pod \"test-curl-7123-139783413037360\" (UID: \"c0062590-15b4-40b7-b38b-438972d58ebf\") " pod="opendatahub/test-curl-7123-139783413037360"
Jul 03 14:51:04.037189 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:04.037144    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413037360"
Jul 03 14:51:04.170211 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:04.170178    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7123-139783413037360"]
Jul 03 14:51:04.172856 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:51:04.172826    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0062590_15b4_40b7_b38b_438972d58ebf.slice/crio-2343ea23d5248f7f9ea6bc0f9200a4b086260fb91b758975163aa5837e512000 WatchSource:0}: Error finding container 2343ea23d5248f7f9ea6bc0f9200a4b086260fb91b758975163aa5837e512000: Status 404 returned error can't find the container with id 2343ea23d5248f7f9ea6bc0f9200a4b086260fb91b758975163aa5837e512000
Jul 03 14:51:04.412367 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:04.412327    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7123-139783413037360" event={"ID":"c0062590-15b4-40b7-b38b-438972d58ebf","Type":"ContainerStarted","Data":"2343ea23d5248f7f9ea6bc0f9200a4b086260fb91b758975163aa5837e512000"}
Jul 03 14:51:04.413501 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:04.413483    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413038800"
Jul 03 14:51:04.413618 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:04.413514    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="96a2413e6a726e1d6269e613bfd05e1d2b7d366421e1b11bbaa2f1c60d0d1b61"
Jul 03 14:51:04.423932 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:51:04.423900    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7123-139783413038800\" is forbidden: User \"system:node:ip-10-0-142-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-233.ec2.internal' and this object" podUID="671c602a-3ff7-45e9-b765-927a8ecbbebb" pod="opendatahub/test-curl-7123-139783413038800"
Jul 03 14:51:05.059356 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:05.059321    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="671c602a-3ff7-45e9-b765-927a8ecbbebb" path="/var/lib/kubelet/pods/671c602a-3ff7-45e9-b765-927a8ecbbebb/volumes"
Jul 03 14:51:05.421093 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:05.421054    2563 generic.go:356] "Generic (PLEG): container finished" podID="c0062590-15b4-40b7-b38b-438972d58ebf" containerID="e5751562be6327b2a3305dcff3b547c38ddcdb9f1660ab50334d3f5386345430" exitCode=0
Jul 03 14:51:05.421273 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:05.421145    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7123-139783413037360" event={"ID":"c0062590-15b4-40b7-b38b-438972d58ebf","Type":"ContainerDied","Data":"e5751562be6327b2a3305dcff3b547c38ddcdb9f1660ab50334d3f5386345430"}
Jul 03 14:51:06.569143 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:06.569117    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413037360"
Jul 03 14:51:06.590364 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:06.590298    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7123-139783413037360_c0062590-15b4-40b7-b38b-438972d58ebf/test-curl-7123-139783413037360/0.log"
Jul 03 14:51:06.616196 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:06.616164    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7123-139783413037360"]
Jul 03 14:51:06.621328 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:06.621299    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7123-139783413037360"]
Jul 03 14:51:06.687259 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:06.687174    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c5g85\" (UniqueName: \"kubernetes.io/projected/c0062590-15b4-40b7-b38b-438972d58ebf-kube-api-access-c5g85\") pod \"c0062590-15b4-40b7-b38b-438972d58ebf\" (UID: \"c0062590-15b4-40b7-b38b-438972d58ebf\") "
Jul 03 14:51:06.689211 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:06.689179    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0062590-15b4-40b7-b38b-438972d58ebf-kube-api-access-c5g85" (OuterVolumeSpecName: "kube-api-access-c5g85") pod "c0062590-15b4-40b7-b38b-438972d58ebf" (UID: "c0062590-15b4-40b7-b38b-438972d58ebf"). InnerVolumeSpecName "kube-api-access-c5g85". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:51:06.740931 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:06.740898    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7123-139783415427408"]
Jul 03 14:51:06.741312 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:06.741298    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0062590-15b4-40b7-b38b-438972d58ebf" containerName="test-curl-7123-139783413037360"
Jul 03 14:51:06.741374 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:06.741313    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0062590-15b4-40b7-b38b-438972d58ebf" containerName="test-curl-7123-139783413037360"
Jul 03 14:51:06.741414 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:06.741374    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0062590-15b4-40b7-b38b-438972d58ebf" containerName="test-curl-7123-139783413037360"
Jul 03 14:51:06.745177 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:06.744689    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783415427408"
Jul 03 14:51:06.750715 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:06.750685    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7123-139783415427408"]
Jul 03 14:51:06.788912 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:06.788877    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5g85\" (UniqueName: \"kubernetes.io/projected/c0062590-15b4-40b7-b38b-438972d58ebf-kube-api-access-c5g85\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:51:06.889397 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:06.889357    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtkhv\" (UniqueName: \"kubernetes.io/projected/17ac624c-324e-4f08-92a8-22873222adb7-kube-api-access-gtkhv\") pod \"test-curl-7123-139783415427408\" (UID: \"17ac624c-324e-4f08-92a8-22873222adb7\") " pod="opendatahub/test-curl-7123-139783415427408"
Jul 03 14:51:06.990348 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:06.990265    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtkhv\" (UniqueName: \"kubernetes.io/projected/17ac624c-324e-4f08-92a8-22873222adb7-kube-api-access-gtkhv\") pod \"test-curl-7123-139783415427408\" (UID: \"17ac624c-324e-4f08-92a8-22873222adb7\") " pod="opendatahub/test-curl-7123-139783415427408"
Jul 03 14:51:06.999610 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:06.999581    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtkhv\" (UniqueName: \"kubernetes.io/projected/17ac624c-324e-4f08-92a8-22873222adb7-kube-api-access-gtkhv\") pod \"test-curl-7123-139783415427408\" (UID: \"17ac624c-324e-4f08-92a8-22873222adb7\") " pod="opendatahub/test-curl-7123-139783415427408"
Jul 03 14:51:07.054709 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:07.054677    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783415427408"
Jul 03 14:51:07.061547 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:07.061515    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0062590-15b4-40b7-b38b-438972d58ebf" path="/var/lib/kubelet/pods/c0062590-15b4-40b7-b38b-438972d58ebf/volumes"
Jul 03 14:51:07.192525 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:07.192495    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7123-139783415427408"]
Jul 03 14:51:07.194614 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:51:07.194582    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17ac624c_324e_4f08_92a8_22873222adb7.slice/crio-a365c15d82b48904071023469088610ba71a410240a6b7e698ee5ad029b474a7 WatchSource:0}: Error finding container a365c15d82b48904071023469088610ba71a410240a6b7e698ee5ad029b474a7: Status 404 returned error can't find the container with id a365c15d82b48904071023469088610ba71a410240a6b7e698ee5ad029b474a7
Jul 03 14:51:07.433677 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:07.433651    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783413037360"
Jul 03 14:51:07.433677 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:07.433650    2563 scope.go:117] "RemoveContainer" containerID="e5751562be6327b2a3305dcff3b547c38ddcdb9f1660ab50334d3f5386345430"
Jul 03 14:51:07.434907 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:07.434871    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7123-139783415427408" event={"ID":"17ac624c-324e-4f08-92a8-22873222adb7","Type":"ContainerStarted","Data":"a365c15d82b48904071023469088610ba71a410240a6b7e698ee5ad029b474a7"}
Jul 03 14:51:08.442042 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:08.442005    2563 generic.go:356] "Generic (PLEG): container finished" podID="17ac624c-324e-4f08-92a8-22873222adb7" containerID="094284a70d3ab28c83dfdc55d6cbb6fe2f70bcf0842264106eb3cf3ac9406a29" exitCode=0
Jul 03 14:51:08.442541 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:08.442101    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7123-139783415427408" event={"ID":"17ac624c-324e-4f08-92a8-22873222adb7","Type":"ContainerDied","Data":"094284a70d3ab28c83dfdc55d6cbb6fe2f70bcf0842264106eb3cf3ac9406a29"}
Jul 03 14:51:09.581558 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:09.581528    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783415427408"
Jul 03 14:51:09.601006 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:09.600980    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7123-139783415427408_17ac624c-324e-4f08-92a8-22873222adb7/test-curl-7123-139783415427408/0.log"
Jul 03 14:51:09.624746 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:09.624716    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7123-139783415427408"]
Jul 03 14:51:09.627911 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:09.627884    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7123-139783415427408"]
Jul 03 14:51:09.715026 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:09.714915    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtkhv\" (UniqueName: \"kubernetes.io/projected/17ac624c-324e-4f08-92a8-22873222adb7-kube-api-access-gtkhv\") pod \"17ac624c-324e-4f08-92a8-22873222adb7\" (UID: \"17ac624c-324e-4f08-92a8-22873222adb7\") "
Jul 03 14:51:09.717040 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:09.717009    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17ac624c-324e-4f08-92a8-22873222adb7-kube-api-access-gtkhv" (OuterVolumeSpecName: "kube-api-access-gtkhv") pod "17ac624c-324e-4f08-92a8-22873222adb7" (UID: "17ac624c-324e-4f08-92a8-22873222adb7"). InnerVolumeSpecName "kube-api-access-gtkhv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 03 14:51:09.816828 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:09.816790    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtkhv\" (UniqueName: \"kubernetes.io/projected/17ac624c-324e-4f08-92a8-22873222adb7-kube-api-access-gtkhv\") on node \"ip-10-0-142-233.ec2.internal\" DevicePath \"\""
Jul 03 14:51:10.453582 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:10.453557    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a365c15d82b48904071023469088610ba71a410240a6b7e698ee5ad029b474a7"
Jul 03 14:51:10.453582 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:10.453565    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7123-139783415427408"
Jul 03 14:51:10.463311 ip-10-0-142-233 kubenswrapper[2563]: E0703 14:51:10.463287    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7123-139783415427408\" is forbidden: User \"system:node:ip-10-0-142-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-233.ec2.internal' and this object" podUID="17ac624c-324e-4f08-92a8-22873222adb7" pod="opendatahub/test-curl-7123-139783415427408"
Jul 03 14:51:10.516127 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:10.516090    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-5f1e04-29718165-7cm2n"]
Jul 03 14:51:10.518409 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:10.518380    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-5f1e04-29718165-7cm2n"]
Jul 03 14:51:11.059277 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:11.059244    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17ac624c-324e-4f08-92a8-22873222adb7" path="/var/lib/kubelet/pods/17ac624c-324e-4f08-92a8-22873222adb7/volumes"
Jul 03 14:51:11.059662 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:11.059570    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbc78d57-b903-480e-8f86-801352bff8fa" path="/var/lib/kubelet/pods/cbc78d57-b903-480e-8f86-801352bff8fa/volumes"
Jul 03 14:51:13.939178 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:13.939141    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:51:26.338112 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:26.338077    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xrsrt"]
Jul 03 14:51:30.046204 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:30.046129    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-977f7bbf6-j5w8d_06a8df1e-5f9f-4593-9204-61af185ad4ff/copy-manifests/0.log"
Jul 03 14:51:30.055540 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:30.055516    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-977f7bbf6-j5w8d_06a8df1e-5f9f-4593-9204-61af185ad4ff/manager/0.log"
Jul 03 14:51:31.810498 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:31.810467    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-8qm8w_673f8290-39c8-41f1-afbf-9da9db2f895d/kuadrant-console-plugin/0.log"
Jul 03 14:51:32.166362 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:32.166334    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-xrsrt_b29859bb-2b3d-4218-92a3-1329e905803a/limitador/0.log"
Jul 03 14:51:32.976177 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:32.976147    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-557657c976-htz8w_d31500b1-8a7c-4d74-b3b0-5b9090730c00/kube-auth-proxy/0.log"
Jul 03 14:51:33.099556 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:33.099529    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-8ksxz_14e864ac-ec94-4ca1-8496-4d1fc83fb11d/istio-proxy/1.log"
Jul 03 14:51:33.985731 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:33.985702    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-xzg2f_4210ce8e-6505-4f3f-b726-ec0e2292e845/main/0.log"
Jul 03 14:51:34.318943 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:34.318862    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-64nd5_f6281067-c0ac-44ee-af48-34cbea8597c2/main/0.log"
Jul 03 14:51:40.757775 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:40.757747    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rcksp_01b1d786-bec1-48da-ace5-ec49bda1a3e3/global-pull-secret-syncer/0.log"
Jul 03 14:51:40.971487 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:40.971455    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-tw77w_a5e9d1f2-6f21-48c0-8046-9f95a5218539/konnectivity-agent/0.log"
Jul 03 14:51:41.042785 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:41.042699    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-233.ec2.internal_61e1c42ac9a08ecfb3c8cc5e93139196/haproxy/0.log"
Jul 03 14:51:46.365880 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:46.365851    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-8qm8w_673f8290-39c8-41f1-afbf-9da9db2f895d/kuadrant-console-plugin/0.log"
Jul 03 14:51:46.597919 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:46.597888    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-xrsrt_b29859bb-2b3d-4218-92a3-1329e905803a/limitador/0.log"
Jul 03 14:51:48.402775 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:48.402747    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_38e7636c-38ae-49dd-8ecd-57d50095e31e/alertmanager/0.log"
Jul 03 14:51:48.429941 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:48.429917    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_38e7636c-38ae-49dd-8ecd-57d50095e31e/config-reloader/0.log"
Jul 03 14:51:48.455353 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:48.455328    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_38e7636c-38ae-49dd-8ecd-57d50095e31e/kube-rbac-proxy-web/0.log"
Jul 03 14:51:48.479369 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:48.479324    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_38e7636c-38ae-49dd-8ecd-57d50095e31e/kube-rbac-proxy/0.log"
Jul 03 14:51:48.522951 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:48.522919    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_38e7636c-38ae-49dd-8ecd-57d50095e31e/kube-rbac-proxy-metric/0.log"
Jul 03 14:51:48.569795 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:48.569771    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_38e7636c-38ae-49dd-8ecd-57d50095e31e/prom-label-proxy/0.log"
Jul 03 14:51:48.592737 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:48.592711    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_38e7636c-38ae-49dd-8ecd-57d50095e31e/init-config-reloader/0.log"
Jul 03 14:51:48.632514 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:48.632489    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-88rx9_d85062a4-19c6-4cc0-9488-19b48622c627/cluster-monitoring-operator/0.log"
Jul 03 14:51:48.744862 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:48.744783    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-649b8755c8-dfnkp_d3a6ec3b-1a67-4151-a95c-eb00d41401f4/metrics-server/0.log"
Jul 03 14:51:48.780210 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:48.780182    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-rv5xq_3a2a55bf-9ea3-4b36-9a2d-81d9b8877039/monitoring-plugin/0.log"
Jul 03 14:51:48.896288 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:48.896259    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b59zs_79e2d165-3d12-4c84-804b-4d7c857419ef/node-exporter/0.log"
Jul 03 14:51:48.916931 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:48.916891    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b59zs_79e2d165-3d12-4c84-804b-4d7c857419ef/kube-rbac-proxy/0.log"
Jul 03 14:51:48.939037 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:48.939013    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b59zs_79e2d165-3d12-4c84-804b-4d7c857419ef/init-textfile/0.log"
Jul 03 14:51:49.052439 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:49.052407    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-d77cl_af196fa9-9734-4b49-a5cd-43c50cf3e54e/kube-rbac-proxy-main/0.log"
Jul 03 14:51:49.074241 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:49.074214    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-d77cl_af196fa9-9734-4b49-a5cd-43c50cf3e54e/kube-rbac-proxy-self/0.log"
Jul 03 14:51:49.094750 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:49.094726    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-d77cl_af196fa9-9734-4b49-a5cd-43c50cf3e54e/openshift-state-metrics/0.log"
Jul 03 14:51:49.131609 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:49.131580    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8b71262a-e44f-4bba-accf-896149be23ae/prometheus/0.log"
Jul 03 14:51:49.151655 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:49.151630    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8b71262a-e44f-4bba-accf-896149be23ae/config-reloader/0.log"
Jul 03 14:51:49.173932 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:49.173910    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8b71262a-e44f-4bba-accf-896149be23ae/thanos-sidecar/0.log"
Jul 03 14:51:49.195046 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:49.195022    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8b71262a-e44f-4bba-accf-896149be23ae/kube-rbac-proxy-web/0.log"
Jul 03 14:51:49.222425 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:49.222404    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8b71262a-e44f-4bba-accf-896149be23ae/kube-rbac-proxy/0.log"
Jul 03 14:51:49.246321 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:49.246296    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8b71262a-e44f-4bba-accf-896149be23ae/kube-rbac-proxy-thanos/0.log"
Jul 03 14:51:49.267370 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:49.267302    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8b71262a-e44f-4bba-accf-896149be23ae/init-config-reloader/0.log"
Jul 03 14:51:49.336852 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:49.336827    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-77xcg_815607d1-89de-4e62-9a6f-301217b897ef/prometheus-operator-admission-webhook/0.log"
Jul 03 14:51:49.363661 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:49.363636    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7cd5b956b9-r25ff_9d6eca64-8303-4ad4-ae80-197d0983d553/telemeter-client/0.log"
Jul 03 14:51:49.382512 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:49.382485    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7cd5b956b9-r25ff_9d6eca64-8303-4ad4-ae80-197d0983d553/reload/0.log"
Jul 03 14:51:49.402284 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:49.402257    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7cd5b956b9-r25ff_9d6eca64-8303-4ad4-ae80-197d0983d553/kube-rbac-proxy/0.log"
Jul 03 14:51:49.428877 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:49.428854    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b8865b898-vrxv9_140d5eb5-547f-4490-aa12-f34472b8a472/thanos-query/0.log"
Jul 03 14:51:49.451833 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:49.451809    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b8865b898-vrxv9_140d5eb5-547f-4490-aa12-f34472b8a472/kube-rbac-proxy-web/0.log"
Jul 03 14:51:49.472521 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:49.472501    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b8865b898-vrxv9_140d5eb5-547f-4490-aa12-f34472b8a472/kube-rbac-proxy/0.log"
Jul 03 14:51:49.492468 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:49.492443    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b8865b898-vrxv9_140d5eb5-547f-4490-aa12-f34472b8a472/prom-label-proxy/0.log"
Jul 03 14:51:49.513281 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:49.513256    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b8865b898-vrxv9_140d5eb5-547f-4490-aa12-f34472b8a472/kube-rbac-proxy-rules/0.log"
Jul 03 14:51:49.537228 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:49.537163    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b8865b898-vrxv9_140d5eb5-547f-4490-aa12-f34472b8a472/kube-rbac-proxy-metrics/0.log"
Jul 03 14:51:50.666986 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:50.666950    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-fc745_e3af58c5-0172-407c-814e-1169456d4bda/networking-console-plugin/0.log"
Jul 03 14:51:51.118732 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:51.118699    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-s2kmc/perf-node-gather-daemonset-45pdn"]
Jul 03 14:51:51.119115 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:51.119100    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17ac624c-324e-4f08-92a8-22873222adb7" containerName="test-curl-7123-139783415427408"
Jul 03 14:51:51.119177 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:51.119126    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="17ac624c-324e-4f08-92a8-22873222adb7" containerName="test-curl-7123-139783415427408"
Jul 03 14:51:51.119214 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:51.119190    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="17ac624c-324e-4f08-92a8-22873222adb7" containerName="test-curl-7123-139783415427408"
Jul 03 14:51:51.122714 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:51.122692    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-45pdn"
Jul 03 14:51:51.125051 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:51.125031    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s2kmc\"/\"openshift-service-ca.crt\""
Jul 03 14:51:51.125153 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:51.125134    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s2kmc\"/\"kube-root-ca.crt\""
Jul 03 14:51:51.125728 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:51.125713    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-s2kmc\"/\"default-dockercfg-fgmwv\""
Jul 03 14:51:51.129562 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:51.129540    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s2kmc/perf-node-gather-daemonset-45pdn"]
Jul 03 14:51:51.302560 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:51.302511    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/84b40713-5895-4ba8-b39d-9c8d9d7bed95-proc\") pod \"perf-node-gather-daemonset-45pdn\" (UID: \"84b40713-5895-4ba8-b39d-9c8d9d7bed95\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-45pdn"
Jul 03 14:51:51.302756 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:51.302583    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/84b40713-5895-4ba8-b39d-9c8d9d7bed95-sys\") pod \"perf-node-gather-daemonset-45pdn\" (UID: \"84b40713-5895-4ba8-b39d-9c8d9d7bed95\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-45pdn"
Jul 03 14:51:51.302756 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:51.302663    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/84b40713-5895-4ba8-b39d-9c8d9d7bed95-lib-modules\") pod \"perf-node-gather-daemonset-45pdn\" (UID: \"84b40713-5895-4ba8-b39d-9c8d9d7bed95\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-45pdn"
Jul 03 14:51:51.302756 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:51.302700    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k69fd\" (UniqueName: \"kubernetes.io/projected/84b40713-5895-4ba8-b39d-9c8d9d7bed95-kube-api-access-k69fd\") pod \"perf-node-gather-daemonset-45pdn\" (UID: \"84b40713-5895-4ba8-b39d-9c8d9d7bed95\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-45pdn"
Jul 03 14:51:51.302870 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:51.302765    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/84b40713-5895-4ba8-b39d-9c8d9d7bed95-podres\") pod \"perf-node-gather-daemonset-45pdn\" (UID: \"84b40713-5895-4ba8-b39d-9c8d9d7bed95\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-45pdn"
Jul 03 14:51:51.403935 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:51.403798    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/84b40713-5895-4ba8-b39d-9c8d9d7bed95-proc\") pod \"perf-node-gather-daemonset-45pdn\" (UID: \"84b40713-5895-4ba8-b39d-9c8d9d7bed95\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-45pdn"
Jul 03 14:51:51.404227 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:51.404197    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/84b40713-5895-4ba8-b39d-9c8d9d7bed95-sys\") pod \"perf-node-gather-daemonset-45pdn\" (UID: \"84b40713-5895-4ba8-b39d-9c8d9d7bed95\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-45pdn"
Jul 03 14:51:51.404428 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:51.404413    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/84b40713-5895-4ba8-b39d-9c8d9d7bed95-lib-modules\") pod \"perf-node-gather-daemonset-45pdn\" (UID: \"84b40713-5895-4ba8-b39d-9c8d9d7bed95\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-45pdn"
Jul 03 14:51:51.404563 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:51.404548    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k69fd\" (UniqueName: \"kubernetes.io/projected/84b40713-5895-4ba8-b39d-9c8d9d7bed95-kube-api-access-k69fd\") pod \"perf-node-gather-daemonset-45pdn\" (UID: \"84b40713-5895-4ba8-b39d-9c8d9d7bed95\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-45pdn"
Jul 03 14:51:51.404691 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:51.404679    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/84b40713-5895-4ba8-b39d-9c8d9d7bed95-podres\") pod \"perf-node-gather-daemonset-45pdn\" (UID: \"84b40713-5895-4ba8-b39d-9c8d9d7bed95\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-45pdn"
Jul 03 14:51:51.405012 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:51.404958    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/84b40713-5895-4ba8-b39d-9c8d9d7bed95-podres\") pod \"perf-node-gather-daemonset-45pdn\" (UID: \"84b40713-5895-4ba8-b39d-9c8d9d7bed95\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-45pdn"
Jul 03 14:51:51.405137 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:51.403884    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/84b40713-5895-4ba8-b39d-9c8d9d7bed95-proc\") pod \"perf-node-gather-daemonset-45pdn\" (UID: \"84b40713-5895-4ba8-b39d-9c8d9d7bed95\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-45pdn"
Jul 03 14:51:51.405291 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:51.405277    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/84b40713-5895-4ba8-b39d-9c8d9d7bed95-sys\") pod \"perf-node-gather-daemonset-45pdn\" (UID: \"84b40713-5895-4ba8-b39d-9c8d9d7bed95\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-45pdn"
Jul 03 14:51:51.405478 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:51.405462    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/84b40713-5895-4ba8-b39d-9c8d9d7bed95-lib-modules\") pod \"perf-node-gather-daemonset-45pdn\" (UID: \"84b40713-5895-4ba8-b39d-9c8d9d7bed95\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-45pdn"
Jul 03 14:51:51.414401 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:51.414377    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k69fd\" (UniqueName: \"kubernetes.io/projected/84b40713-5895-4ba8-b39d-9c8d9d7bed95-kube-api-access-k69fd\") pod \"perf-node-gather-daemonset-45pdn\" (UID: \"84b40713-5895-4ba8-b39d-9c8d9d7bed95\") " pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-45pdn"
Jul 03 14:51:51.434483 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:51.434455    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-45pdn"
Jul 03 14:51:51.572428 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:51.572405    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s2kmc/perf-node-gather-daemonset-45pdn"]
Jul 03 14:51:51.574663 ip-10-0-142-233 kubenswrapper[2563]: W0703 14:51:51.574636    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod84b40713_5895_4ba8_b39d_9c8d9d7bed95.slice/crio-ba138afb64820911d83d0532978bda69cb374b848bfadeca98230719c8f6a05e WatchSource:0}: Error finding container ba138afb64820911d83d0532978bda69cb374b848bfadeca98230719c8f6a05e: Status 404 returned error can't find the container with id ba138afb64820911d83d0532978bda69cb374b848bfadeca98230719c8f6a05e
Jul 03 14:51:51.663421 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:51.663333    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-45pdn" event={"ID":"84b40713-5895-4ba8-b39d-9c8d9d7bed95","Type":"ContainerStarted","Data":"ba138afb64820911d83d0532978bda69cb374b848bfadeca98230719c8f6a05e"}
Jul 03 14:51:51.873469 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:51.873438    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-s7j5g_ee8e766e-8e65-4d2d-abc5-7b7a0dc21b33/download-server/0.log"
Jul 03 14:51:52.669898 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:52.669857    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-45pdn" event={"ID":"84b40713-5895-4ba8-b39d-9c8d9d7bed95","Type":"ContainerStarted","Data":"c1e35540fb451f858454c874cc9e35b17f654fa6dc537ec3644bd38ddeb90e12"}
Jul 03 14:51:52.670105 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:52.669998    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-45pdn"
Jul 03 14:51:52.700911 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:52.700860    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-45pdn" podStartSLOduration=1.700846933 podStartE2EDuration="1.700846933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 14:51:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 14:51:52.700409183 +0000 UTC m=+2626.150542655" watchObservedRunningTime="2026-07-03 14:51:52.700846933 +0000 UTC m=+2626.150980372"
Jul 03 14:51:53.536653 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:53.536603    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bf7t2_e58a995c-7842-4473-b890-7c93a46980f3/dns/0.log"
Jul 03 14:51:53.558451 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:53.558425    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bf7t2_e58a995c-7842-4473-b890-7c93a46980f3/kube-rbac-proxy/0.log"
Jul 03 14:51:53.687115 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:53.687077    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9k9sd_11f255a7-6652-4c64-97b8-c980552c363f/dns-node-resolver/0.log"
Jul 03 14:51:54.263692 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:54.263658    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7b795_2df513c5-3c49-4255-9106-9e36d40d4fa1/node-ca/0.log"
Jul 03 14:51:55.487854 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:55.487824    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-557657c976-htz8w_d31500b1-8a7c-4d74-b3b0-5b9090730c00/kube-auth-proxy/0.log"
Jul 03 14:51:55.518900 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:55.518866    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-8ksxz_14e864ac-ec94-4ca1-8496-4d1fc83fb11d/istio-proxy/0.log"
Jul 03 14:51:55.544781 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:55.544751    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-8ksxz_14e864ac-ec94-4ca1-8496-4d1fc83fb11d/istio-proxy/1.log"
Jul 03 14:51:56.387278 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:56.387241    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-srkff_620719fe-f6f2-4310-9f6e-0f351f68264c/serve-healthcheck-canary/0.log"
Jul 03 14:51:57.032238 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:57.032210    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v6t5n_f499c073-70d1-4178-8c52-1e64babb8a37/kube-rbac-proxy/0.log"
Jul 03 14:51:57.051294 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:57.051271    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v6t5n_f499c073-70d1-4178-8c52-1e64babb8a37/exporter/0.log"
Jul 03 14:51:57.071689 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:57.071657    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v6t5n_f499c073-70d1-4178-8c52-1e64babb8a37/extractor/0.log"
Jul 03 14:51:58.685086 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:58.685058    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-s2kmc/perf-node-gather-daemonset-45pdn"
Jul 03 14:51:59.923521 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:59.923494    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-977f7bbf6-j5w8d_06a8df1e-5f9f-4593-9204-61af185ad4ff/manager/0.log"
Jul 03 14:51:59.949427 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:51:59.949396    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-977f7bbf6-j5w8d_06a8df1e-5f9f-4593-9204-61af185ad4ff/copy-manifests/0.log"
Jul 03 14:52:01.373397 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:52:01.373369    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-557657c976-htz8w_d31500b1-8a7c-4d74-b3b0-5b9090730c00/kube-auth-proxy/0.log"
Jul 03 14:52:01.393221 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:52:01.393188    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-8ksxz_14e864ac-ec94-4ca1-8496-4d1fc83fb11d/istio-proxy/0.log"
Jul 03 14:52:01.420288 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:52:01.420253    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-8ksxz_14e864ac-ec94-4ca1-8496-4d1fc83fb11d/istio-proxy/1.log"
Jul 03 14:52:01.588174 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:52:01.588093    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-78cbbf6689-pgfl6_6fc5b6b3-ed32-44fd-8d81-1bc1c035e90d/manager/0.log"
Jul 03 14:52:03.015397 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:52:03.015309    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-557657c976-htz8w_d31500b1-8a7c-4d74-b3b0-5b9090730c00/kube-auth-proxy/0.log"
Jul 03 14:52:03.052661 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:52:03.052600    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-8ksxz_14e864ac-ec94-4ca1-8496-4d1fc83fb11d/istio-proxy/0.log"
Jul 03 14:52:03.080511 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:52:03.080478    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-8ksxz_14e864ac-ec94-4ca1-8496-4d1fc83fb11d/istio-proxy/1.log"
Jul 03 14:52:07.029019 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:52:07.028965    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-l9qcj_8a71b7ee-121f-474e-a3b6-dbbbaa61daa9/kube-storage-version-migrator-operator/1.log"
Jul 03 14:52:07.029845 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:52:07.029826    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-l9qcj_8a71b7ee-121f-474e-a3b6-dbbbaa61daa9/kube-storage-version-migrator-operator/0.log"
Jul 03 14:52:07.353116 ip-10-0-142-233 kubenswrapper[2563]: I0703 14:52:07.353087    2563 scope.go:117] "RemoveContainer" containerID="0edfbc7566626f3cf2c0153c83d5be4f61523af1d385285e930b6cfdc05a8205"